Difference between revisions of "Latest Luau"

From GiderosMobile
 
(4 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
__NOTOC__
 
__NOTOC__
 +
'''Luau getting started: https://luau.org/getting-started'''
  
−
'''from https://luau-lang.org/news/'''
+
 
 +
'''Luau below recaps from: https://luau.org/news/'''
  
 
----
 
----
 +
 +
Luau Recap: July 2024
 +
 +
July 23, 2024
 +
 +
Hello everyone!
 +
 +
While the Luau team is actively working on a big rewrite of the type inference and type checking engines (more news about that in the near future), we wanted to go over other changes and updates since our last recap back in October.
 +
Official Luau mascotPermalink
 +
 +
Luau has recently adopted a Hawaiian monk seal mascot named Hina, after the Hawaiian goddess of the moon.
 +
 +
Please welcome Hina the Seal!
 +
 +
Hina the Seal
 +
Native Code GenerationPermalink
 +
 +
We are happy to announce that the native code feature is out from the ‘Preview’ state and is fully supported for X64 (Intel/AMD) or A64 (ARM) processor architectures.
 +
 +
As a refresher, native code generation is the feature that allows Luau scripts that have been previously executed by interpreting bytecode inside the Luau VM to instead compile to machine code that the CPU understands and executes directly.
 +
 +
Since the release of the Preview, we have worked on improving code performance, memory use of the system, correctness and stability.
 +
 +
Some highlights:
 +
 +
    Improved performance of the bit32 library functions
 +
    Improved performance of numerical loops
 +
    Optimized table array and property lookups
 +
    Added native support for new buffer type operations
 +
    Code optimizations based on knowing which types are returned from operations
 +
    Code optimizations based on function argument type annotations
 +
        This includes support for SIMD operations for annotated vector arguments
 +
 +
There are many other small improvements in generated code performance and size and we have plans for additional optimizations.
 +
Native function attributePermalink
 +
 +
For a better control of what code runs natively, we have introduced new syntax for function attributes:
 +
 +
<syntaxhighlight lang="lua">
 +
@native -- function compiles natively
 +
local function foo()
 +
    ...
 +
end
 +
</syntaxhighlight>
 +
 +
This is the first attribute to become available and we are working on the ability to mark functions as deprecated using the @deprecated attribute. More on that here.
 +
Type information for runtime optimizationsPermalink
 +
 +
Native code generation works on any code without having to modify it. In certain situations, this means that the native compiler cannot be sure about the types involved in the operation.
 +
 +
Consider a simple function, working on a few values:
 +
 +
<syntaxhighlight lang="lua">
 +
local function MulAddScaled(a, b, c)
 +
    return a * b * 0.75 + c * 0.25
 +
end
 +
</syntaxhighlight>
 +
 +
Native compiler assumes that operations are most likely being performed on numbers and generates the appropriate fast path.
 +
 +
But what if the function is actually called with a vector type?
 +
 +
<syntaxhighlight lang="lua">
 +
local intlPos = MulAddScaled(Part.Position, v, vector(12, 0, 0))
 +
</syntaxhighlight>
 +
 +
To handle this, a slower path was generated to handle any other potential type of the argument. Because this path is not chosen as the first possible option, extra checking overhead prevents code from running as fast as it can.
 +
 +
When we announced the last update, we had already added some support for following the types used as arguments.
 +
 +
<syntaxhighlight lang="lua">
 +
local function MulAddScaled(a: vector, b: vector, c: vector)
 +
    return a * b * 0.75 + c * 0.25
 +
end
 +
</syntaxhighlight>
 +
 +
    NOTE: vector type is not enabled by default, check out defaultOptions and setupVectorHelpers functions in Conformance.test.cpp file as an example of the vector library setup.
 +
 +
Since then, we have extended this to support type information on locals, following complex types and even inferring results of additional operations.
 +
 +
<syntaxhighlight lang="lua">
 +
type Vertex = { p: vector, uv: vector, n: vector, t: vector, b: vector, h: number }
 +
type Mesh = { vertices: {Vertex}, indices: {number} }
 +
 +
function calculate_normals(mesh: Mesh)
 +
    for i = 1,#mesh.indices,3 do
 +
        local a = mesh.vertices[mesh.indices[i]]
 +
        local b = mesh.vertices[mesh.indices[i + 1]]
 +
        local c = mesh.vertices[mesh.indices[i + 2]]
 +
 +
        local vba = a.p - b.p -- Inferred as a vector operation
 +
        local vca = a.p - c.p
 +
 +
        local n = vba:Cross(vca) -- Knows that Cross returns vector
 +
 +
        a.n += n -- Inferred as a vector operation
 +
        b.n += n
 +
        c.n += n
 +
    end
 +
end
 +
</syntaxhighlight>
 +
 +
As can be seen, often it’s enough to annotate the type of the data structure and correct fast-path vector code will be generated from that without having to specify the type of each local or temporary.
 +
 +
    NOTE: Advanced inference and operation lowering is enabled by using custom HostIrHooks callbacks. Check out ‘Vector’ test with ‘IrHooks’ option in Conformance.test.cpp and ConformanceIrHooks.h file for an example of the setup.
 +
 +
Note that support for native lowering hooks allows generation of CPU code that is multiple times faster than a generic metatable call.
 +
 +
Even when native compiler doesn’t have a specific optimization for a type, if the type can be resolved, shorter code sequences are generated and more optimizations can be made between separate operations.
 +
 +
    NOTE: HostIrHooks callbacks also enable type inference and lowering for your custom userdata types. Check out ‘NativeUserdata’ test with ‘IrHooks’ option in Conformance.test.cpp and ConformanceIrHooks.h file for an example of the setup.
 +
 +
Runtime changesPermalink
 +
Stricter utf8 library validationPermalink
 +
 +
utf8 library will now correctly validate UTF-8 and reject inputs that have surrogates. utf8.len will return nil followed by the byte offset, utf8.codepoint and utf8.codes will error. This matches how other kinds of input errors were previously handled by those functions.
 +
 +
Strings that are validated using utf8.len will now always work properly with utf8.nfcnormalize and utf8.graphemes functions. Custom per-character validation logic is no longer required to check if a string is valid under utf8 requirements.
 +
Imprecise integer number warningPermalink
 +
 +
Luau stores numbers as 64-bit floating-point values. Integer values up to 2^53 are supported, but higher numbers might experience rounding.
 +
 +
For example, both 10000000000000000 and 9223372036854775808 are larger than 2^53, but match the rounding, while 10000000000000001 gets rounded down to 10000000000000000.
 +
 +
In cases where rounding takes place, you will get a warning message. If the large value is intended and rounding can be ignored, just add “.0” to the number to remove the warning:
 +
 +
<syntaxhighlight lang="lua">
 +
local a = 10000000000000001 -- Number literal exceeded available precision and was truncated to closest representable number
 +
local b = 10000000000000001.0 -- Ok, but rounds to 10000000000000000
 +
</syntaxhighlight>
 +
 +
Leading | and & in typesPermalink
 +
 +
It is now possible to start your union and intersection types with a symbol. This can help align the type components more cleanly:
 +
 +
<syntaxhighlight lang="lua">
 +
type Options =
 +
    | { tag: "cat", laziness: number }
 +
    | { tag: "dog", happiness: number }
 +
</syntaxhighlight>
 +
 +
You can find more information and examples in the proposal
 +
Analysis ImprovementsPermalink
 +
 +
While our main focus is on a type-checking engine rewrite that is nearing completion, we have fixed some of the issues in the current one.
 +
 +
    Relational operator errors are more conservative now and generate less false positive errors
 +
    It is not an error to iterate over table properties when indexer is not part of the type
 +
    Type packs with cycles are now correctly described in error messages
 +
    Improved error message when value that is not a function is being used in a call
 +
    Fixed stability issues which caused Studio to crash
 +
    Improved performance for code bases with large number of scripts and complex types
 +
 +
Runtime ImprovementsPermalink
 +
 +
When converting numbers to strings in scientific notation, we will now skip the trailing ‘.’.
 +
 +
For example, tostring(1e+30) now outputs ‘1e+30’ instead of ‘1.e+30’. This improves compatibility with data formats like JSON. But please keep in mind that unless you are using JSON5, Luau can still output ‘inf’ and ‘nan’ numbers which might not be supported.
 +
 +
    Construction of tables with 17-32 properties or 33-64 array elements is now 30% faster.
 +
    table.concat method is now 2x faster when the separator is not used and 40% faster otherwise.
 +
    table.maxn method is now 5-14x faster.
 +
    vector constants are now stored in the constant table and avoid runtime construction.
 +
    Operations like 5/x and 5-x with any constant on the left-hand-side are now performed faster, one less minor thing to think about!
 +
    It is no longer possible to crash the server on a hang in the string library methods.
 +
 +
Luau as a supported language on GitHubPermalink
 +
 +
Lastly, if you have open-source or even private projects on GitHub which use Luau, you might be happy to learn that Luau now has official support on GitHub for .luau file extension. This includes recognizing files as using Luau programming language and having support for syntax highlighting.
 +
 +
A big thanks goes to our open source community for their generous contributions including pushing for broader Luau support:
 +
 +
    birds3345
 +
    bjornbytes
 +
    Gskartwii
 +
    jackdotink
 +
    JohnnyMorganz
 +
    khvzak
 +
    kostadinsh
 +
    mttsner
 +
    mxruben
 +
    petrihakkinen
 +
    zeux
 +
 +
Updated: July 23, 2024
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
Luau Recap: October 2023
 +
 +
November 1, 2023
 +
 +
We’re still quite busy working on some big type checking updates that we hope to talk about soon, but we have a few equally exciting updates to share in the meantime!
 +
 +
Let’s dive in!
 +
Floor DivisionPermalink
 +
 +
Luau now has a floor division operator. It is spelled //:
 +
 +
<syntaxhighlight lang="lua">
 +
local a = 10 // 3 -- a == 3
 +
a //= 2          -- a == 1
 +
</syntaxhighlight>
 +
 +
For numbers, a // b is equivalent to math.floor(a / b), and you can also overload this operator by implementing the __idiv metamethod. The syntax and semantics are borrowed from Lua 5.3 (although Lua 5.3 has an integer type while we don’t, we tried to match the behavior to be as close as possible).
 +
Native Codegen PreviewPermalink
 +
 +
We are actively working on our new native code generation module that can significantly improve the performance of compute-dense scripts by compiling them to X64 (Intel/AMD) or A64 (ARM) machine code and executing that natively. We aim to support all AArch64 hardware with the current focus being Apple Silicon (M1-M3) chips, and all Intel/AMD hardware that supports AVX1 (with no planned support for earlier systems). When the hardware does not support native code generation, any code that would be compiled as native just falls back to the interpreted execution.
 +
 +
When working with open-source releases, binaries now have native code generation support compiled in by default; you need to pass --codegen command line flag to enable it. If you use Luau as a library in a third-party application, you would need to manually link Luau.CodeGen library and call the necessary functions to compile specific modules as needed - or keep using the interpreter if you want to! If you work in Roblox Studio, we have integrated native code generation preview as a beta feature, which currently requires manual annotation of select scripts with --!native comment.
 +
 +
Our goal for the native code generation is to help reach ultimate performance for code that needs to process data very efficiently, but not necessarily to accelerate every line of code, and not to replace the interpreter. We remain committed to maximizing interpreted execution performance, as not all platforms will support native code generation, and it’s not always practical to use native code generation for large code bases because it has a larger memory impact than bytecode. We intend for this to unlock new performance opportunities for complex features and algorithms, e.g. code that spends a lot of time working with numbers and arrays, but not to dramatically change performance on UI code or code that spends a lot of its time calling Lua functions like table.sort, or external C functions (like Roblox engine APIs).
 +
 +
Importantly, native code generation does not change our behavior or correctness expectations. Code compiled natively should give the same results when it executes as non-native code (just take a little less time), and it should not result in any memory safety or sandboxing issues. If you ever notice native code giving a different result from non-native code, please submit a bug report.
 +
 +
We continue to work on many code size and performance improvements; here’s a short summary of what we’ve done in the last couple of months, and there’s more to come!
 +
 +
    Repeated access to table fields with the same object and name are now optimized (e.g. t.x = t.x + 5 is faster)
 +
    Numerical for loops are now compiled more efficiently, yielding significant speedups on hot loops
 +
    Bit operations with constants are now compiled more efficiently on X64 (for example, bit32.lshift(x, 1) is faster); this optimization was already in place for A64
 +
    Repeated access to array elements with the same object and index is now faster in certain cases
 +
    Performance of function calls has been marginally improved on X64 and A64
 +
    Fix code generation for some bit32.extract variants where we could produce incorrect results
 +
    table.insert is now faster when called with two arguments as it’s compiled directly to native code
 +
    To reduce code size, module code outside of functions is not compiled natively unless it has loops
 +
 +
Analysis ImprovementsPermalink
 +
 +
The break and continue keywords can now be used in loop bodies to refine variables. This was contributed by a community member - thank you, AmberGraceSoftware!
 +
 +
<syntaxhighlight lang="lua">
 +
function f(objects: { { value: string? } })
 +
    for _, object in objects do
 +
        if not object.value then
 +
            continue
 +
        end
 +
       
 +
        local x: string = object.value -- ok!
 +
    end
 +
end
 +
</syntaxhighlight>
 +
 +
When type information is present, we will now emit a warning when # or ipairs is used on a table that has no numeric keys or indexers. This helps avoid common bugs like using #t == 0 to check if a dictionary is empty.
 +
 +
<syntaxhighlight lang="lua">
 +
local message = { data = { 1, 2, 3 } }
 +
 +
if #message == 0 then -- Using '#' on a table without an array part is likely a bug
 +
end
 +
</syntaxhighlight>
 +
 +
Finally, some uses of getfenv/setfenv are now flagged as deprecated. We do not plan to remove support for getfenv/setfenv but we actively discourage its use as it disables many optimizations throughout the compiler, runtime, and native code generation, and interferes with type checking and linting.
 +
Autocomplete ImprovementsPermalink
 +
 +
We used to have a bug that would arise in the following situation:
 +
 +
<syntaxhighlight lang="lua">
 +
--!strict
 +
type Direction = "Left" | "Right"
 +
local dir: Direction = "Left"
 +
 +
if dir == ""| then
 +
end
 +
</syntaxhighlight>
 +
 +
(imagine the cursor is at the position of the | character in the if statement)
 +
 +
We used to suggest Left and Right even though they are not valid completions at that position. This is now fixed.
 +
 +
We’ve also added a complete suggestion for anonymous functions if one would be valid at the requested position. For example:
 +
 +
<syntaxhighlight lang="lua">
 +
local p = Instance.new('Part')
 +
p.Touched:Connect(
 +
</syntaxhighlight>
 +
 +
You will see a completion suggestion function (anonymous autofilled). Selecting that will cause the following to be inserted into your code:
 +
 +
<syntaxhighlight lang="lua">
 +
local p = Instance.new('Part')
 +
p.Touched:Connect(function(otherPart: BasePart)  end
 +
</syntaxhighlight>
 +
 +
We also fixed some confusing editor feedback in the following case:
 +
 +
<syntaxhighlight lang="lua">
 +
game:FindFirstChild(
 +
</syntaxhighlight>
 +
 +
Previously, the signature help tooltip would erroneously tell you that you needed to pass a self argument. We now correctly offer the signature FindFirstChild(name: string, recursive: boolean?): Instance
 +
Runtime ImprovementsPermalink
 +
 +
    string.format’s handling of %* and %s is now 1.5-2x faster
 +
    tonumber and tostring are now 1.5x and 2.5x faster respectively when working on primitive types
 +
    Compiler now recognizes math.pi and math.huge and performs constant folding on the expressions that involve these at -O2; for example, math.pi*2 is now free.
 +
    Compiler now optimizes if...then...else expressions into AND/OR form when possible (for example, if x then x else y now compiles as x or y)
 +
    We had a few bugs around repeat..until statements when the until condition referred to local variables defined in the loop body. These bugs have been fixed.
 +
    Fix an oversight that could lead to string.char and string.sub generating potentially unlimited amounts of garbage and exhausting all available memory.
 +
    We had a bug that could cause the compiler to unroll loops that it really shouldn’t. This could result in massive bytecode bloat. It is now fixed.
 +
 +
luau-lang on GitHubPermalink
 +
 +
If you’ve been paying attention to our GitHub projects, you may have noticed that we’ve moved luau repository to a new luau-lang GitHub organization! This is purely an organizational change but it’s helping us split a few repositories for working with documentation and RFCs and be more organized with pull requests in different areas. Make sure to update your bookmarks and star our main repository if you haven’t already!
 +
 +
Lastly, a big thanks to our open source community for their generous contributions:
 +
 +
    MagelessMayhem
 +
    cassanof
 +
    LoganDark
 +
    j-hui
 +
    xgqt
 +
    jdpatdiscord
 +
    Someon1e
 +
    AmberGraceSoftware
 +
    RadiantUwU
 +
    SamuraiCrow
 +
 +
Updated: November 1, 2023
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
Luau Recap: July 2023
 +
 +
July 28, 2023
 +
 +
Our team is still spending a lot of time working on upcoming replacement for our type inference engine as well as working on native code generation to improve runtime performance.
 +
 +
However, we also worked on unrelated improvements during this time that are summarized here.
 +
 +
[Cross-posted to the Roblox Developer Forum.]
 +
Analysis improvementsPermalink
 +
 +
Indexing table intersections using x["prop"] syntax has been fixed and no longer reports a false positive error:
 +
 +
<syntaxhighlight lang="lua">
 +
type T = { foo: string } & { bar: number }
 +
local x: T = { foo = "1", bar = 2 }
 +
 +
local y = x["bar"] -- This is no longer an error
 +
</syntaxhighlight>
 +
 +
Generic T... type is now convertible to ...any variadic parameter.
 +
 +
This solves issues people had with variadic functions and variadic argument:
 +
 +
<syntaxhighlight lang="lua">
 +
local function foo(...: any)
 +
    print(...)
 +
end
 +
 +
local function bar<T...>(...: T...)
 +
    foo(...) -- This is no longer an error
 +
end
 +
</syntaxhighlight>
 +
 +
We have also improved our general typechecking performance by ~17% and by additional ~30% in modules with complex types.
 +
 +
Other fixes include:
 +
 +
    Fixed issue with type T? not being convertible to T | T or T? which could’ve generated confusing errors
 +
    Return type of os.date is now inferred as DateTypeResult when argument is “t” or “!t”
 +
 +
Runtime improvementsPermalink
 +
 +
Out-of-memory exception handling has been improved. xpcall handlers will now actually be called with a “not enough memory” string and whatever string/object they return will be correctly propagated.
 +
 +
Other runtime improvements we’ve made:
 +
 +
    Performance of table.sort was improved further. It now guarantees N*log(N) time complexity in the worst case
 +
    Performance of table.concat was improved by ~5-7%
 +
    Performance of math.noise was improved by ~30%
 +
    Inlining of functions is now possible even when they used to compute their own arguments
 +
    Improved logic for determining whether inlining a function or unrolling a loop is profitable
 +
 +
Autocomplete improvementsPermalink
 +
 +
An issue with exported types not being suggested is now fixed.
 +
Debugger improvementsPermalink
 +
 +
We have fixed the search for the closest executable breakpoint line.
 +
 +
Previously, breakpoints might have been skipped in else blocks at the end of a function. This simplified example shows the issue:
 +
 +
<syntaxhighlight lang="lua">
 +
local function foo(isIt)
 +
    if isIt then
 +
        print("yes")
 +
    else
 +
        -- When 'true' block exits the function, breakpoint couldn't be placed here
 +
        print("no")
 +
    end
 +
end
 +
</syntaxhighlight>
 +
 +
ThanksPermalink
 +
 +
A very special thanks to all of our open source contributors:
 +
 +
    Petri Häkkinen
 +
    JohnnyMorganz
 +
    Gael
 +
    Jan
 +
    Alex Orlenko
 +
    mundusnine
 +
    Ben Mactavsin
 +
    RadiatedExodus
 +
    Lodinu Kalugalage
 +
    MagelessMayhem
 +
    Someon1e
 +
 +
Updated: July 28, 2023
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
Luau Recap: March 2023
 +
 +
March 31, 2023
 +
 +
How the time flies! The team has been busy since the last November Luau Recap working on some large updates that are coming in the future, but before those arrive, we have some improvements that you can already use!
 +
 +
[Cross-posted to the Roblox Developer Forum.]
 +
Improved type refinementsPermalink
 +
 +
Type refinements handle constraints placed on variables inside conditional blocks.
 +
 +
In the following example, while variable a is declared to have type number?, inside the if block we know that it cannot be nil:
 +
 +
<syntaxhighlight lang="lua">
 +
local function f(a: number?)
 +
    if a ~= nil then
 +
        a *= 2 -- no type errors
 +
    end
 +
    ...
 +
end
 +
</syntaxhighlight>
 +
 +
One limitation we had previously is that after a conditional block, refinements were discarded.
 +
 +
But there are cases where if is used to exit the function early, making the following code essentially act as a hidden else block.
 +
 +
We now correctly preserve such refinements and you should be able to remove assert function calls that were only used to get rid of false positive errors about types being nil.
 +
 +
<syntaxhighlight lang="lua">
 +
local function f(x: string?)
 +
    if not x then return end
 +
 +
    -- x is a 'string' here
 +
end
 +
</syntaxhighlight>
 +
 +
Throwing calls like error() or assert(false) instead of a return statement are also recognized.
 +
 +
<syntaxhighlight lang="lua">
 +
local function f(x: string?)
 +
    if not x then error('first argument is nil') end
 +
 +
    -- x is 'string' here
 +
end
 +
</syntaxhighlight>
 +
 +
Existing complex refinements like type/typeof, tagged union checks and other are expected to work as expected.
 +
Marking table.getn/foreach/foreachi as deprecatedPermalink
 +
 +
table.getn, table.foreach and table.foreachi were deprecated in Lua 5.1 that Luau is based on, and removed in Lua 5.2.
 +
 +
table.getn(x) is equivalent to rawlen(x) when ‘x’ is a table; when ‘x’ is not a table, table.getn produces an error.
 +
 +
It’s difficult to imagine code where table.getn(x) is better than either #x (idiomatic) or rawlen(x) (fully compatible replacement).
 +
 +
table.getn is also slower than both alternatives and was marked as deprecated.
 +
 +
table.foreach is equivalent to a for .. pairs loop; table.foreachi is equivalent to a for .. ipairs loop; both may also be replaced by generalized iteration.
 +
 +
Both functions are significantly slower than equivalent for loop replacements, are more restrictive because the function can’t yield.
 +
 +
Because both functions bring no value over other library or language alternatives, they were marked deprecated as well.
 +
 +
You may have noticed linter warnings about places where these functions are used. For compatibility, these functions are not going to be removed.
 +
Autocomplete improvementsPermalink
 +
 +
When table key type is defined to be a union of string singletons, those keys can now autocomplete in locations marked as ‘^’:
 +
 +
<syntaxhighlight lang="lua">
 +
type Direction = "north" | "south" | "east" | "west"
 +
 +
local a: {[Direction]: boolean} = {[^] = true}
 +
local b: {[Direction]: boolean} = {["^"]}
 +
local b: {[Direction]: boolean} = {^}
 +
</syntaxhighlight>
 +
 +
We also fixed incorrect and incomplete suggestions inside the header of if, for and while statements.
 +
Runtime improvementsPermalink
 +
 +
On the runtime side, we added multiple optimizations.
 +
 +
table.sort is now ~4.1x faster (when not using a predicate) and ~2.1x faster when using a simple predicate.
 +
 +
We also have ideas on how improve the sorting performance in the future.
 +
 +
math.floor, math.ceil and math.round now use specialized processor instructions. We have measured ~7-9% speedup in math benchmarks that heavily used those functions.
 +
 +
A small improvement was made to builtin library function calls, getting a 1-2% improvement in code that contains a lot of fastcalls.
 +
 +
Finally, a fix was made to table array part resizing that brings large improvement to performance of large tables filled as an array, but at an offset (for example, starting at 10000 instead of 1).
 +
 +
Aside from performance, a correctness issue was fixed in multi-assignment expressions.
 +
 +
<syntaxhighlight lang="lua">
 +
arr[1], n = n, n - 1
 +
</syntaxhighlight>
 +
 +
In this example, n - 1 was assigned to n before n was assigned to arr[1]. This issue has now been fixed.
 +
Analysis improvementsPermalink
 +
 +
Multiple changes were made to improve error messages and type presentation.
 +
 +
    Table type strings are now shown with newlines, to make them easier to read
 +
    Fixed unions of nil types displaying as a single ? character
 +
    “Type pack A cannot be converted to B” error is not reported instead of a cryptic “Failed to unify type packs”
 +
    Improved error message for value count mismatch in assignments like local a, b = 2
 +
 +
You may have seen error messages like Type 'string' cannot be converted to 'string?' even though usually it is valid to assign local s: string? = 'hello' because string is a sub-type of string?.
 +
 +
This is true in what is called Covariant use contexts, but doesn’t hold in Invariant use contexts, like in the example below:
 +
 +
<syntaxhighlight lang="lua">
 +
local a: { x: Model }
 +
local b: { x: Instance } = a -- Type 'Model' could not be converted into 'Instance' in an invariant context
 +
</syntaxhighlight>
 +
 +
In this example, while Model is a sub-type of Instance and can be used where Instance is required.
 +
 +
The same is not true for a table field because when using table b, b.x can be assigned an Instance that is not a Model. When b is an alias to a, this assignment is not compatible with a’s type annotation.
 +
 +
Some other light changes to type inference include:
 +
 +
    string.match and string.gmatch are now defined to return optional values as match is not guaranteed at runtime
 +
    Added an error when unrelated types are compared with ==/~=
 +
    Fixed issues where variable after typeof(x) == 'table' could not have been used as a table
 +
 +
ThanksPermalink
 +
 +
A very special thanks to all of our open source contributors:
 +
 +
    niansa/tuxifan
 +
    B. Gibbons
 +
    Epix
 +
    Harold Cindy
 +
    Qualadore
 +
 +
Updated: March 31, 2023
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
String Interpolation
 +
 +
February 2, 2023
 +
 +
String interpolation is the new syntax introduced to Luau that allows you to create a string literal with expressions inside of that string literal.
 +
 +
In short, it’s a safer and more ergonomic alternative over string.format.
 +
 +
Here’s a quick example of a string interpolation:
 +
 +
<syntaxhighlight lang="lua">
 +
local combos = {2, 7, 1, 8, 5}
 +
print(`The lock combination is {table.concat(combos)}. Again, {table.concat(combos, ", ")}.`)
 +
--> The lock combination is 27185. Again, 2, 7, 1, 8, 5.
 +
</syntaxhighlight>
 +
 +
String interpolation also composes well with the __tostring metamethod.
 +
 +
<syntaxhighlight lang="lua">
 +
local balance = setmetatable({ value = 500 }, {
 +
    __tostring = function(self)
 +
        return "$" .. tostring(self.value)
 +
    end
 +
})
 +
 +
print(`You have {balance}!`)
 +
--> You have $500!
 +
</syntaxhighlight>
 +
 +
To find out more details about this feature, check out Luau Syntax page.
 +
 +
This is also the first major language feature implemented in a contribution from the open-source community. Thanks Kampfkarren!
 +
 +
Updated: February 2, 2023
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
Luau Recap: November 2022
 +
 +
November 30, 2022
 +
 +
While the team is busy to bring some bigger things in the future, we have made some small improvements this month.
 +
 +
[Cross-posted to the Roblox Developer Forum.]
 +
Analysis improvementsPermalink
 +
 +
We have improved tagged union type refinements to only include unhandled type cases in the else branch of the if statement:
 +
 +
<syntaxhighlight lang="lua">
 +
type Ok<T> = { tag: "ok", value: T }
 +
type Err = { tag: "error", msg: string }
 +
type Result<T> = Ok<T> | Err
 +
 +
function unwrap<T>(r: Result<T>): T?
 +
    if r.tag == "ok" then
 +
        return r.value
 +
    else
 +
        -- Luau now understands that 'r' here can only be the 'Err' part
 +
        print(r.msg)
 +
        return nil
 +
    end
 +
end
 +
</syntaxhighlight>
 +
 +
For better inference, we updated the definition of Enum.SomeType:GetEnumItems() to return {Enum.SomeType} instead of common {EnumItem} and the return type of next function now includes the possibility of key being nil.
 +
 +
Finally, if you use and operator on non-boolean values, boolean type will no longer be added by the type inference:
 +
 +
<syntaxhighlight lang="lua">
 +
local function f1(a: number?)
 +
    -- 'x' is still a 'number?' and doesn't become 'boolean | number'
 +
    local x = a and 5
 +
end
 +
</syntaxhighlight>
 +
 +
Error message improvementsPermalink
 +
 +
We now give an error when built-in types are being redefined:
 +
 +
<syntaxhighlight lang="lua">
 +
type string = number -- Now an error: Redefinition of type 'string'
 +
</syntaxhighlight>
 +
 +
We also had a parse error missing in case you forgot your default type pack parameter value. We accepted the following code silently without raising an issue:
 +
 +
<syntaxhighlight lang="lua">
 +
type Foo<T... = > = nil -- Now an error: Expected type, got '>'
 +
</syntaxhighlight>
 +
 +
Error about function argument count mismatch no longer points at the last argument, but instead at the function in question. So, instead of:
 +
 +
<syntaxhighlight lang="lua">
 +
function myfunction(a: number, b:number) end
 +
myfunction(123)
 +
          ---
 +
</syntaxhighlight>
 +
 +
We now highlight this:
 +
 +
<syntaxhighlight lang="lua">
 +
function myfunction(a: number, b:number) end
 +
myfunction(123)
 +
----------
 +
</syntaxhighlight>
 +
 +
If you iterate over a table value that could also be nil, you get a better explanation in the error message:
 +
 +
<syntaxhighlight lang="lua">
 +
local function f(t: {number}?)
 +
    for i,v in t do -- Value of type {number}? could be nil
 +
        --...
 +
    end
 +
end
 +
</syntaxhighlight>
 +
 +
Previously it was Cannot call non-function {number}? which was confusing.
 +
 +
And speaking of confusing, some of you might have seen an error like Type 'string' could not be converted into 'string'.
 +
 +
This was caused by Luau having both a primitive type string and a table type coming from string library. Since the way you can get the type of the string library table is by using typeof(string), the updated error message will mirror that and report Type 'string' could not be converted into 'typeof(string)'.
 +
 +
Parsing now recovers with a more precise error message if you forget a comma in table constructor spanning multiple lines:
 +
 +
<syntaxhighlight lang="lua">
 +
local t = {
 +
    a = 1
 +
    b = 2 -- Expected ',' after table constructor element
 +
    c = 3 -- Expected ',' after table constructor element
 +
}
 +
</syntaxhighlight>
 +
 +
Updated: November 30, 2022
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
Luau origins and evolution
 +
 +
November 4, 2022
 +
 +
At the heart of Roblox technology lies Luau, a scripting language derived from Lua 5.1 that is being developed by an internal team of programming language experts with the help of open source contributors.
 +
 +
It powers all user-generated content on Roblox, providing access to a very rich set of APIs that allows manipulation of objects in the 3D world, backend API access, UI interaction and more. Hundreds of thousands of developers write code in Luau every month, with top experiences using hundreds of thousands of lines of code, adding up to hundreds of millions of lines of code across the platform. For many of them, it is the first programming language they learn, and one they spend the majority of their time programming in. Using a set of extended APIs developers also customize their workflows by writing plugins to Roblox Studio, where they work on their experiences, using an extended API surface to interact with all aspects of the editor.
 +
 +
It also powers a lot of application code that Roblox engineers are writing: Universal App, the gateway to the worlds of Roblox that is used by tens of millions of people every day, has 95% of its functionality implemented in Luau, and Roblox Studio has a lot of builtin critical functionality such as part and terrain editors, marketplace browser, avatar and animation editors, material manager and more, implemented in Luau as a plugin, mostly using the same APIs that developers have access to. Every week, updates to this internal codebase that is now over 2 million lines large, are shipped to all Roblox users.
 +
 +
In addition to Roblox use cases, Luau is also open-source and is seeing an increased adoption in other projects and applications.
 +
 +
But why did we use Lua in the first place, and why did we decide to pursue building a new language on top of it?
 +
Early beginningsPermalink
 +
 +
Around 2006, when a very early version of the Roblox platform was developed, the question of user generated behaviors emerged. Before that, users were able to build non-interactive content on Roblox, and the only form of interaction was physics simulation. While this provided rich emergent behavior, it was hard to build gameplay on top of this: for example, to build a Capture The Flag game, you need to handle collision between players and flags spread throughout the map with a bit of logic that dictates how to adjust team points and when to remove or recreate the objects.
 +
 +
After an early and brief misstep when we decided to add a few gameplay objects to the core definition of Roblox worlds (some developers may recognize FlagStand as a class name…), the Roblox co-founder Erik Cassel realized that an approach like this is fundamentally limiting the power of user generated content. It’s not enough to give creators the basic blocks on top of which to build their creations, it’s critical to expose the power of a full Turing-complete programming language. Without this, the expressive capability and the reach of the platform would have been restricted far too much.
 +
 +
But which programming language to choose? This is where Lua, which was, and still is, one of the dominant programming languages used in video games, comes in.
 +
 +
In addition to its simplicity, which made the language easy to learn and get productive in, Lua was the fastest scripting language compared to popular alternatives like Python or JavaScript at the time1, designed to be embedded which meant an easy ability to expose APIs from the host application to the scripts as well as high degree of execution control from the host, and implemented coroutines, a very powerful concurrency primitive that allowed to easily and intuitively script behaviors for independent actors in game using linear control flow.
 +
 +
Instead of having a large standard library, the expectation was that the embedding application would define a set of APIs that that application needed, as well as establish policies of running the code - which gave us a lot of freedom in how to structure the APIs and when the scripts would get triggered during the simulation of a single frame.
 +
Power of simplicityPermalink
 +
 +
Lua is a simple language. What does simplicity mean for us?
 +
 +
Being a simple language means having a small set of features. Lua has all the fundamental features but doesn’t have a lot of syntax sugar - this means the language is easier to teach and learn, and you rarely run into code that’s difficult to understand syntactically because it uses an unfamiliar construct. Of course, this also means that some programs in Lua are longer than equivalent programs in languages that have more dedicated constructs to solve specific problems, such as list comprehensions in Python.
 +
 +
Being a simple language means having a minimal set of rules for every feature. Lua does deviate from this in certain respects (which is to say, the language could have been even simpler!), but notably for a dynamic language the behavior of fundamental operators is generally easy to explain and unsurprising - for example, two values in Lua are equal iff they have the same type and the same value, as such 0 == “0” is false; as another example, for loops introduce unique variable bindings on every iteration, as such capturing the iteration variable in a closure produces unique values. These decisions lead to more concise and efficient implementation and eliminate a class of bugs in programs.
 +
 +
Being a simple language means having a small implementation. This may be immaterial to people writing code in the language, but it leads to an implementation that can be of higher quality; simpler implementations can also be easier to optimize for memory or performance, and are easier to build upon.
 +
 +
Developers on the Roblox platform have very diverse programming backgrounds. Some are writing their first line of code in Roblox Studio, while others have computer science degrees and experience working in multiple different programming languages. While it’s always possible to support two different programming languages that target different segments of the audience, that fragments the ecosystem and makes the programming story less consistent (impacting documentation, tutorials, code reuse, ability for community members to help each other write code, presents challenges with interaction between different languages in the same experience and more). A better outcome is one where a single language can serve both audiences - this requires a language that strikes a balance between simplicity and generality, and while Lua isn’t perfect here, it’s great as a foundation for a language like this2.
 +
 +
In many ways, Lua is simultaneously simple and pragmatic: many parts of the language are difficult to make much better without a lot of added complexity, but at the same time it requires little in the way of extra functionality to be able to solve problems efficiently. That said, no language is perfect, and within several areas of Lua we felt that the tradeoffs weren’t quite right for our use case.
 +
Respectful evolutionPermalink
 +
 +
In 2019, we decided to build Luau - a language derived from Lua and compatible with Lua 5.1, which is the version we’ve been using all these years. At the time we evaluated other routes, but ultimately settled on this as the most optimal long-term.
 +
 +
On one hand, we loved a lot of things about Lua - both design wise and implementation wise, while there were some decisions we felt were suboptimal, by and large it was an almost perfect foundation for what we’ve set out to achieve.
 +
 +
On the other hand, we’ve been running into the limitations of Lua on large code bases in absence of type checking, performance was good but not great, and some missing features would have been great to have.
 +
 +
Some of the things we’ve been missing have been added in later versions of Lua, yet we were still using Lua 5.1. While we would have loved to use a later version of the language standard, Lua 5.x releases are not backwards compatible, and some releases remove support for features that are in wide use at Roblox. For Roblox, backwards compatibility is an essential feature of the platform - while we don’t have a guarantee that content created 10 years ago still works, to the extent that we can achieve that without restricting the platform evolution too much, we try.
 +
 +
What we’ve realized is that Lua is a great foundation for a perfect language that we can build for Roblox.
 +
 +
We would maintain backwards compatibility with Lua 5.1 but evolve the language from there; sometimes this means taking later features from Lua that don’t conflict with the existing language or our design values, sometimes this means innovating beyond what Lua has done. Crucially, we must maintain the balance between simplicity and power - we still value simplicity, we still need to avoid a feature explosion to ensure that the features compose and are of high quality, and we still need the language to be a good fit for beginners.
 +
 +
One of the largest limitations that we’ve seen is the lack of type checking making it easy to make mistakes in large code bases, as such support for type checking was a requirement for Luau. However, it’s important that the type checker is mostly transparent to the developers who don’t want to invest the time to learn it - anything else would change the learning curve too much for the language to be suitable for beginners. As such, we’ve investing in gradual typing, and our type checker is learning to strike a balance between inferring useful types for completely untyped programs (which, among other things, greatly enhances editing experience through type-aware autocomplete), and the lack of false positive diagnostics that can be confusing and distracting.
 +
 +
While we did need to introduce extra syntax to the language - most notably, to support optional type annotations - it was important for us to maintain the cohesion of the overall syntax. We aren’t seeking to make a new language with a syntax alien to Lua programmers - Luau programs are still recognizably Lua, and to the extent possible we try to avoid new syntactic features. In a sense, we still want the syntax, semantics, and the runtime to be simple and minimal - but at the same time we have important problems to solve with respect to ergonomics, robustness and performance of the language, and solving some of them requires having slightly more complex syntax, semantics, or implementation.
 +
 +
So in finding ways to evolve Luau, we strive to design features that feel like they would be at home in Lua. At the same time, we’ve adopted a more open evolution process - the language development is driven through RFCs that are designs open to the public that anyone can contribute to - this is in contrast with Lua, which has a very closed development process, and is one of the reasons why it would have been difficult for us to keep using Lua as we wouldn’t get a say in its development. At the same time, to ensure the design criterias are met, it’s important that the Luau development team at Roblox maintains a final say over design and implementation of the language3, while taking the community’s proposals and input into consideration.
 +
Importance of co-designPermalink
 +
 +
Luau language is developed in concert with the language compiler, runtime, type checker and other analysis tools, autocomplete engine and other tooling, and that development is guided by the vast volume of existing Luau code, both internal and external.
 +
 +
This is one of the key principles behind our evolution philosophy - neither layer is developed in isolation, and instead concerns at every level inform all other aspects of the language design and implementation.
 +
 +
This means that when designing language features, we make sure that they can be implemented efficiently, type checked properly, can be supported well in editing and analysis tools and have a positive impact on the code internal and external engineers write. When we find issues in any component, we can always ask, what changes to other components or even language design would make for a better overall solution.
 +
 +
This avoids some classes of design problems, for example we won’t specify a language feature that has a prohibitively high implementation cost, as it violates our simplicity criteria, or that is impractical to implement efficiently, as that would create a performance hazard. This also means that when implementing various components of the language we cross-check the concerns and applicability of these across the entire stack - for example, we’ve reworked our auto-complete system to use the same type inference engine that the type checking / analysis tools use, which had immense benefits for the experience of editing code, but also applied significant back pressure on the type inference itself, forcing us to improve it substantially and fix a lot of corner cases that would otherwise have lingered unnoticed.
 +
 +
Whenever we develop features, optimizations, improve our analysis engine or enhance the standard libraries, we also heavily rely on code written in Luau to validate our hypotheses. When working on new features we find motivation in the real problems that we see our developers face. For example, we implemented the new ternary operator after seeing a large set of cases where existing Lua’s a and b or c pattern was error-prone for boolean values, which made it easy to accidentally introduce a mistake that was hard to identify automatically. All optimizations and new analysis features are validated on our internal 2M LOC codebase before being added to Luau, which allows us to quickly get initial validation of ideas, or invalidate some approaches as infeasible / unhelpful.
 +
 +
In addition to that, while we don’t have direct access to community-developed source code for privacy reasons, we can run experiments and collect telemetry4, which also helps us make decisions regarding backwards compatibility. Due to Hyrum’s law, technically any change in the language or libraries, no matter how small, would be backwards incompatible - instead we adopt the notion of pragmatic balance between strict backwards compatibility5 and pragmatic compatibility concerns. For example, later versions of Lua make some library functions like table.insert/table.remove more strict with how they handle out of range indices. We have evaluated this change for compatibility by collecting telemetry on the use of out of range indices in these functions on the Roblox platform and concluded that applying the stricter checking would break existing programs, and instead had to slightly adjust the rules for out of range behavior in ways that was benign for existing code but prevented catastrophic performance degradation for large out of range indices. Because we couldn’t afford to introduce new runtime errors in this case, we also added a set of linting rules to our analysis engine to flag potential misuse of table.insert/table.remove before the code ever gets to run - this diagnostics is informational and as such doesn’t affect backwards compatibility, but does help prevent mistakes.
 +
 +
There are also cases where this co-design approach prevents introduction of features that can lead to easy misuse, which can be difficult to see in the design of the feature itself, but becomes more apparent when you consider features in context of the entire ecosystem. This is a good thing - it means co-design acts as a forcing function on the language simplicity and makes it easier to flag potential bad interactions between different language features, or language features and tooling, or language features and existing programming patterns that are in widespread use in real-world code. By making sure that all features are validated for their impact across the stack and on code written in Luau, we ultimately get a better, simpler and more cohesive language.
 +
Efficient executionPermalink
 +
 +
One of the critical goals in front of Luau is efficiency, both from the performance and memory perspective. There’s only so many milliseconds in a frame, and we simultaneously see the need to increase the scale and complexity of simulated experiences, which requires more memory and computation, as well as the need to fit more comfortably into smaller budgets of performance memory for better experience on smaller devices. In fact, one of the motivations for Luau in 2019 has been improved performance, as we saw many opportunities to go beyond Lua with a redesigned implementation.
 +
 +
Crucially, our performance needs are somewhat unique and require somewhat unique solutions.
 +
 +
We need Luau to run on many platforms where native code generation is either prohibited by the platform vendor or impractical due to tight memory constraints. As such, in terms of execution performance it’s critical that we have a very fast interpreter6. However, we have freedom in terms of high level design of the entire stack - for example, clients never see the source code of the scripts as all compilation to bytecode happens on the server; this gives us an opportunity to perform more involved and expensive optimizations during that process as well as have the smallest possible startup time on the client without complex pre-parse steps. Notably, our bytecode compiler performs a series of high level optimizations including function inlining and loop unrolling that in other dynamic languages is often left to the just-in-time compiler.
 +
 +
Another area where performance is critical is garbage collection. Garbage collection is crucial for the language’s simplicity as it makes memory management easier to reason about, but it does require a substantial amount of implementation effort to keep it efficient. For Roblox and for any other game engine or interactive simulation, latency is critical and so our collector is heavily optimized for that - to the extent possible collection is incremental and stop-the-world pauses are very brief. Another part of the performance story here however is the language and data structure design - by making sure that core data types are efficient in how they are laid out in memory we reduce the amount of work garbage collector takes to trace the heap, and, as another example of co-design, we try to make sure that language features are conscious of the impact they have on memory and garbage collection efficiency.
 +
 +
However, from a whole-platform standpoint there’s a lot of performance aspects that go beyond single-threaded execution. This is an active area of research and development for the team, as to really leverage the hardware the code is running on we need to think about SIMD, hardware thread utilization as well as running code in a cluster of nodes. These considerations inform current and future development of the runtime and the language (for example, our runtime now supports efficient operations on short SIMD vectors even in interpreted mode, and the VM is fairly lightweight to instantiate which makes running many VMs per core practical, with message passing or access to shared Roblox data model used to make gameplay features feasible to implement), but we’re definitely in the early days here - our first implementation of parallel script execution in Roblox just shipped earlier this year. This is likely the area where a lot of future innovations will happen as well.
 +
FuturePermalink
 +
 +
We’re very happy with the success of Luau - in several years we’ve established consistent processes for evolving the language and so far we found a good balance between simplicity, ease of use, performance and robustness of the language, its implementation and the tooling surrounding it. The language keeps continuously evolving but at a pace that is easy to stay on top of - in 2022 we shipped a few syntactic extensions for type annotations but no changes to the syntax of the language outside of types, and only one major semantic change to the for loop iteration that actually made the language easier to use by avoiding the need to specify the table traversal style via pairs/ipairs. We try to make sure that the features are general and provide enough extensibility so that libraries can be built on top of the language to make it easier to write code, while also making it practical to use the language without complex supporting frameworks.
 +
 +
There’s still a lot of ground to cover, and we’ll be working on Luau for years to come. We’re in the process of building the next version of our type inference / checking engine to make sure that all users of the language regardless of their expertise benefit from it, we’ve started investing in native code generation as we’re reaching the limits of interpreted performance (although some exciting opportunities for compiler optimization are still on the horizon), and there’s still a lot of hard design and implementation work ahead of us for some important language features and standard libraries. And as mentioned, our execution model will likely see a lot of innovation as we push the boundaries of hardware utilization across cores and nodes.
 +
 +
Overall, Luau is like an iceberg - the surface is simple to learn and use, but it hides the tremendous amount of careful design, engineering and attention to detail, and we plan to continue to invest in it while trying to keep the outer surface comparatively small. We’re excited to see how far we can take it!
 +
 +
    High-performance JavaScript engines didn’t exist at the time! LuaJIT was around the corner and redefined the performance expectations of dynamic languages. ↩
 +
 +
    In fact, scaling to large teams of expert programmers is one of the core motivations behind our creating Luau, while a requirement to still be suitable for beginner programmers guides our evolution direction. ↩
 +
 +
    This would have been difficult to drive in any existing large established language like JavaScript or Python. ↩
 +
 +
    This is limited to Roblox platform and doesn’t exist in open-source releases. ↩
 +
 +
    Which we do follow in some areas, such as syntactic compatibility - all existing programs that parse must continue to parse the same way as the language evolves. ↩
 +
 +
    Some design decisions and implementation techniques are documented on our performance page. ↩
 +
 +
Updated: November 4, 2022
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
Luau Recap: September & October 2022
 +
 +
November 1, 2022
 +
 +
Luau is our new language that you can read more about at https://luau-lang.org.
 +
 +
[Cross-posted to the Roblox Developer Forum.]
 +
Semantic subtypingPermalink
 +
 +
One of the most important goals for Luau is to avoid false positives, that is cases where Script Analysis reports a type error, but in fact the code is correct. This is very frustrating, especially for beginners. Spending time chasing down a gnarly type error only to discover that it was the type system that’s wrong is nobody’s idea of fun!
 +
 +
We are pleased to announce that a major component of minimizing false positives has landed, semantic subtyping, which removes a class of false positives caused by failures of subtyping. For example, in the program
 +
 +
<syntaxhighlight lang="lua">
 +
  local x : CFrame = CFrame.new()
 +
  local y : Vector3 | CFrame
 +
  if (math.random()) then
 +
    y = CFrame.new()
 +
  else
 +
    y = Vector3.new()
 +
  end
 +
  local z : Vector3 | CFrame = x * y -- Type Error!
 +
</syntaxhighlight>
 +
 +
an error is reported, even though there is no problem at runtime. This is because CFrame’s multiplication has two overloads:
 +
 +
<syntaxhighlight lang="lua">
 +
    ((CFrame, CFrame) -> CFrame)
 +
  & ((CFrame, Vector3) -> Vector3)
 +
</syntaxhighlight>
 +
 +
The current syntax-driven algorithm for subtyping is not sophisticated enough to realize that this is a subtype of the desired type:
 +
 +
<syntaxhighlight lang="lua">
 +
  (CFrame, Vector3 | CFrame) -> (Vector3 | CFrame)
 +
</syntaxhighlight>
 +
 +
Our new algorithm is driven by the semantics of subtyping, not the syntax of types, and eliminates this class of false positives.
 +
 +
If you want to know more about semantic subtyping in Luau, check out our technical blog post on the subject.
 +
Other analysis improvementsPermalink
 +
 +
    Improve stringification of function types.
 +
    Improve parse error warnings in the case of missing tokens after a comma.
 +
    Improve typechecking of expressions involving variadics such as { ... }.
 +
    Make sure modules don’t return unbound generic types.
 +
    Improve cycle detection in stringifying types.
 +
    Improve type inference of combinations of intersections and generic functions.
 +
    Improve typechecking when calling a function which returns a variadic e.g. () -> (number...).
 +
    Improve typechecking when passing a function expression as a parameter to a function.
 +
    Improve error reporting locations.
 +
    Remove some sources of memory corruption and crashes.
 +
 +
Other runtime and debugger improvementsPermalink
 +
 +
    Improve performance of accessing debug info.
 +
    Improve performance of getmetatable and setmetatable.
 +
    Remove a source of freezes in the debugger.
 +
    Improve GC accuracy and performance.
 +
 +
ThanksPermalink
 +
 +
Thanks for all the contributions!
 +
 +
    AllanJeremy
 +
    JohnnyMorganz
 +
    jujhar16
 +
    petrihakkinen
 +
 +
Updated: November 1, 2022
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
Semantic Subtyping in Luau
 +
 +
October 31, 2022
 +
 +
Luau is the first programming language to put the power of semantic subtyping in the hands of millions of creators.
 +
Minimizing false positivesPermalink
 +
 +
One of the issues with type error reporting in tools like the Script Analysis widget in Roblox Studio is false positives. These are warnings that are artifacts of the analysis, and don’t correspond to errors which can occur at runtime. For example, the program
 +
 +
<syntaxhighlight lang="lua">
 +
  local x = CFrame.new()
 +
  local y
 +
  if (math.random()) then
 +
    y = CFrame.new()
 +
  else
 +
    y = Vector3.new()
 +
  end
 +
  local z = x * y
 +
</syntaxhighlight>
 +
 +
reports a type error which cannot happen at runtime, since CFrame supports multiplication by both Vector3 and CFrame. (Its type is ((CFrame, CFrame) -> CFrame) & ((CFrame, Vector3) -> Vector3).)
 +
 +
False positives are especially poor for onboarding new users. If a type-curious creator switches on typechecking and is immediately faced with a wall of spurious red squiggles, there is a strong incentive to immediately switch it off again.
 +
 +
Inaccuracies in type errors are inevitable, since it is impossible to decide ahead of time whether a runtime error will be triggered. Type system designers have to choose whether to live with false positives or false negatives. In Luau this is determined by the mode: strict mode errs on the side of false positives, and nonstrict mode errs on the side of false negatives.
 +
 +
While inaccuracies are inevitable, we try to remove them whenever possible, since they result in spurious errors, and imprecision in type-driven tooling like autocomplete or API documentation.
 +
Subtyping as a source of false positivesPermalink
 +
 +
One of the sources of false positives in Luau (and many other similar languages like TypeScript or Flow) is subtyping. Subtyping is used whenever a variable is initialized or assigned to, and whenever a function is called: the type system checks that the type of the expression is a subtype of the type of the variable. For example, if we add types to the above program
 +
 +
<syntaxhighlight lang="lua">
 +
  local x : CFrame = CFrame.new()
 +
  local y : Vector3 | CFrame
 +
  if (math.random()) then
 +
    y = CFrame.new()
 +
  else
 +
    y = Vector3.new()
 +
  end
 +
  local z : Vector3 | CFrame = x * y
 +
</syntaxhighlight>
 +
 +
then the type system checks that the type of CFrame multiplication is a subtype of (CFrame, Vector3 | CFrame) -> (Vector3 | CFrame).
 +
 +
Subtyping is a very useful feature, and it supports rich type constructs like type union (T | U) and intersection (T & U). For example, number? is implemented as a union type (number | nil), inhabited by values that are either numbers or nil.
 +
 +
Unfortunately, the interaction of subtyping with intersection and union types can have odd results. A simple (but rather artificial) case in older Luau was:
 +
 +
<syntaxhighlight lang="lua">
 +
  local x : (number?) & (string?) = nil
 +
  local y : nil = nil
 +
  y = x -- Type '(number?) & (string?)' could not be converted into 'nil'
 +
  x = y
 +
</syntaxhighlight>
 +
 +
This error is caused by a failure of subtyping, the old subtyping algorithm reports that (number?) & (string?) is not a subtype of nil. This is a false positive, since number & string is uninhabited, so the only possible inhabitant of (number?) & (string?) is nil.
 +
 +
This is an artificial example, but there are real issues raised by creators caused by the problems, for example https://devforum.roblox.com/t/luau-recap-july-2021/1382101/5. Currently, these issues mostly affect creators making use of sophisticated type system features, but as we make type inference more accurate, union and intersection types will become more common, even in code with no type annotations.
 +
 +
This class of false positives no longer occurs in Luau, as we have moved from our old approach of syntactic subtyping to an alternative called semantic subtyping.
 +
Syntactic subtypingPermalink
 +
 +
AKA “what we did before.”
 +
 +
Syntactic subtyping is a syntax-directed recursive algorithm. The interesting cases to deal with intersection and union types are:
 +
 +
    Reflexivity: T is a subtype of T
 +
    Intersection L: (T₁ & … & Tⱼ) is a subtype of U whenever some of the Tᵢ are subtypes of U
 +
    Union L: (T₁ | … | Tⱼ) is a subtype of U whenever all of the Tᵢ are subtypes of U
 +
    Intersection R: T is a subtype of (U₁ & … & Uⱼ) whenever T is a subtype of all of the Uᵢ
 +
    Union R: T is a subtype of (U₁ | … | Uⱼ) whenever T is a subtype of some of the Uᵢ.
 +
 +
For example:
 +
 +
    By Reflexivity: nil is a subtype of nil
 +
    so by Union R: nil is a subtype of number?
 +
    and: nil is a subtype of string?
 +
    so by Intersection R: nil is a subtype of (number?) & (string?).
 +
 +
Yay! Unfortunately, using these rules:
 +
 +
    number isn’t a subtype of nil
 +
    so by Union L: (number?) isn’t a subtype of nil
 +
    and: string isn’t a subtype of nil
 +
    so by Union L: (string?) isn’t a subtype of nil
 +
    so by Intersection L: (number?) & (string?) isn’t a subtype of nil.
 +
 +
This is typical of syntactic subtyping: when it returns a “yes” result, it is correct, but when it returns a “no” result, it might be wrong. The algorithm is a conservative approximation, and since a “no” result can lead to type errors, this is a source of false positives.
 +
Semantic subtypingPermalink
 +
 +
AKA “what we do now.”
 +
 +
Rather than thinking of subtyping as being syntax-directed, we first consider its semantics, and later return to how the semantics is implemented. For this, we adopt semantic subtyping:
 +
 +
    The semantics of a type is a set of values.
 +
    Intersection types are thought of as intersections of sets.
 +
    Union types are thought of as unions of sets.
 +
    Subtyping is thought of as set inclusion.
 +
 +
For example:
 +
Type Semantics
 +
number { 1, 2, 3, … }
 +
string { “foo”, “bar”, … }
 +
nil { nil }
 +
number? { nil, 1, 2, 3, … }
 +
string? { nil, “foo”, “bar”, … }
 +
(number?) & (string?) { nil, 1, 2, 3, … } ∩ { nil, “foo”, “bar”, … } = { nil }
 +
 +
and since subtypes are interpreted as set inclusions:
 +
Subtype Supertype Because
 +
nil number? { nil } ⊆ { nil, 1, 2, 3, … }
 +
nil string? { nil } ⊆ { nil, “foo”, “bar”, … }
 +
nil (number?) & (string?) { nil } ⊆ { nil }
 +
(number?) & (string?) nil { nil } ⊆ { nil }
 +
 +
So according to semantic subtyping, (number?) & (string?) is equivalent to nil, but syntactic subtyping only supports one direction.
 +
 +
This is all fine and good, but if we want to use semantic subtyping in tools, we need an algorithm, and it turns out checking semantic subtyping is non-trivial.
 +
Semantic subtyping is hardPermalink
 +
 +
NP-hard to be precise.
 +
 +
We can reduce graph coloring to semantic subtyping by coding up a graph as a Luau type such that checking subtyping on types has the same result as checking for the impossibility of coloring the graph
 +
 +
For example, coloring a three-node, two color graph can be done using types:
 +
 +
<syntaxhighlight lang="lua">
 +
type Red = "red"
 +
type Blue = "blue"
 +
type Color = Red | Blue
 +
type Coloring = (Color) -> (Color) -> (Color) -> boolean
 +
type Uncolorable = (Color) -> (Color) -> (Color) -> false
 +
</syntaxhighlight>
 +
 +
Then a graph can be encoded as an overload function type with subtype Uncolorable and supertype Coloring, as an overloaded function which returns false when a constraint is violated. Each overload encodes one constraint. For example a line has constraints saying that adjacent nodes cannot have the same color:
 +
 +
<syntaxhighlight lang="lua">
 +
type Line = Coloring
 +
  & ((Red) -> (Red) -> (Color) -> false)
 +
  & ((Blue) -> (Blue) -> (Color) -> false)
 +
  & ((Color) -> (Red) -> (Red) -> false)
 +
  & ((Color) -> (Blue) -> (Blue) -> false)
 +
</syntaxhighlight>
 +
 +
A triangle is similar, but the end points also cannot have the same color:
 +
 +
<syntaxhighlight lang="lua">
 +
type Triangle = Line
 +
  & ((Red) -> (Color) -> (Red) -> false)
 +
  & ((Blue) -> (Color) -> (Blue) -> false)
 +
</syntaxhighlight>
 +
 +
Now, Triangle is a subtype of Uncolorable, but Line is not, since the line can be 2-colored. This can be generalized to any finite graph with any finite number of colors, and so subtype checking is NP-hard.
 +
 +
We deal with this in two ways:
 +
 +
    we cache types to reduce memory footprint, and
 +
    give up with a “Code Too Complex” error if the cache of types gets too large.
 +
 +
Hopefully this doesn’t come up in practice much. There is good evidence that issues like this don’t arise in practice from experience with type systems like that of Standard ML, which is EXPTIME-complete, but in practice you have to go out of your way to code up Turing Machine tapes as types.
 +
Type normalizationPermalink
 +
 +
The algorithm used to decide semantic subtyping is type normalization. Rather than being directed by syntax, we first rewrite types to be normalized, then check subtyping on normalized types.
 +
 +
A normalized type is a union of:
 +
 +
    a normalized nil type (either never or nil)
 +
    a normalized number type (either never or number)
 +
    a normalized boolean type (either never or true or false or boolean)
 +
    a normalized function type (either never or an intersection of function types) etc
 +
 +
Once types are normalized, it is straightforward to check semantic subtyping.
 +
 +
Every type can be normalized (sigh, with some technical restrictions around generic type packs). The important steps are:
 +
 +
    removing intersections of mismatched primitives, e.g. number & bool is replaced by never, and
 +
    removing unions of functions, e.g. ((number?) -> number) | ((string?) -> string) is replaced by (nil) -> (number | string).
 +
 +
For example, normalizing (number?) & (string?) removes number & string, so all that is left is nil.
 +
 +
Our first attempt at implementing type normalization applied it liberally, but this resulted in dreadful performance (complex code went from typechecking in less than a minute to running overnight). The reason for this is annoyingly simple: there is an optimization in Luau’s subtyping algorithm to handle reflexivity (T is a subtype of T) that performs a cheap pointer equality check. Type normalization can convert pointer-identical types into semantically-equivalent (but not pointer-identical) types, which significantly degrades performance.
 +
 +
Because of these performance issues, we still use syntactic subtyping as our first check for subtyping, and only perform type normalization if the syntactic algorithm fails. This is sound, because syntactic subtyping is a conservative approximation to semantic subtyping.
 +
Pragmatic semantic subtypingPermalink
 +
 +
Off-the-shelf semantic subtyping is slightly different from what is implemented in Luau, because it requires models to be set-theoretic, which requires that inhabitants of function types “act like functions.” There are two reasons why we drop this requirement.
 +
 +
Firstly, we normalize function types to an intersection of functions, for example a horrible mess of unions and intersections of functions:
 +
 +
<syntaxhighlight lang="lua">
 +
((number?) -> number?) | (((number) -> number) & ((string?) -> string?))
 +
</syntaxhighlight>
 +
 +
normalizes to an overloaded function:
 +
 +
<syntaxhighlight lang="lua">
 +
((number) -> number?) & ((nil) -> (number | string)?)
 +
</syntaxhighlight>
 +
 +
Set-theoretic semantic subtyping does not support this normalization, and instead normalizes functions to disjunctive normal form (unions of intersections of functions). We do not do this for ergonomic reasons: overloaded functions are idiomatic in Luau, but DNF is not, and we do not want to present users with such non-idiomatic types.
 +
 +
Our normalization relies on rewriting away unions of function types:
 +
 +
<syntaxhighlight lang="lua">
 +
((A) -> B) | ((C) -> D)  →  (A & C) -> (B | D)
 +
</syntaxhighlight>
 +
 +
This normalization is sound in our model, but not in set-theoretic models.
 +
 +
Secondly, in Luau, the type of a function application f(x) is B if f has type (A) -> B and x has type A. Unexpectedly, this is not always true in set-theoretic models, due to uninhabited types. In set-theoretic models, if x has type never then f(x) has type never. We do not want to burden users with the idea that function application has a special corner case, especially since that corner case can only arise in dead code.
 +
 +
In set-theoretic models, (never) -> A is a subtype of (never) -> B, no matter what A and B are. This is not true in Luau.
 +
 +
For these two reasons (which are largely about ergonomics rather than anything technical) we drop the set-theoretic requirement, and use pragmatic semantic subtyping.
 +
Negation typesPermalink
 +
 +
The other difference between Luau’s type system and off-the-shelf semantic subtyping is that Luau does not support all negated types.
 +
 +
The common case for wanting negated types is in typechecking conditionals:
 +
 +
<syntaxhighlight lang="lua">
 +
-- initially x has type T
 +
if (type(x) == "string") then
 +
  --  in this branch x has type T & string
 +
else
 +
  -- in this branch x has type T & ~string
 +
end
 +
</syntaxhighlight>
 +
 +
This uses a negated type ~string inhabited by values that are not strings.
 +
 +
In Luau, we only allow this kind of typing refinement on test types like string, function, Part and so on, and not on structural types like (A) -> B, which avoids the common case of general negated types.
 +
Prototyping and verificationPermalink
 +
 +
During the design of Luau’s semantic subtyping algorithm, there were changes made (for example initially we thought we were going to be able to use set-theoretic subtyping). During this time of rapid change, it was important to be able to iterate quickly, so we initially implemented a prototype rather than jumping straight to a production implementation.
 +
 +
Validating the prototype was important, since subtyping algorithms can have unexpected corner cases. For this reason, we adopted Agda as the prototyping language. As well as supporting unit testing, Agda supports mechanized verification, so we are confident in the design.
 +
 +
The prototype does not implement all of Luau, just the functional subset, but this was enough to discover subtle feature interactions that would probably have surfaced as difficult-to-fix bugs in production.
 +
 +
Prototyping is not perfect, for example the main issues that we hit in production were about performance and the C++ standard library, which are never going to be caught by a prototype. But the production implementation was otherwise fairly straightforward (or at least as straightforward as a 3kLOC change can be).
 +
Next stepsPermalink
 +
 +
Semantic subtyping has removed one source of false positives, but we still have others to track down:
 +
 +
    overloaded function applications and operators,
 +
    property access on expressions of complex type,
 +
    read-only properties of tables,
 +
    variables that change type over time (aka typestates),
 +
    …
 +
 +
The quest to remove spurious red squiggles continues!
 +
AcknowledgmentsPermalink
 +
 +
Thanks to Giuseppe Castagna and Ben Greenman for helpful comments on drafts of this post.
 +
Further readingPermalink
 +
 +
If you want to find out more about Luau and semantic subtyping, you might want to check out…
 +
 +
    Luau. https://luau-lang.org/
 +
    Lily Brown, Andy Friesen and Alan Jeffrey, Goals of the Luau Type System, Human Aspects of Types and Reasoning Assistants (HATRA), 2021. https://arxiv.org/abs/2109.11397
 +
    Luau Typechecker Prototype. https://github.com/luau-lang/agda-typeck
 +
    Agda. https://agda.readthedocs.io/
 +
    Andrew M. Kent. Down and Dirty with Semantic Set-theoretic Types, 2021. https://pnwamk.github.io/sst-tutorial/
 +
    Giuseppe Castagna, Covariance and Contravariance, Logical Methods in Computer Science 16(1), 2022. https://arxiv.org/abs/1809.01427
 +
    Giuseppe Castagna and Alain Frisch, A gentle introduction to semantic subtyping, Proc. Principles and practice of declarative programming (PPDP), pp 198–208, 2005. https://doi.org/10.1145/1069774.1069793
 +
    Giuseppe Castagna, Mickaël Laurent, Kim Nguyễn, Matthew Lutze, On Type-Cases, Union Elimination, and Occurrence Typing, Principles of Programming Languages (POPL), 2022. https://doi.org/10.1145/3498674
 +
    Giuseppe Castagna, Programming with union, intersection, and negation types, 2022. https://arxiv.org/abs/2111.03354
 +
    Sam Tobin-Hochstadt and Matthias Felleisen, Logical types for untyped languages. International Conference on Functional Programming (ICFP), 2010. https://doi.org/10.1145/1863543.1863561
 +
    José Valim, My Future with Elixir: set-theoretic types, 2022. https://elixir-lang.org/blog/2022/10/05/my-future-with-elixir-set-theoretic-types/
 +
 +
Some other languages which support semantic subtyping…
 +
 +
    ℂDuce https://www.cduce.org/
 +
    Ballerina https://ballerina.io
 +
    Elixir https://elixir-lang.org/
 +
    eqWAlizer https://github.com/WhatsApp/eqwalizer
 +
 +
And if you want to see the production code, it’s in the C++ definitions of tryUnifyNormalizedTypes and NormalizedType in the open source Luau repo.
 +
 +
Updated: October 31, 2022
 +
 +
 +
 +
 +
 +
 +
 +
 +
  
 
Luau Recap: July & August 2022
 
Luau Recap: July & August 2022

Latest revision as of 04:03, 2 November 2024

Luau getting started: https://luau.org/getting-started


Luau below recaps from: https://luau.org/news/

Luau Recap: July 2024

July 23, 2024

Hello everyone!

While the Luau team is actively working on a big rewrite of the type inference and type checking engines (more news about that in the near future), we wanted to go over other changes and updates since our last recap back in October. Official Luau mascotPermalink

Luau has recently adopted a Hawaiian monk seal mascot named Hina, after the Hawaiian goddess of the moon.

Please welcome Hina the Seal!

Hina the Seal Native Code GenerationPermalink

We are happy to announce that the native code feature is out from the ‘Preview’ state and is fully supported for X64 (Intel/AMD) or A64 (ARM) processor architectures.

As a refresher, native code generation is the feature that allows Luau scripts that have been previously executed by interpreting bytecode inside the Luau VM to instead compile to machine code that the CPU understands and executes directly.

Since the release of the Preview, we have worked on improving code performance, memory use of the system, correctness and stability.

Some highlights: 

   Improved performance of the bit32 library functions
   Improved performance of numerical loops
   Optimized table array and property lookups
   Added native support for new buffer type operations
   Code optimizations based on knowing which types are returned from operations
   Code optimizations based on function argument type annotations
       This includes support for SIMD operations for annotated vector arguments

There are many other small improvements in generated code performance and size and we have plans for additional optimizations. Native function attributePermalink

For a better control of what code runs natively, we have introduced new syntax for function attributes: 

@native -- function compiles natively
local function foo()
    ...
end

This is the first attribute to become available and we are working on the ability to mark functions as deprecated using the @deprecated attribute. More on that here. Type information for runtime optimizationsPermalink

Native code generation works on any code without having to modify it. In certain situations, this means that the native compiler cannot be sure about the types involved in the operation.

Consider a simple function, working on a few values: 

local function MulAddScaled(a, b, c)
    return a * b * 0.75 + c * 0.25
end

Native compiler assumes that operations are most likely being performed on numbers and generates the appropriate fast path.

But what if the function is actually called with a vector type? 

local intlPos = MulAddScaled(Part.Position, v, vector(12, 0, 0))

To handle this, a slower path was generated to handle any other potential type of the argument. Because this path is not chosen as the first possible option, extra checking overhead prevents code from running as fast as it can.

When we announced the last update, we had already added some support for following the types used as arguments. 

local function MulAddScaled(a: vector, b: vector, c: vector)
    return a * b * 0.75 + c * 0.25
end

   NOTE: vector type is not enabled by default, check out defaultOptions and setupVectorHelpers functions in Conformance.test.cpp file as an example of the vector library setup.

Since then, we have extended this to support type information on locals, following complex types and even inferring results of additional operations. 

type Vertex = { p: vector, uv: vector, n: vector, t: vector, b: vector, h: number }
type Mesh = { vertices: {Vertex}, indices: {number} }

function calculate_normals(mesh: Mesh)
    for i = 1,#mesh.indices,3 do
        local a = mesh.vertices[mesh.indices[i]]
        local b = mesh.vertices[mesh.indices[i + 1]]
        local c = mesh.vertices[mesh.indices[i + 2]]

        local vba = a.p - b.p -- Inferred as a vector operation
        local vca = a.p - c.p

        local n = vba:Cross(vca) -- Knows that Cross returns vector

        a.n += n -- Inferred as a vector operation
        b.n += n
        c.n += n
    end
end

As can be seen, often it’s enough to annotate the type of the data structure and correct fast-path vector code will be generated from that without having to specify the type of each local or temporary. 

   NOTE: Advanced inference and operation lowering is enabled by using custom HostIrHooks callbacks. Check out ‘Vector’ test with ‘IrHooks’ option in Conformance.test.cpp and ConformanceIrHooks.h file for an example of the setup.

Note that support for native lowering hooks allows generation of CPU code that is multiple times faster than a generic metatable call.

Even when native compiler doesn’t have a specific optimization for a type, if the type can be resolved, shorter code sequences are generated and more optimizations can be made between separate operations. 

   NOTE: HostIrHooks callbacks also enable type inference and lowering for your custom userdata types. Check out ‘NativeUserdata’ test with ‘IrHooks’ option in Conformance.test.cpp and ConformanceIrHooks.h file for an example of the setup.

Runtime changesPermalink Stricter utf8 library validationPermalink

utf8 library will now correctly validate UTF-8 and reject inputs that have surrogates. utf8.len will return nil followed by the byte offset, utf8.codepoint and utf8.codes will error. This matches how other kinds of input errors were previously handled by those functions.

Strings that are validated using utf8.len will now always work properly with utf8.nfcnormalize and utf8.graphemes functions. Custom per-character validation logic is no longer required to check if a string is valid under utf8 requirements. Imprecise integer number warningPermalink

Luau stores numbers as 64-bit floating-point values. Integer values up to 2^53 are supported, but higher numbers might experience rounding.

For example, both 10000000000000000 and 9223372036854775808 are larger than 2^53, but match the rounding, while 10000000000000001 gets rounded down to 10000000000000000.

In cases where rounding takes place, you will get a warning message. If the large value is intended and rounding can be ignored, just add “.0” to the number to remove the warning: 

local a = 10000000000000001 -- Number literal exceeded available precision and was truncated to closest representable number
local b = 10000000000000001.0 -- Ok, but rounds to 10000000000000000

Leading | and & in typesPermalink

It is now possible to start your union and intersection types with a symbol. This can help align the type components more cleanly: 

type Options =
    | { tag: "cat", laziness: number }
    | { tag: "dog", happiness: number }

You can find more information and examples in the proposal Analysis ImprovementsPermalink

While our main focus is on a type-checking engine rewrite that is nearing completion, we have fixed some of the issues in the current one. 

   Relational operator errors are more conservative now and generate less false positive errors
   It is not an error to iterate over table properties when indexer is not part of the type
   Type packs with cycles are now correctly described in error messages
   Improved error message when value that is not a function is being used in a call
   Fixed stability issues which caused Studio to crash
   Improved performance for code bases with large number of scripts and complex types

Runtime ImprovementsPermalink

When converting numbers to strings in scientific notation, we will now skip the trailing ‘.’.

For example, tostring(1e+30) now outputs ‘1e+30’ instead of ‘1.e+30’. This improves compatibility with data formats like JSON. But please keep in mind that unless you are using JSON5, Luau can still output ‘inf’ and ‘nan’ numbers which might not be supported. 

   Construction of tables with 17-32 properties or 33-64 array elements is now 30% faster.
   table.concat method is now 2x faster when the separator is not used and 40% faster otherwise.
   table.maxn method is now 5-14x faster.
   vector constants are now stored in the constant table and avoid runtime construction.
   Operations like 5/x and 5-x with any constant on the left-hand-side are now performed faster, one less minor thing to think about!
   It is no longer possible to crash the server on a hang in the string library methods.

Luau as a supported language on GitHubPermalink

Lastly, if you have open-source or even private projects on GitHub which use Luau, you might be happy to learn that Luau now has official support on GitHub for .luau file extension. This includes recognizing files as using Luau programming language and having support for syntax highlighting.

A big thanks goes to our open source community for their generous contributions including pushing for broader Luau support: 

   birds3345
   bjornbytes
   Gskartwii
   jackdotink
   JohnnyMorganz
   khvzak
   kostadinsh
   mttsner
   mxruben
   petrihakkinen
   zeux

Updated: July 23, 2024






Luau Recap: October 2023

November 1, 2023

We’re still quite busy working on some big type checking updates that we hope to talk about soon, but we have a few equally exciting updates to share in the meantime!

Let’s dive in! Floor DivisionPermalink

Luau now has a floor division operator. It is spelled //: 

local a = 10 // 3 -- a == 3
a //= 2           -- a == 1

For numbers, a // b is equivalent to math.floor(a / b), and you can also overload this operator by implementing the __idiv metamethod. The syntax and semantics are borrowed from Lua 5.3 (although Lua 5.3 has an integer type while we don’t, we tried to match the behavior to be as close as possible). Native Codegen PreviewPermalink

We are actively working on our new native code generation module that can significantly improve the performance of compute-dense scripts by compiling them to X64 (Intel/AMD) or A64 (ARM) machine code and executing that natively. We aim to support all AArch64 hardware with the current focus being Apple Silicon (M1-M3) chips, and all Intel/AMD hardware that supports AVX1 (with no planned support for earlier systems). When the hardware does not support native code generation, any code that would be compiled as native just falls back to the interpreted execution.

When working with open-source releases, binaries now have native code generation support compiled in by default; you need to pass --codegen command line flag to enable it. If you use Luau as a library in a third-party application, you would need to manually link Luau.CodeGen library and call the necessary functions to compile specific modules as needed - or keep using the interpreter if you want to! If you work in Roblox Studio, we have integrated native code generation preview as a beta feature, which currently requires manual annotation of select scripts with --!native comment.

Our goal for the native code generation is to help reach ultimate performance for code that needs to process data very efficiently, but not necessarily to accelerate every line of code, and not to replace the interpreter. We remain committed to maximizing interpreted execution performance, as not all platforms will support native code generation, and it’s not always practical to use native code generation for large code bases because it has a larger memory impact than bytecode. We intend for this to unlock new performance opportunities for complex features and algorithms, e.g. code that spends a lot of time working with numbers and arrays, but not to dramatically change performance on UI code or code that spends a lot of its time calling Lua functions like table.sort, or external C functions (like Roblox engine APIs).

Importantly, native code generation does not change our behavior or correctness expectations. Code compiled natively should give the same results when it executes as non-native code (just take a little less time), and it should not result in any memory safety or sandboxing issues. If you ever notice native code giving a different result from non-native code, please submit a bug report.

We continue to work on many code size and performance improvements; here’s a short summary of what we’ve done in the last couple of months, and there’s more to come! 

   Repeated access to table fields with the same object and name are now optimized (e.g. t.x = t.x + 5 is faster)
   Numerical for loops are now compiled more efficiently, yielding significant speedups on hot loops
   Bit operations with constants are now compiled more efficiently on X64 (for example, bit32.lshift(x, 1) is faster); this optimization was already in place for A64
   Repeated access to array elements with the same object and index is now faster in certain cases
   Performance of function calls has been marginally improved on X64 and A64
   Fix code generation for some bit32.extract variants where we could produce incorrect results
   table.insert is now faster when called with two arguments as it’s compiled directly to native code
   To reduce code size, module code outside of functions is not compiled natively unless it has loops

Analysis ImprovementsPermalink

The break and continue keywords can now be used in loop bodies to refine variables. This was contributed by a community member - thank you, AmberGraceSoftware! 

function f(objects: { { value: string? } })
    for _, object in objects do
        if not object.value then
            continue
        end
        
        local x: string = object.value -- ok!
    end
end

When type information is present, we will now emit a warning when # or ipairs is used on a table that has no numeric keys or indexers. This helps avoid common bugs like using #t == 0 to check if a dictionary is empty. 

local message = { data = { 1, 2, 3 } }

if #message == 0 then -- Using '#' on a table without an array part is likely a bug
end

Finally, some uses of getfenv/setfenv are now flagged as deprecated. We do not plan to remove support for getfenv/setfenv but we actively discourage its use as it disables many optimizations throughout the compiler, runtime, and native code generation, and interferes with type checking and linting. Autocomplete ImprovementsPermalink

We used to have a bug that would arise in the following situation: 

--!strict
type Direction = "Left" | "Right"
local dir: Direction = "Left"

if dir == ""| then
end

(imagine the cursor is at the position of the | character in the if statement)

We used to suggest Left and Right even though they are not valid completions at that position. This is now fixed.

We’ve also added a complete suggestion for anonymous functions if one would be valid at the requested position. For example: 

local p = Instance.new('Part')
p.Touched:Connect(

You will see a completion suggestion function (anonymous autofilled). Selecting that will cause the following to be inserted into your code: 

local p = Instance.new('Part')
p.Touched:Connect(function(otherPart: BasePart)  end

We also fixed some confusing editor feedback in the following case: 

game:FindFirstChild(

Previously, the signature help tooltip would erroneously tell you that you needed to pass a self argument. We now correctly offer the signature FindFirstChild(name: string, recursive: boolean?): Instance Runtime ImprovementsPermalink 

   string.format’s handling of %* and %s is now 1.5-2x faster
   tonumber and tostring are now 1.5x and 2.5x faster respectively when working on primitive types
   Compiler now recognizes math.pi and math.huge and performs constant folding on the expressions that involve these at -O2; for example, math.pi*2 is now free.
   Compiler now optimizes if...then...else expressions into AND/OR form when possible (for example, if x then x else y now compiles as x or y)
   We had a few bugs around repeat..until statements when the until condition referred to local variables defined in the loop body. These bugs have been fixed.
   Fix an oversight that could lead to string.char and string.sub generating potentially unlimited amounts of garbage and exhausting all available memory.
   We had a bug that could cause the compiler to unroll loops that it really shouldn’t. This could result in massive bytecode bloat. It is now fixed.

luau-lang on GitHubPermalink

If you’ve been paying attention to our GitHub projects, you may have noticed that we’ve moved luau repository to a new luau-lang GitHub organization! This is purely an organizational change but it’s helping us split a few repositories for working with documentation and RFCs and be more organized with pull requests in different areas. Make sure to update your bookmarks and star our main repository if you haven’t already!

Lastly, a big thanks to our open source community for their generous contributions: 

   MagelessMayhem
   cassanof
   LoganDark
   j-hui
   xgqt
   jdpatdiscord
   Someon1e
   AmberGraceSoftware
   RadiantUwU
   SamuraiCrow

Updated: November 1, 2023






Luau Recap: July 2023

July 28, 2023

Our team is still spending a lot of time working on upcoming replacement for our type inference engine as well as working on native code generation to improve runtime performance.

However, we also worked on unrelated improvements during this time that are summarized here.

[Cross-posted to the Roblox Developer Forum.] Analysis improvementsPermalink

Indexing table intersections using x["prop"] syntax has been fixed and no longer reports a false positive error: 

type T = { foo: string } & { bar: number }
local x: T = { foo = "1", bar = 2 }

local y = x["bar"] -- This is no longer an error

Generic T... type is now convertible to ...any variadic parameter.

This solves issues people had with variadic functions and variadic argument: 

local function foo(...: any)
    print(...)
end

local function bar<T...>(...: T...)
    foo(...) -- This is no longer an error
end

We have also improved our general typechecking performance by ~17% and by additional ~30% in modules with complex types.

Other fixes include: 

   Fixed issue with type T? not being convertible to T | T or T? which could’ve generated confusing errors
   Return type of os.date is now inferred as DateTypeResult when argument is “t” or “!t”

Runtime improvementsPermalink

Out-of-memory exception handling has been improved. xpcall handlers will now actually be called with a “not enough memory” string and whatever string/object they return will be correctly propagated.

Other runtime improvements we’ve made: 

   Performance of table.sort was improved further. It now guarantees N*log(N) time complexity in the worst case
   Performance of table.concat was improved by ~5-7%
   Performance of math.noise was improved by ~30%
   Inlining of functions is now possible even when they used to compute their own arguments
   Improved logic for determining whether inlining a function or unrolling a loop is profitable

Autocomplete improvementsPermalink

An issue with exported types not being suggested is now fixed. Debugger improvementsPermalink

We have fixed the search for the closest executable breakpoint line.

Previously, breakpoints might have been skipped in else blocks at the end of a function. This simplified example shows the issue: 

local function foo(isIt)
    if isIt then
        print("yes")
    else
        -- When 'true' block exits the function, breakpoint couldn't be placed here
        print("no")
    end
end

ThanksPermalink

A very special thanks to all of our open source contributors: 

   Petri Häkkinen
   JohnnyMorganz
   Gael
   Jan
   Alex Orlenko
   mundusnine
   Ben Mactavsin
   RadiatedExodus
   Lodinu Kalugalage
   MagelessMayhem
   Someon1e

Updated: July 28, 2023






Luau Recap: March 2023

March 31, 2023

How the time flies! The team has been busy since the last November Luau Recap working on some large updates that are coming in the future, but before those arrive, we have some improvements that you can already use!

[Cross-posted to the Roblox Developer Forum.] Improved type refinementsPermalink

Type refinements handle constraints placed on variables inside conditional blocks.

In the following example, while variable a is declared to have type number?, inside the if block we know that it cannot be nil: 

local function f(a: number?)
    if a ~= nil then
        a *= 2 -- no type errors
    end
    ...
end

One limitation we had previously is that after a conditional block, refinements were discarded.

But there are cases where if is used to exit the function early, making the following code essentially act as a hidden else block.

We now correctly preserve such refinements and you should be able to remove assert function calls that were only used to get rid of false positive errors about types being nil. 

local function f(x: string?)
    if not x then return end

    -- x is a 'string' here
end

Throwing calls like error() or assert(false) instead of a return statement are also recognized. 

local function f(x: string?)
    if not x then error('first argument is nil') end

    -- x is 'string' here
end

Existing complex refinements like type/typeof, tagged union checks and other are expected to work as expected. Marking table.getn/foreach/foreachi as deprecatedPermalink

table.getn, table.foreach and table.foreachi were deprecated in Lua 5.1 that Luau is based on, and removed in Lua 5.2.

table.getn(x) is equivalent to rawlen(x) when ‘x’ is a table; when ‘x’ is not a table, table.getn produces an error.

It’s difficult to imagine code where table.getn(x) is better than either #x (idiomatic) or rawlen(x) (fully compatible replacement).

table.getn is also slower than both alternatives and was marked as deprecated.

table.foreach is equivalent to a for .. pairs loop; table.foreachi is equivalent to a for .. ipairs loop; both may also be replaced by generalized iteration.

Both functions are significantly slower than equivalent for loop replacements, are more restrictive because the function can’t yield.

Because both functions bring no value over other library or language alternatives, they were marked deprecated as well.

You may have noticed linter warnings about places where these functions are used. For compatibility, these functions are not going to be removed. Autocomplete improvementsPermalink

When table key type is defined to be a union of string singletons, those keys can now autocomplete in locations marked as ‘^’: 

type Direction = "north" | "south" | "east" | "west"

local a: {[Direction]: boolean} = {[^] = true}
local b: {[Direction]: boolean} = {["^"]}
local b: {[Direction]: boolean} = {^}

We also fixed incorrect and incomplete suggestions inside the header of if, for and while statements. Runtime improvementsPermalink

On the runtime side, we added multiple optimizations.

table.sort is now ~4.1x faster (when not using a predicate) and ~2.1x faster when using a simple predicate.

We also have ideas on how improve the sorting performance in the future.

math.floor, math.ceil and math.round now use specialized processor instructions. We have measured ~7-9% speedup in math benchmarks that heavily used those functions.

A small improvement was made to builtin library function calls, getting a 1-2% improvement in code that contains a lot of fastcalls.

Finally, a fix was made to table array part resizing that brings large improvement to performance of large tables filled as an array, but at an offset (for example, starting at 10000 instead of 1).

Aside from performance, a correctness issue was fixed in multi-assignment expressions. 

arr[1], n = n, n - 1

In this example, n - 1 was assigned to n before n was assigned to arr[1]. This issue has now been fixed. Analysis improvementsPermalink

Multiple changes were made to improve error messages and type presentation. 

   Table type strings are now shown with newlines, to make them easier to read
   Fixed unions of nil types displaying as a single ? character
   “Type pack A cannot be converted to B” error is not reported instead of a cryptic “Failed to unify type packs”
   Improved error message for value count mismatch in assignments like local a, b = 2

You may have seen error messages like Type 'string' cannot be converted to 'string?' even though usually it is valid to assign local s: string? = 'hello' because string is a sub-type of string?.

This is true in what is called Covariant use contexts, but doesn’t hold in Invariant use contexts, like in the example below: 

local a: { x: Model }
local b: { x: Instance } = a -- Type 'Model' could not be converted into 'Instance' in an invariant context

In this example, while Model is a sub-type of Instance and can be used where Instance is required.

The same is not true for a table field because when using table b, b.x can be assigned an Instance that is not a Model. When b is an alias to a, this assignment is not compatible with a’s type annotation.

Some other light changes to type inference include: 

   string.match and string.gmatch are now defined to return optional values as match is not guaranteed at runtime
   Added an error when unrelated types are compared with ==/~=
   Fixed issues where variable after typeof(x) == 'table' could not have been used as a table

ThanksPermalink

A very special thanks to all of our open source contributors: 

   niansa/tuxifan
   B. Gibbons
   Epix
   Harold Cindy
   Qualadore

Updated: March 31, 2023






String Interpolation

February 2, 2023

String interpolation is the new syntax introduced to Luau that allows you to create a string literal with expressions inside of that string literal.

In short, it’s a safer and more ergonomic alternative over string.format.

Here’s a quick example of a string interpolation: 

local combos = {2, 7, 1, 8, 5}
print(`The lock combination is {table.concat(combos)}. Again, {table.concat(combos, ", ")}.`)
--> The lock combination is 27185. Again, 2, 7, 1, 8, 5.

String interpolation also composes well with the __tostring metamethod. 

local balance = setmetatable({ value = 500 }, {
    __tostring = function(self)
        return "$" .. tostring(self.value)
    end
})

print(`You have {balance}!`)
--> You have $500!

To find out more details about this feature, check out Luau Syntax page.

This is also the first major language feature implemented in a contribution from the open-source community. Thanks Kampfkarren!

Updated: February 2, 2023






Luau Recap: November 2022

November 30, 2022

While the team is busy to bring some bigger things in the future, we have made some small improvements this month.

[Cross-posted to the Roblox Developer Forum.] Analysis improvementsPermalink

We have improved tagged union type refinements to only include unhandled type cases in the else branch of the if statement: 

type Ok<T> = { tag: "ok", value: T }
type Err = { tag: "error", msg: string }
type Result<T> = Ok<T> | Err

function unwrap<T>(r: Result<T>): T?
    if r.tag == "ok" then
        return r.value
    else
        -- Luau now understands that 'r' here can only be the 'Err' part
        print(r.msg)
        return nil
    end
end

For better inference, we updated the definition of Enum.SomeType:GetEnumItems() to return {Enum.SomeType} instead of common {EnumItem} and the return type of next function now includes the possibility of key being nil.

Finally, if you use and operator on non-boolean values, boolean type will no longer be added by the type inference: 

local function f1(a: number?)
    -- 'x' is still a 'number?' and doesn't become 'boolean | number'
    local x = a and 5
end

Error message improvementsPermalink

We now give an error when built-in types are being redefined: 

type string = number -- Now an error: Redefinition of type 'string'

We also had a parse error missing in case you forgot your default type pack parameter value. We accepted the following code silently without raising an issue: 

type Foo<T... = > = nil -- Now an error: Expected type, got '>'

Error about function argument count mismatch no longer points at the last argument, but instead at the function in question. So, instead of: 

function myfunction(a: number, b:number) end
myfunction(123)
           ---

We now highlight this: 

function myfunction(a: number, b:number) end
myfunction(123)
----------

If you iterate over a table value that could also be nil, you get a better explanation in the error message: 

local function f(t: {number}?)
    for i,v in t do -- Value of type {number}? could be nil
        --...
    end
end

Previously it was Cannot call non-function {number}? which was confusing.

And speaking of confusing, some of you might have seen an error like Type 'string' could not be converted into 'string'.

This was caused by Luau having both a primitive type string and a table type coming from string library. Since the way you can get the type of the string library table is by using typeof(string), the updated error message will mirror that and report Type 'string' could not be converted into 'typeof(string)'.

Parsing now recovers with a more precise error message if you forget a comma in table constructor spanning multiple lines: 

local t = {
    a = 1
    b = 2 -- Expected ',' after table constructor element
    c = 3 -- Expected ',' after table constructor element
}

Updated: November 30, 2022






Luau origins and evolution

November 4, 2022

At the heart of Roblox technology lies Luau, a scripting language derived from Lua 5.1 that is being developed by an internal team of programming language experts with the help of open source contributors.

It powers all user-generated content on Roblox, providing access to a very rich set of APIs that allows manipulation of objects in the 3D world, backend API access, UI interaction and more. Hundreds of thousands of developers write code in Luau every month, with top experiences using hundreds of thousands of lines of code, adding up to hundreds of millions of lines of code across the platform. For many of them, it is the first programming language they learn, and one they spend the majority of their time programming in. Using a set of extended APIs developers also customize their workflows by writing plugins to Roblox Studio, where they work on their experiences, using an extended API surface to interact with all aspects of the editor.

It also powers a lot of application code that Roblox engineers are writing: Universal App, the gateway to the worlds of Roblox that is used by tens of millions of people every day, has 95% of its functionality implemented in Luau, and Roblox Studio has a lot of builtin critical functionality such as part and terrain editors, marketplace browser, avatar and animation editors, material manager and more, implemented in Luau as a plugin, mostly using the same APIs that developers have access to. Every week, updates to this internal codebase that is now over 2 million lines large, are shipped to all Roblox users.

In addition to Roblox use cases, Luau is also open-source and is seeing an increased adoption in other projects and applications.

But why did we use Lua in the first place, and why did we decide to pursue building a new language on top of it? Early beginningsPermalink

Around 2006, when a very early version of the Roblox platform was developed, the question of user generated behaviors emerged. Before that, users were able to build non-interactive content on Roblox, and the only form of interaction was physics simulation. While this provided rich emergent behavior, it was hard to build gameplay on top of this: for example, to build a Capture The Flag game, you need to handle collision between players and flags spread throughout the map with a bit of logic that dictates how to adjust team points and when to remove or recreate the objects.

After an early and brief misstep when we decided to add a few gameplay objects to the core definition of Roblox worlds (some developers may recognize FlagStand as a class name…), the Roblox co-founder Erik Cassel realized that an approach like this is fundamentally limiting the power of user generated content. It’s not enough to give creators the basic blocks on top of which to build their creations, it’s critical to expose the power of a full Turing-complete programming language. Without this, the expressive capability and the reach of the platform would have been restricted far too much.

But which programming language to choose? This is where Lua, which was, and still is, one of the dominant programming languages used in video games, comes in.

In addition to its simplicity, which made the language easy to learn and get productive in, Lua was the fastest scripting language compared to popular alternatives like Python or JavaScript at the time1, designed to be embedded which meant an easy ability to expose APIs from the host application to the scripts as well as high degree of execution control from the host, and implemented coroutines, a very powerful concurrency primitive that allowed to easily and intuitively script behaviors for independent actors in game using linear control flow.

Instead of having a large standard library, the expectation was that the embedding application would define a set of APIs that that application needed, as well as establish policies of running the code - which gave us a lot of freedom in how to structure the APIs and when the scripts would get triggered during the simulation of a single frame. Power of simplicityPermalink

Lua is a simple language. What does simplicity mean for us?

Being a simple language means having a small set of features. Lua has all the fundamental features but doesn’t have a lot of syntax sugar - this means the language is easier to teach and learn, and you rarely run into code that’s difficult to understand syntactically because it uses an unfamiliar construct. Of course, this also means that some programs in Lua are longer than equivalent programs in languages that have more dedicated constructs to solve specific problems, such as list comprehensions in Python.

Being a simple language means having a minimal set of rules for every feature. Lua does deviate from this in certain respects (which is to say, the language could have been even simpler!), but notably for a dynamic language the behavior of fundamental operators is generally easy to explain and unsurprising - for example, two values in Lua are equal iff they have the same type and the same value, as such 0 == “0” is false; as another example, for loops introduce unique variable bindings on every iteration, as such capturing the iteration variable in a closure produces unique values. These decisions lead to more concise and efficient implementation and eliminate a class of bugs in programs.

Being a simple language means having a small implementation. This may be immaterial to people writing code in the language, but it leads to an implementation that can be of higher quality; simpler implementations can also be easier to optimize for memory or performance, and are easier to build upon.

Developers on the Roblox platform have very diverse programming backgrounds. Some are writing their first line of code in Roblox Studio, while others have computer science degrees and experience working in multiple different programming languages. While it’s always possible to support two different programming languages that target different segments of the audience, that fragments the ecosystem and makes the programming story less consistent (impacting documentation, tutorials, code reuse, ability for community members to help each other write code, presents challenges with interaction between different languages in the same experience and more). A better outcome is one where a single language can serve both audiences - this requires a language that strikes a balance between simplicity and generality, and while Lua isn’t perfect here, it’s great as a foundation for a language like this2.

In many ways, Lua is simultaneously simple and pragmatic: many parts of the language are difficult to make much better without a lot of added complexity, but at the same time it requires little in the way of extra functionality to be able to solve problems efficiently. That said, no language is perfect, and within several areas of Lua we felt that the tradeoffs weren’t quite right for our use case. Respectful evolutionPermalink

In 2019, we decided to build Luau - a language derived from Lua and compatible with Lua 5.1, which is the version we’ve been using all these years. At the time we evaluated other routes, but ultimately settled on this as the most optimal long-term.

On one hand, we loved a lot of things about Lua - both design wise and implementation wise, while there were some decisions we felt were suboptimal, by and large it was an almost perfect foundation for what we’ve set out to achieve.

On the other hand, we’ve been running into the limitations of Lua on large code bases in absence of type checking, performance was good but not great, and some missing features would have been great to have.

Some of the things we’ve been missing have been added in later versions of Lua, yet we were still using Lua 5.1. While we would have loved to use a later version of the language standard, Lua 5.x releases are not backwards compatible, and some releases remove support for features that are in wide use at Roblox. For Roblox, backwards compatibility is an essential feature of the platform - while we don’t have a guarantee that content created 10 years ago still works, to the extent that we can achieve that without restricting the platform evolution too much, we try.

What we’ve realized is that Lua is a great foundation for a perfect language that we can build for Roblox.

We would maintain backwards compatibility with Lua 5.1 but evolve the language from there; sometimes this means taking later features from Lua that don’t conflict with the existing language or our design values, sometimes this means innovating beyond what Lua has done. Crucially, we must maintain the balance between simplicity and power - we still value simplicity, we still need to avoid a feature explosion to ensure that the features compose and are of high quality, and we still need the language to be a good fit for beginners.

One of the largest limitations that we’ve seen is the lack of type checking making it easy to make mistakes in large code bases, as such support for type checking was a requirement for Luau. However, it’s important that the type checker is mostly transparent to the developers who don’t want to invest the time to learn it - anything else would change the learning curve too much for the language to be suitable for beginners. As such, we’ve investing in gradual typing, and our type checker is learning to strike a balance between inferring useful types for completely untyped programs (which, among other things, greatly enhances editing experience through type-aware autocomplete), and the lack of false positive diagnostics that can be confusing and distracting.

While we did need to introduce extra syntax to the language - most notably, to support optional type annotations - it was important for us to maintain the cohesion of the overall syntax. We aren’t seeking to make a new language with a syntax alien to Lua programmers - Luau programs are still recognizably Lua, and to the extent possible we try to avoid new syntactic features. In a sense, we still want the syntax, semantics, and the runtime to be simple and minimal - but at the same time we have important problems to solve with respect to ergonomics, robustness and performance of the language, and solving some of them requires having slightly more complex syntax, semantics, or implementation.

So in finding ways to evolve Luau, we strive to design features that feel like they would be at home in Lua. At the same time, we’ve adopted a more open evolution process - the language development is driven through RFCs that are designs open to the public that anyone can contribute to - this is in contrast with Lua, which has a very closed development process, and is one of the reasons why it would have been difficult for us to keep using Lua as we wouldn’t get a say in its development. At the same time, to ensure the design criterias are met, it’s important that the Luau development team at Roblox maintains a final say over design and implementation of the language3, while taking the community’s proposals and input into consideration. Importance of co-designPermalink

Luau language is developed in concert with the language compiler, runtime, type checker and other analysis tools, autocomplete engine and other tooling, and that development is guided by the vast volume of existing Luau code, both internal and external.

This is one of the key principles behind our evolution philosophy - neither layer is developed in isolation, and instead concerns at every level inform all other aspects of the language design and implementation.

This means that when designing language features, we make sure that they can be implemented efficiently, type checked properly, can be supported well in editing and analysis tools and have a positive impact on the code internal and external engineers write. When we find issues in any component, we can always ask, what changes to other components or even language design would make for a better overall solution.

This avoids some classes of design problems, for example we won’t specify a language feature that has a prohibitively high implementation cost, as it violates our simplicity criteria, or that is impractical to implement efficiently, as that would create a performance hazard. This also means that when implementing various components of the language we cross-check the concerns and applicability of these across the entire stack - for example, we’ve reworked our auto-complete system to use the same type inference engine that the type checking / analysis tools use, which had immense benefits for the experience of editing code, but also applied significant back pressure on the type inference itself, forcing us to improve it substantially and fix a lot of corner cases that would otherwise have lingered unnoticed.

Whenever we develop features, optimizations, improve our analysis engine or enhance the standard libraries, we also heavily rely on code written in Luau to validate our hypotheses. When working on new features we find motivation in the real problems that we see our developers face. For example, we implemented the new ternary operator after seeing a large set of cases where existing Lua’s a and b or c pattern was error-prone for boolean values, which made it easy to accidentally introduce a mistake that was hard to identify automatically. All optimizations and new analysis features are validated on our internal 2M LOC codebase before being added to Luau, which allows us to quickly get initial validation of ideas, or invalidate some approaches as infeasible / unhelpful.

In addition to that, while we don’t have direct access to community-developed source code for privacy reasons, we can run experiments and collect telemetry4, which also helps us make decisions regarding backwards compatibility. Due to Hyrum’s law, technically any change in the language or libraries, no matter how small, would be backwards incompatible - instead we adopt the notion of pragmatic balance between strict backwards compatibility5 and pragmatic compatibility concerns. For example, later versions of Lua make some library functions like table.insert/table.remove more strict with how they handle out of range indices. We have evaluated this change for compatibility by collecting telemetry on the use of out of range indices in these functions on the Roblox platform and concluded that applying the stricter checking would break existing programs, and instead had to slightly adjust the rules for out of range behavior in ways that was benign for existing code but prevented catastrophic performance degradation for large out of range indices. Because we couldn’t afford to introduce new runtime errors in this case, we also added a set of linting rules to our analysis engine to flag potential misuse of table.insert/table.remove before the code ever gets to run - this diagnostics is informational and as such doesn’t affect backwards compatibility, but does help prevent mistakes.

There are also cases where this co-design approach prevents introduction of features that can lead to easy misuse, which can be difficult to see in the design of the feature itself, but becomes more apparent when you consider features in context of the entire ecosystem. This is a good thing - it means co-design acts as a forcing function on the language simplicity and makes it easier to flag potential bad interactions between different language features, or language features and tooling, or language features and existing programming patterns that are in widespread use in real-world code. By making sure that all features are validated for their impact across the stack and on code written in Luau, we ultimately get a better, simpler and more cohesive language. Efficient executionPermalink

One of the critical goals in front of Luau is efficiency, both from the performance and memory perspective. There’s only so many milliseconds in a frame, and we simultaneously see the need to increase the scale and complexity of simulated experiences, which requires more memory and computation, as well as the need to fit more comfortably into smaller budgets of performance memory for better experience on smaller devices. In fact, one of the motivations for Luau in 2019 has been improved performance, as we saw many opportunities to go beyond Lua with a redesigned implementation.

Crucially, our performance needs are somewhat unique and require somewhat unique solutions.

We need Luau to run on many platforms where native code generation is either prohibited by the platform vendor or impractical due to tight memory constraints. As such, in terms of execution performance it’s critical that we have a very fast interpreter6. However, we have freedom in terms of high level design of the entire stack - for example, clients never see the source code of the scripts as all compilation to bytecode happens on the server; this gives us an opportunity to perform more involved and expensive optimizations during that process as well as have the smallest possible startup time on the client without complex pre-parse steps. Notably, our bytecode compiler performs a series of high level optimizations including function inlining and loop unrolling that in other dynamic languages is often left to the just-in-time compiler.

Another area where performance is critical is garbage collection. Garbage collection is crucial for the language’s simplicity as it makes memory management easier to reason about, but it does require a substantial amount of implementation effort to keep it efficient. For Roblox and for any other game engine or interactive simulation, latency is critical and so our collector is heavily optimized for that - to the extent possible collection is incremental and stop-the-world pauses are very brief. Another part of the performance story here however is the language and data structure design - by making sure that core data types are efficient in how they are laid out in memory we reduce the amount of work garbage collector takes to trace the heap, and, as another example of co-design, we try to make sure that language features are conscious of the impact they have on memory and garbage collection efficiency.

However, from a whole-platform standpoint there’s a lot of performance aspects that go beyond single-threaded execution. This is an active area of research and development for the team, as to really leverage the hardware the code is running on we need to think about SIMD, hardware thread utilization as well as running code in a cluster of nodes. These considerations inform current and future development of the runtime and the language (for example, our runtime now supports efficient operations on short SIMD vectors even in interpreted mode, and the VM is fairly lightweight to instantiate which makes running many VMs per core practical, with message passing or access to shared Roblox data model used to make gameplay features feasible to implement), but we’re definitely in the early days here - our first implementation of parallel script execution in Roblox just shipped earlier this year. This is likely the area where a lot of future innovations will happen as well. FuturePermalink

We’re very happy with the success of Luau - in several years we’ve established consistent processes for evolving the language and so far we found a good balance between simplicity, ease of use, performance and robustness of the language, its implementation and the tooling surrounding it. The language keeps continuously evolving but at a pace that is easy to stay on top of - in 2022 we shipped a few syntactic extensions for type annotations but no changes to the syntax of the language outside of types, and only one major semantic change to the for loop iteration that actually made the language easier to use by avoiding the need to specify the table traversal style via pairs/ipairs. We try to make sure that the features are general and provide enough extensibility so that libraries can be built on top of the language to make it easier to write code, while also making it practical to use the language without complex supporting frameworks.

There’s still a lot of ground to cover, and we’ll be working on Luau for years to come. We’re in the process of building the next version of our type inference / checking engine to make sure that all users of the language regardless of their expertise benefit from it, we’ve started investing in native code generation as we’re reaching the limits of interpreted performance (although some exciting opportunities for compiler optimization are still on the horizon), and there’s still a lot of hard design and implementation work ahead of us for some important language features and standard libraries. And as mentioned, our execution model will likely see a lot of innovation as we push the boundaries of hardware utilization across cores and nodes.

Overall, Luau is like an iceberg - the surface is simple to learn and use, but it hides the tremendous amount of careful design, engineering and attention to detail, and we plan to continue to invest in it while trying to keep the outer surface comparatively small. We’re excited to see how far we can take it! 

   High-performance JavaScript engines didn’t exist at the time! LuaJIT was around the corner and redefined the performance expectations of dynamic languages. ↩

   In fact, scaling to large teams of expert programmers is one of the core motivations behind our creating Luau, while a requirement to still be suitable for beginner programmers guides our evolution direction. ↩

   This would have been difficult to drive in any existing large established language like JavaScript or Python. ↩

   This is limited to Roblox platform and doesn’t exist in open-source releases. ↩

   Which we do follow in some areas, such as syntactic compatibility - all existing programs that parse must continue to parse the same way as the language evolves. ↩

   Some design decisions and implementation techniques are documented on our performance page. ↩

Updated: November 4, 2022






Luau Recap: September & October 2022

November 1, 2022

Luau is our new language that you can read more about at https://luau-lang.org.

[Cross-posted to the Roblox Developer Forum.] Semantic subtypingPermalink

One of the most important goals for Luau is to avoid false positives, that is cases where Script Analysis reports a type error, but in fact the code is correct. This is very frustrating, especially for beginners. Spending time chasing down a gnarly type error only to discover that it was the type system that’s wrong is nobody’s idea of fun!

We are pleased to announce that a major component of minimizing false positives has landed, semantic subtyping, which removes a class of false positives caused by failures of subtyping. For example, in the program 

  local x : CFrame = CFrame.new()
  local y : Vector3 | CFrame
  if (math.random()) then
    y = CFrame.new()
  else
    y = Vector3.new()
  end
  local z : Vector3 | CFrame = x * y -- Type Error!

an error is reported, even though there is no problem at runtime. This is because CFrame’s multiplication has two overloads: 

    ((CFrame, CFrame) -> CFrame)
  & ((CFrame, Vector3) -> Vector3)

The current syntax-driven algorithm for subtyping is not sophisticated enough to realize that this is a subtype of the desired type: 

  (CFrame, Vector3 | CFrame) -> (Vector3 | CFrame)

Our new algorithm is driven by the semantics of subtyping, not the syntax of types, and eliminates this class of false positives.

If you want to know more about semantic subtyping in Luau, check out our technical blog post on the subject. Other analysis improvementsPermalink 

   Improve stringification of function types.
   Improve parse error warnings in the case of missing tokens after a comma.
   Improve typechecking of expressions involving variadics such as { ... }.
   Make sure modules don’t return unbound generic types.
   Improve cycle detection in stringifying types.
   Improve type inference of combinations of intersections and generic functions.
   Improve typechecking when calling a function which returns a variadic e.g. () -> (number...).
   Improve typechecking when passing a function expression as a parameter to a function.
   Improve error reporting locations.
   Remove some sources of memory corruption and crashes.

Other runtime and debugger improvementsPermalink 

   Improve performance of accessing debug info.
   Improve performance of getmetatable and setmetatable.
   Remove a source of freezes in the debugger.
   Improve GC accuracy and performance.

ThanksPermalink

Thanks for all the contributions! 

   AllanJeremy
   JohnnyMorganz
   jujhar16
   petrihakkinen

Updated: November 1, 2022






Semantic Subtyping in Luau

October 31, 2022

Luau is the first programming language to put the power of semantic subtyping in the hands of millions of creators. Minimizing false positivesPermalink

One of the issues with type error reporting in tools like the Script Analysis widget in Roblox Studio is false positives. These are warnings that are artifacts of the analysis, and don’t correspond to errors which can occur at runtime. For example, the program 

  local x = CFrame.new()
  local y
  if (math.random()) then
    y = CFrame.new()
  else
    y = Vector3.new()
  end
  local z = x * y

reports a type error which cannot happen at runtime, since CFrame supports multiplication by both Vector3 and CFrame. (Its type is ((CFrame, CFrame) -> CFrame) & ((CFrame, Vector3) -> Vector3).)

False positives are especially poor for onboarding new users. If a type-curious creator switches on typechecking and is immediately faced with a wall of spurious red squiggles, there is a strong incentive to immediately switch it off again.

Inaccuracies in type errors are inevitable, since it is impossible to decide ahead of time whether a runtime error will be triggered. Type system designers have to choose whether to live with false positives or false negatives. In Luau this is determined by the mode: strict mode errs on the side of false positives, and nonstrict mode errs on the side of false negatives.

While inaccuracies are inevitable, we try to remove them whenever possible, since they result in spurious errors, and imprecision in type-driven tooling like autocomplete or API documentation. Subtyping as a source of false positivesPermalink

One of the sources of false positives in Luau (and many other similar languages like TypeScript or Flow) is subtyping. Subtyping is used whenever a variable is initialized or assigned to, and whenever a function is called: the type system checks that the type of the expression is a subtype of the type of the variable. For example, if we add types to the above program 

  local x : CFrame = CFrame.new()
  local y : Vector3 | CFrame
  if (math.random()) then
    y = CFrame.new()
  else
    y = Vector3.new()
  end
  local z : Vector3 | CFrame = x * y

then the type system checks that the type of CFrame multiplication is a subtype of (CFrame, Vector3 | CFrame) -> (Vector3 | CFrame).

Subtyping is a very useful feature, and it supports rich type constructs like type union (T | U) and intersection (T & U). For example, number? is implemented as a union type (number | nil), inhabited by values that are either numbers or nil.

Unfortunately, the interaction of subtyping with intersection and union types can have odd results. A simple (but rather artificial) case in older Luau was: 

  local x : (number?) & (string?) = nil
  local y : nil = nil
  y = x -- Type '(number?) & (string?)' could not be converted into 'nil'
  x = y

This error is caused by a failure of subtyping, the old subtyping algorithm reports that (number?) & (string?) is not a subtype of nil. This is a false positive, since number & string is uninhabited, so the only possible inhabitant of (number?) & (string?) is nil.

This is an artificial example, but there are real issues raised by creators caused by the problems, for example https://devforum.roblox.com/t/luau-recap-july-2021/1382101/5. Currently, these issues mostly affect creators making use of sophisticated type system features, but as we make type inference more accurate, union and intersection types will become more common, even in code with no type annotations.

This class of false positives no longer occurs in Luau, as we have moved from our old approach of syntactic subtyping to an alternative called semantic subtyping. Syntactic subtypingPermalink

AKA “what we did before.”

Syntactic subtyping is a syntax-directed recursive algorithm. The interesting cases to deal with intersection and union types are: 

   Reflexivity: T is a subtype of T
   Intersection L: (T₁ & … & Tⱼ) is a subtype of U whenever some of the Tᵢ are subtypes of U
   Union L: (T₁ | … | Tⱼ) is a subtype of U whenever all of the Tᵢ are subtypes of U
   Intersection R: T is a subtype of (U₁ & … & Uⱼ) whenever T is a subtype of all of the Uᵢ
   Union R: T is a subtype of (U₁ | … | Uⱼ) whenever T is a subtype of some of the Uᵢ.

For example: 

   By Reflexivity: nil is a subtype of nil
   so by Union R: nil is a subtype of number?
   and: nil is a subtype of string?
   so by Intersection R: nil is a subtype of (number?) & (string?).

Yay! Unfortunately, using these rules: 

   number isn’t a subtype of nil
   so by Union L: (number?) isn’t a subtype of nil
   and: string isn’t a subtype of nil
   so by Union L: (string?) isn’t a subtype of nil
   so by Intersection L: (number?) & (string?) isn’t a subtype of nil.

This is typical of syntactic subtyping: when it returns a “yes” result, it is correct, but when it returns a “no” result, it might be wrong. The algorithm is a conservative approximation, and since a “no” result can lead to type errors, this is a source of false positives. Semantic subtypingPermalink

AKA “what we do now.”

Rather than thinking of subtyping as being syntax-directed, we first consider its semantics, and later return to how the semantics is implemented. For this, we adopt semantic subtyping: 

   The semantics of a type is a set of values.
   Intersection types are thought of as intersections of sets.
   Union types are thought of as unions of sets.
   Subtyping is thought of as set inclusion.

For example: Type Semantics number { 1, 2, 3, … } string { “foo”, “bar”, … } nil { nil } number? { nil, 1, 2, 3, … } string? { nil, “foo”, “bar”, … } (number?) & (string?) { nil, 1, 2, 3, … } ∩ { nil, “foo”, “bar”, … } = { nil }

and since subtypes are interpreted as set inclusions: Subtype Supertype Because nil number? { nil } ⊆ { nil, 1, 2, 3, … } nil string? { nil } ⊆ { nil, “foo”, “bar”, … } nil (number?) & (string?) { nil } ⊆ { nil } (number?) & (string?) nil { nil } ⊆ { nil }

So according to semantic subtyping, (number?) & (string?) is equivalent to nil, but syntactic subtyping only supports one direction.

This is all fine and good, but if we want to use semantic subtyping in tools, we need an algorithm, and it turns out checking semantic subtyping is non-trivial. Semantic subtyping is hardPermalink

NP-hard to be precise.

We can reduce graph coloring to semantic subtyping by coding up a graph as a Luau type such that checking subtyping on types has the same result as checking for the impossibility of coloring the graph

For example, coloring a three-node, two color graph can be done using types: 

type Red = "red"
type Blue = "blue"
type Color = Red | Blue
type Coloring = (Color) -> (Color) -> (Color) -> boolean
type Uncolorable = (Color) -> (Color) -> (Color) -> false

Then a graph can be encoded as an overload function type with subtype Uncolorable and supertype Coloring, as an overloaded function which returns false when a constraint is violated. Each overload encodes one constraint. For example a line has constraints saying that adjacent nodes cannot have the same color: 

type Line = Coloring
  & ((Red) -> (Red) -> (Color) -> false)
  & ((Blue) -> (Blue) -> (Color) -> false)
  & ((Color) -> (Red) -> (Red) -> false)
  & ((Color) -> (Blue) -> (Blue) -> false)

A triangle is similar, but the end points also cannot have the same color: 

type Triangle = Line
  & ((Red) -> (Color) -> (Red) -> false)
  & ((Blue) -> (Color) -> (Blue) -> false)

Now, Triangle is a subtype of Uncolorable, but Line is not, since the line can be 2-colored. This can be generalized to any finite graph with any finite number of colors, and so subtype checking is NP-hard.

We deal with this in two ways: 

   we cache types to reduce memory footprint, and
   give up with a “Code Too Complex” error if the cache of types gets too large.

Hopefully this doesn’t come up in practice much. There is good evidence that issues like this don’t arise in practice from experience with type systems like that of Standard ML, which is EXPTIME-complete, but in practice you have to go out of your way to code up Turing Machine tapes as types. Type normalizationPermalink

The algorithm used to decide semantic subtyping is type normalization. Rather than being directed by syntax, we first rewrite types to be normalized, then check subtyping on normalized types.

A normalized type is a union of: 

   a normalized nil type (either never or nil)
   a normalized number type (either never or number)
   a normalized boolean type (either never or true or false or boolean)
   a normalized function type (either never or an intersection of function types) etc

Once types are normalized, it is straightforward to check semantic subtyping.

Every type can be normalized (sigh, with some technical restrictions around generic type packs). The important steps are: 

   removing intersections of mismatched primitives, e.g. number & bool is replaced by never, and
   removing unions of functions, e.g. ((number?) -> number) | ((string?) -> string) is replaced by (nil) -> (number | string).

For example, normalizing (number?) & (string?) removes number & string, so all that is left is nil.

Our first attempt at implementing type normalization applied it liberally, but this resulted in dreadful performance (complex code went from typechecking in less than a minute to running overnight). The reason for this is annoyingly simple: there is an optimization in Luau’s subtyping algorithm to handle reflexivity (T is a subtype of T) that performs a cheap pointer equality check. Type normalization can convert pointer-identical types into semantically-equivalent (but not pointer-identical) types, which significantly degrades performance.

Because of these performance issues, we still use syntactic subtyping as our first check for subtyping, and only perform type normalization if the syntactic algorithm fails. This is sound, because syntactic subtyping is a conservative approximation to semantic subtyping. Pragmatic semantic subtypingPermalink

Off-the-shelf semantic subtyping is slightly different from what is implemented in Luau, because it requires models to be set-theoretic, which requires that inhabitants of function types “act like functions.” There are two reasons why we drop this requirement.

Firstly, we normalize function types to an intersection of functions, for example a horrible mess of unions and intersections of functions: 

((number?) -> number?) | (((number) -> number) & ((string?) -> string?))

normalizes to an overloaded function: 

((number) -> number?) & ((nil) -> (number | string)?)

Set-theoretic semantic subtyping does not support this normalization, and instead normalizes functions to disjunctive normal form (unions of intersections of functions). We do not do this for ergonomic reasons: overloaded functions are idiomatic in Luau, but DNF is not, and we do not want to present users with such non-idiomatic types.

Our normalization relies on rewriting away unions of function types: 

((A) -> B) | ((C) -> D)   →   (A & C) -> (B | D)

This normalization is sound in our model, but not in set-theoretic models.

Secondly, in Luau, the type of a function application f(x) is B if f has type (A) -> B and x has type A. Unexpectedly, this is not always true in set-theoretic models, due to uninhabited types. In set-theoretic models, if x has type never then f(x) has type never. We do not want to burden users with the idea that function application has a special corner case, especially since that corner case can only arise in dead code.

In set-theoretic models, (never) -> A is a subtype of (never) -> B, no matter what A and B are. This is not true in Luau.

For these two reasons (which are largely about ergonomics rather than anything technical) we drop the set-theoretic requirement, and use pragmatic semantic subtyping. Negation typesPermalink

The other difference between Luau’s type system and off-the-shelf semantic subtyping is that Luau does not support all negated types.

The common case for wanting negated types is in typechecking conditionals: 

-- initially x has type T
if (type(x) == "string") then
  --  in this branch x has type T & string
else
  -- in this branch x has type T & ~string
end

This uses a negated type ~string inhabited by values that are not strings.

In Luau, we only allow this kind of typing refinement on test types like string, function, Part and so on, and not on structural types like (A) -> B, which avoids the common case of general negated types. Prototyping and verificationPermalink

During the design of Luau’s semantic subtyping algorithm, there were changes made (for example initially we thought we were going to be able to use set-theoretic subtyping). During this time of rapid change, it was important to be able to iterate quickly, so we initially implemented a prototype rather than jumping straight to a production implementation.

Validating the prototype was important, since subtyping algorithms can have unexpected corner cases. For this reason, we adopted Agda as the prototyping language. As well as supporting unit testing, Agda supports mechanized verification, so we are confident in the design.

The prototype does not implement all of Luau, just the functional subset, but this was enough to discover subtle feature interactions that would probably have surfaced as difficult-to-fix bugs in production.

Prototyping is not perfect, for example the main issues that we hit in production were about performance and the C++ standard library, which are never going to be caught by a prototype. But the production implementation was otherwise fairly straightforward (or at least as straightforward as a 3kLOC change can be). Next stepsPermalink

Semantic subtyping has removed one source of false positives, but we still have others to track down: 

   overloaded function applications and operators,
   property access on expressions of complex type,
   read-only properties of tables,
   variables that change type over time (aka typestates),
   …

The quest to remove spurious red squiggles continues! AcknowledgmentsPermalink

Thanks to Giuseppe Castagna and Ben Greenman for helpful comments on drafts of this post. Further readingPermalink

If you want to find out more about Luau and semantic subtyping, you might want to check out… 

   Luau. https://luau-lang.org/
   Lily Brown, Andy Friesen and Alan Jeffrey, Goals of the Luau Type System, Human Aspects of Types and Reasoning Assistants (HATRA), 2021. https://arxiv.org/abs/2109.11397
   Luau Typechecker Prototype. https://github.com/luau-lang/agda-typeck
   Agda. https://agda.readthedocs.io/
   Andrew M. Kent. Down and Dirty with Semantic Set-theoretic Types, 2021. https://pnwamk.github.io/sst-tutorial/
   Giuseppe Castagna, Covariance and Contravariance, Logical Methods in Computer Science 16(1), 2022. https://arxiv.org/abs/1809.01427
   Giuseppe Castagna and Alain Frisch, A gentle introduction to semantic subtyping, Proc. Principles and practice of declarative programming (PPDP), pp 198–208, 2005. https://doi.org/10.1145/1069774.1069793
   Giuseppe Castagna, Mickaël Laurent, Kim Nguyễn, Matthew Lutze, On Type-Cases, Union Elimination, and Occurrence Typing, Principles of Programming Languages (POPL), 2022. https://doi.org/10.1145/3498674
   Giuseppe Castagna, Programming with union, intersection, and negation types, 2022. https://arxiv.org/abs/2111.03354
   Sam Tobin-Hochstadt and Matthias Felleisen, Logical types for untyped languages. International Conference on Functional Programming (ICFP), 2010. https://doi.org/10.1145/1863543.1863561
   José Valim, My Future with Elixir: set-theoretic types, 2022. https://elixir-lang.org/blog/2022/10/05/my-future-with-elixir-set-theoretic-types/

Some other languages which support semantic subtyping… 

   ℂDuce https://www.cduce.org/
   Ballerina https://ballerina.io
   Elixir https://elixir-lang.org/
   eqWAlizer https://github.com/WhatsApp/eqwalizer

And if you want to see the production code, it’s in the C++ definitions of tryUnifyNormalizedTypes and NormalizedType in the open source Luau repo.

Updated: October 31, 2022






Luau Recap: July & August 2022

August 29, 2022

Luau is our new language that you can read more about at https://luau-lang.org.

[Cross-posted to the Roblox Developer Forum.] Tables now support __len metamethodPermalink

See the RFC Support __len metamethod for tables and rawlen function for more details.

With generalized iteration released in May, custom containers are easier than ever to use. The only thing missing was the fact that tables didn’t respect __len.

Simply, tables now honor the __len metamethod, and rawlen is also added with similar semantics as rawget and rawset:

local my_cool_container = setmetatable({ items = { 1, 2 } }, { 

   __len = function(self) return #self.items end

})

print(#my_cool_container) --> 2 print(rawlen(my_cool_container)) --> 0

never and unknown typesPermalink

See the RFC never and unknown types for more details.

We’ve added two new types, never and unknown. These two types are the opposites of each other by the fact that there’s no value that inhabits the type never, and the dual of that is every value inhabits the type unknown.

Type inference may infer a variable to have the type never if and only if the set of possible types becomes empty, for example through type refinements.

function f(x: string | number) 

   if typeof(x) == "string" and typeof(x) == "number" then
       -- x: never
   end

end

This is useful because we still needed to ascribe a type to x here, but the type we used previously had unsound semantics. For example, it was possible to be able to expand the domain of a variable once the user had proved it impossible. With never, narrowing a type from never yields never.

Conversely, unknown can be used to enforce a stronger contract than any. That is, unknown and any are similar in terms of allowing every type to inhabit them, and other than unknown or any, any allows itself to inhabit into a different type, whereas unknown does not.

function any(): any return 5 end function unknown(): unknown return 5 end

-- no type error, but assigns a number to x which expects string local x: string = any()

-- has type error, unknown cannot be converted into string local y: string = unknown()

To be able to do this soundly, you must apply type refinements on a variable of type unknown.

local u = unknown()

if typeof(u) == "string" then 

   local y: string = u -- no type error

end

A use case of unknown is to enforce type safety at implementation sites for data that do not originate in code, but from over the wire. Argument names in type packs when instantiating a typePermalink

We had a bug in the parser which erroneously allowed argument names in type packs that didn’t fold into a function type. That is, the below syntax did not generate a parse error when it should have.

Foo<(a: number, b: string)>

New IntegerParsing lintPermalink

See the announcement for more details. We include this here for posterity.

We’ve introduced a new lint called IntegerParsing. Right now, it lints three classes of errors: 

   Truncation of binary literals that resolves to a value over 64 bits,
   Truncation of hexadecimal literals that resolves to a value over 64 bits, and
   Double hexadecimal prefix.

For 1.) and 2.), they are currently not planned to become a parse error, so action is not strictly required here.

For 3.), this will be a breaking change! See the rollout plan for details. New ComparisonPrecedence lintPermalink

We’ve also introduced a new lint called ComparisonPrecedence. It fires in two particular cases: 

   not X op Y where op is == or ~=, or
   X op Y op Z where op is any of the comparison or equality operators.

In languages that uses ! to negate the boolean i.e. !x == y looks fine because !x visually binds more tightly than Lua’s equivalent, not x. Unfortunately, the precedences here are identical, that is !x == y is (!x) == y in the same way that not x == y is (not x) == y. We also apply this on other operators e.g. x <= y == y.

-- not X == Y is equivalent to (not X) == Y; consider using X ~= Y, or wrap one of the expressions in parentheses to silence if not x == y then end

-- not X ~= Y is equivalent to (not X) ~= Y; consider using X == Y, or wrap one of the expressions in parentheses to silence if not x ~= y then end

-- not X <= Y is equivalent to (not X) <= Y; wrap one of the expressions in parentheses to silence if not x <= y then end

-- X <= Y == Z is equivalent to (X <= Y) == Z; wrap one of the expressions in parentheses to silence if x <= y == 0 then end

As a special exception, this lint pass will not warn for cases like x == not y or not x == not y, which both looks intentional as it is written and interpreted. Function calls returning singleton types incorrectly widenedPermalink

Fix a bug where widening was a little too happy to fire in the case of function calls returning singleton types or union thereof. This was an artifact of the logic that knows not to infer singleton types in cases that makes no sense to.

function f(): "abc" | "def" 

   return if math.random() > 0.5 then "abc" else "def"

end

-- previously reported that 'string' could not be converted into '"abc" | "def"' local x: "abc" | "def" = f()

string can be a subtype of a table with a shape similar to stringPermalink

The function my_cool_lower is a function <a...>(t: t1) -> a... where t1 = {+ lower: (t1) -> a... +}.

function my_cool_lower(t) 

   return t:lower()

end

Even though t1 is a table type, we know string is a subtype of t1 because string also has lower which is a subtype of t1’s lower, so this call site now type checks.

local s: string = my_cool_lower("HI")

Other analysis improvementsPermalink 

   string.gmatch/string.match/string.find may now return more precise type depending on the patterns used
   Fix a bug where type arena ownership invariant could be violated, causing stability issues
   Fix a bug where internal type error could be presented to the user
   Fix a false positive with optionals & nested tables
   Fix a false positive in non-strict mode when using generalized iteration
   Improve autocomplete behavior in certain cases for : calls
   Fix minor inconsistencies in synthesized names for types with metatables
   Fix autocomplete not suggesting globals defined after the cursor
   Fix DeprecatedGlobal warning text in cases when the global is deprecated without a suggested alternative
   Fix an off-by-one error in type error text for incorrect use of string.format

Other runtime improvementsPermalink 

   Comparisons with constants are now significantly faster when using clang as a compiler (10-50% gains on internal benchmarks)
   When calling non-existent methods on tables or strings, foo:bar now produces a more precise error message
   Improve performance for iteration of tables
   Fix a bug with negative zero in vector components when using vectors as table keys
   Compiler can now constant fold builtins under -O2, for example string.byte("A") is compiled to a constant
   Compiler can model the cost of builtins for the purpose of inlining/unrolling
   Local reassignment i.e. local x = y :: T is free iff neither x nor y is mutated/captured
   Improve debug.traceback performance by 1.15-1.75x depending on the platform
   Fix a corner case with table assignment semantics when key didn’t exist in the table and __newindex was defined: we now use Lua 5.2 semantics and call __newindex, which results in less wasted space, support for NaN keys in __newindex path and correct support for frozen tables
   Reduce parser C stack consumption which fixes some stack overflow crashes on deeply nested sources
   Improve performance of bit32.extract/replace when width is implied (~3% faster chess)
   Improve performance of bit32.extract when field/width are constants (~10% faster base64)
   string.format now supports a new format specifier, %*, that accepts any value type and formats it using tostring rules

ThanksPermalink

Thanks for all the contributions! 

   natteko
   JohnnyMorganz
   khvzak
   Anaminus
   memery-rbx
   jaykru
   Kampfkarren
   XmiliaH
   Mactavsin

Updated: August 29, 2022






Luau Recap: June 2022

July 7, 2022

Luau is our new language that you can read more about at https://luau-lang.org.

[Cross-posted to the Roblox Developer Forum.] Lower bounds calculationPermalink

A common problem that Luau has is that it primarily works by inspecting expressions in your program and narrowing the upper bounds of the values that can inhabit particular variables. In other words, each time we see a variable used, we eliminate possible sets of values from that variable’s domain.

There are some important cases where this doesn’t produce a helpful result. Take this function for instance:

function find_first_if(vec, f) for i, e in ipairs(vec) do if f(e) then return i end end

return nil end

Luau scans the function from top to bottom and first sees the line return i. It draws from this the inference that find_first_if must return the type of i, namely number.

This is fine, but things go sour when we see the line return nil. Since we are always narrowing, we take from this line the judgement that the return type of the function is nil. Since we have already concluded that the function must return number, Luau reports an error.

What we actually want to do in this case is to take these return statements as inferences about the lower bound of the function’s return type. Instead of saying “this function must return values of type nil,” we should instead say “this function may also return values of type nil.”

Lower bounds calculation does precisely this. Moving forward, Luau will instead infer the type number? for the above function.

This does have one unfortunate consequence: If a function has no return type annotation, we will no longer ever report a type error on a return statement. We think this is the right balance but we’ll be keeping an eye on things just to be sure.

Lower-bounds calculation is larger and a little bit riskier than other things we’ve been working on so we’ve set up a beta feature in Roblox Studio to enable them. It is called “Experimental Luau language features.”

Please try it out and let us know what you think! Known bugPermalink

We have a known bug with certain kinds of cyclic types when lower-bounds calculation is enabled. The following, for instance, is known to be problematic.

type T = {T?}? -- spuriously reduces to {nil}?

We hope to have this fixed soon. All table literals now result in unsealed tablesPermalink

Previously, the only way to create a sealed table was by with a literal empty table. We have relaxed this somewhat: Any table created by a {} expression is considered to be unsealed within the scope where it was created:

local T = {} T.x = 5 -- OK

local V = {x=5} V.y = 2 -- previously disallowed. Now OK.

function mkTable() 

   return {x = 5}

end

local U = mkTable() U.y = 2 -- Still disallowed: U is sealed

Other fixesPermalink 

   Adjust indentation and whitespace when creating multiline string representations of types, resulting in types that are easier to read.
   Some small bugfixes to autocomplete
   Fix a case where accessing a nonexistent property of a table would not result in an error being reported.
   Improve parser recovery for the incorrect code function foo() -> ReturnType (the correct syntax is function foo(): ReturnType)
   Improve the parse error offered for code that improperly uses the function keyword to start a type eg type T = function
   Some small crash fixes and performance improvements

Thanks!Permalink

A very special thanks to all of our open source contributors: 

   Allan N Jeremy
   Daniel Nachun
   JohnnyMorganz
   Petri Häkkinen
   Qualadore

Updated: July 7, 2022






Luau Recap: May 2022

June 1, 2022

This month Luau team has worked to bring you a new language feature together with more typechecking improvements and bugfixes!

[Cross-posted to the Roblox Developer Forum.] Generalized iterationPermalink

We have extended the semantics of standard Lua syntax for iterating through containers, for vars in values with support for generalized iteration. In Lua, to iterate over a table you need to use an iterator like next or a function that returns one like pairs or ipairs. In Luau, you can now simply iterate over a table:

for k, v in {1, 4, 9} do 

   assert(k * k == v)

end

This works for tables but can also be customized for tables or userdata by implementing __iter metamethod. It is called before the iteration begins, and should return an iterator function like next (or a custom one):

local obj = { items = {1, 4, 9} } setmetatable(obj, { __iter = function(o) return next, o.items end })

for k, v in obj do 

   assert(k * k == v)

end

The default iteration order for tables is specified to be consecutive for elements 1..#t and unordered after that, visiting every element. Similar to iteration using pairs, modifying the table entries for keys other than the current one results in unspecified behavior. Typechecking improvementsPermalink

We have added a missing check to compare implicit table keys against the key type of the table indexer:

-- error is correctly reported, implicit keys (1,2,3) are not compatible with [string] local t: { [string]: boolean } = { true, true, false }

Rules for == and ~= have been relaxed for union types, if any of the union parts can be compared, operation succeeds:

--!strict local function compare(v1: Vector3, v2: Vector3?) 

   return v1 == v2 -- no longer an error

end

Table value type propagation now correctly works with [any] key type:

--!strict type X = {[any]: string | boolean} local x: X = { key = "str" } -- no longer gives an incorrect error

If a generic function doesn’t provide type annotations for all arguments and the return value, additional generic type parameters might be added automatically:

-- previously it was foo<T>, now it's foo<T, b>, because second argument is also generic function foo<T>(x: T, y) end

We have also fixed various issues that have caused crashes, with many of them coming from your bug reports. Linter improvementsPermalink

GlobalUsedAsLocal lint warning has been extended to notice when global variable writes always happen before their use in a local scope, suggesting that they can be replaced with a local variable:

function bar() 

   foo = 6 -- Global 'foo' is never read before being written. Consider changing it to local
   return foo

end function baz() 

   foo = 10
   return foo

end

Performance improvementsPermalink

Garbage collection CPU utilization has been tuned to further reduce frame time spikes of individual collection steps and to bring different GC stages to the same level of CPU utilization.

Returning a type-cast local (return a :: type) as well as returning multiple local variables (return a, b, c) is now a little bit more efficient. Function inlining and loop unrollingPermalink

In the open-source release of Luau, when optimization level 2 is enabled, the compiler will now perform function inlining and loop unrolling.

Only loops with loop bounds known at compile time, such as for i=1,4 do, can be unrolled. The loop body must be simple enough for the optimization to be profitable; compiler uses heuristics to estimate the performance benefit and automatically decide if unrolling should be performed.

Only local functions (defined either as local function foo or local foo = function) can be inlined. The function body must be simple enough for the optimization to be profitable; compiler uses heuristics to estimate the performance benefit and automatically decide if each call to the function should be inlined instead. Additionally recursive invocations of a function can’t be inlined at this time, and inlining is completely disabled for modules that use getfenv/setfenv functions.

Updated: June 1, 2022






Luau Recap: April 2022

May 2, 2022

Luau is our new language that you can read more about at https://luau-lang.org.

[Cross-posted to the Roblox Developer Forum.]

It’s been a bit of a quiet month. We mostly have small optimizations and bugfixes for you.

It is now allowed to define functions on sealed tables that have string indexers. These functions will be typechecked against the indexer type. For example, the following is now valid:

local a : {[string]: () -> number} = {}

function a.y() return 4 end -- OK

Autocomplete will now provide string literal suggestions for singleton types. eg

local function f(x: "a" | "b") end f("_") -- suggest "a" and "b"

Improve error recovery in the case where we encounter a type pack variable in a place where one is not allowed. eg type Foo<A...> = { value: A... }

When code does not pass enough arguments to a variadic function, the error feedback is now better.

For example, the following script now produces a much nicer error message:

type A = { [number]: number } type B = { [number]: string }

local a: A = { 1, 2, 3 }

-- ERROR: Type 'A' could not be converted into 'B' -- caused by: -- Property '[indexer value]' is not compatible. Type 'number' could not be converted into 'string' local b: B = a

If the following code were to error because Hello was undefined, we would erroneously include the comment in the span of the error. This is now fixed.

type Foo = Hello -- some comment over here

Fix a crash that could occur when strict scripts have cyclic require() dependencies.

Add an option to autocomplete to cause it to abort processing after a certain amount of time has elapsed.

Updated: May 2, 2022






Luau Recap: March 2022

March 31, 2022

Luau is our new language that you can read more about at https://luau-lang.org.

[Cross-posted to the Roblox Developer Forum.] Singleton typesPermalink

We added support for singleton types! These allow you to use string or boolean literals in types. These types are only inhabited by the literal, for example if a variable x has type "foo", then x == "foo" is guaranteed to be true.

Singleton types are particularly useful when combined with union types, for example:

type Animals = "Dog" | "Cat" | "Bird"

or:

type Falsey = false | nil

In particular, singleton types play well with unions of tables, allowing tagged unions (also known as discriminated unions):

type Ok<T> = { type: "ok", value: T } type Err<E> = { type: "error", error: E } type Result<T, E> = Ok<T> | Err<E>

local result: Result<number, string> = ... if result.type == "ok" then 

   -- result :: Ok<number>
   print(result.value)

elseif result.type == "error" then 

   -- result :: Err<string>
   error(result.error)

end

The RFC for singleton types is https://github.com/Roblox/luau/blob/master/rfcs/syntax-singleton-types.md Width subtypingPermalink

A common idiom for programming with tables is to provide a public interface type, but to keep some of the concrete implementation private, for example:

type Interface = { 

   name: string,

}

type Concrete = { 

   name: string,
   id: number,

}

Within a module, a developer might use the concrete type, but export functions using the interface type:

local x: Concrete = { 

   name = "foo",
   id = 123,

}

local function get(): Interface 

   return x

end

Previously examples like this did not typecheck but now they do!

This language feature is called width subtyping (it allows tables to get wider, that is to have more properties).

The RFC for width subtyping is https://github.com/Roblox/luau/blob/master/rfcs/sealed-table-subtyping.md Typechecking improvementsPermalink 

   Generic function type inference now works the same for generic types and generic type packs.
   We improved some error messages.
   There are now fewer crashes (hopefully none!) due to mutating types inside the Luau typechecker.
   We fixed a bug that could cause two incompatible copies of the same class to be created.
   Luau now copes better with cyclic metatable types (it gives a type error rather than hanging).
   Fixed a case where types are not properly bound to all of the subtype when the subtype is a union.
   We fixed a bug that confused union and intersection types of table properties.
   Functions declared as function f(x : any) can now be called as f() without a type error.

API improvementsPermalink 

   Implement table.clone which takes a table and returns a new table that has the same keys/values/metatable. The cloning is shallow - if some keys refer to tables that need to be cloned, that can be done manually by modifying the resulting table.

Debugger improvementsPermalink 

   Use the property name as the name of methods in the debugger.

Performance improvementsPermalink 

   Optimize table rehashing (~15% faster dictionary table resize on average)
   Improve performance of freeing tables (~5% lift on some GC benchmarks)
   Improve gathering performance metrics for GC.
   Reduce stack memory reallocation.

Updated: March 31, 2022






Luau Recap: February 2022

February 28, 2022

Luau is our new language that you can read more about at https://luau-lang.org.

[Cross-posted to the Roblox Developer Forum.] Default type alias type parametersPermalink

We have introduced a syntax to provide default type arguments inside the type alias type parameter list.

It is now possible to have type functions where the instantiation can omit some type arguments.

You can provide concrete types:

--!strict type FieldResolver<T, Data = {[string]: any}> = (T, Data) -> number

local a: FieldResolver<number> = ... local b: FieldResolver<number, {name: string}> = ...

Or reference parameters defined earlier in the list:

--!strict type EqComp<T, U = T> = (l: T, r: U) -> boolean

local a: EqComp<number> = ... -- (l: number, r: number) -> boolean local b: EqComp<number, string> = ... -- (l: number, r: string) -> boolean

Type pack parameters can also have a default type pack:

--!strict type Process<T, U... = ...string> = (T) -> U...

local a: Process<number> = ... -- (number) -> ...string local b: Process<number, (boolean, string)> = ... -- (number) -> (boolean, string)

If all type parameters have a default type, it is now possible to reference that without providing any type arguments:

--!strict type All<T = string, U = number> = (T) -> U

local a: All -- ok local b: All<> -- ok as well

For more details, you can read the original RFC proposal. Typechecking improvementsPermalink

This month we had many fixes to improve our type inference and reduce false positive errors.

if-then-else expression can now have different types in each branch:

--!strict local a = if x then 5 else nil -- 'a' will have type 'number?' local b = if x then 1 else '2' -- 'b' will have type 'number | string'

And if the expected result type is known, you will not get an error in cases like these:

--!strict type T = {number | string} -- different array element types don't give an error if that is expected local c: T = if x then {1, "x", 2, "y"} else {0}

assert result is now known to not be ‘falsy’ (false or nil):

--!strict local function f(x: number?): number 

   return assert(x) -- no longer an error

end

We fixed cases where length operator # reported an error when used on a compatible type:

--!strict local union: {number} | {string} local a = #union -- no longer an error

Functions with different variadic argument/return types are no longer compatible:

--!strict local function f(): (number, ...string) 

   return 2, "a", "b"

end

local g: () -> (number, ...boolean) = f -- error

We have also fixed: 

   false positive errors caused by incorrect reuse of generic types across different function declarations
   issues with forward-declared intersection types
   wrong return type annotation for table.move
   various crashes reported by developers

Linter improvementsPermalink

A new static analysis warning was introduced to mark incorrect use of a ‘a and b or c’ pattern. When ‘b’ is ‘falsy’ (false or nil), result will always be ‘c’, even if the expression ‘a’ was true:

local function f(x: number) 

   -- The and-or expression always evaluates to the second alternative because the first alternative is false; consider using if-then-else expression instead
   return x < 0.5 and false or 42

end

Like we say in the warning, new if-then-else expression doesn’t have this pitfall:

local function g(x: number) 

   return if x < 0.5 then false else 42

end

We have also introduced a check for misspelled comment directives:

--!non-strict -- ^ Unknown comment directive 'non-strict'; did you mean 'nonstrict'?

Performance improvementsPermalink

For performance, we have changed how our Garbage Collector collects unreachable memory. This rework makes it possible to free memory 2.5x faster and also comes with a small change to how we store Luau objects in memory. For example, each table now uses 16 fewer bytes on 64-bit platforms.

Another optimization was made for select(_, ...) call. It is now using a special fast path that has constant-time complexity in number of arguments (~3x faster with 10 arguments). ThanksPermalink

A special thanks to all the fine folks who contributed PRs this month! 

   mikejsavage
   TheGreatSageEqualToHeaven
   petrihakkinen

Updated: February 28, 2022






Luau Recap: January 2022

January 27, 2022

Luau is our programming language that you can read more about at https://luau-lang.org.

Find us on GitHub!

[Cross-posted to the Roblox Developer Forum.] Performance improvementsPermalink

The implementation of tostring has been rewritten. This change replaces the default number->string conversion with a new algorithm called Schubfach, which allows us to produce the shortest precise round-trippable representation of any input number very quickly.

While performance is not the main driving factor, this also happens to be significantly faster than our old implementation (up to 10x depending on the number and the platform).

Make tonumber(x) ~2x faster by avoiding reparsing string arguments.

The Luau compiler now optimizes table literals where keys are constant variables the same way as if they were constants, eg

local r, g, b = 1, 2, 3 local col = { [r] = 255, [g] = 0, [b] = 255 }

Improvements to type assertionsPermalink

The :: type assertion operator can now be used to coerce a value between any two related types. Previously, it could only be used for downcasts or casts to any. The following used to be invalid, but is now valid:

local t = {x=0, y=0} local a = t :: {x: number}

Typechecking improvementsPermalink

An issue surrounding table literals and indexers has been fixed:

type RecolorMap = {[string]: RecolorMap | Color3}

local hatRecolorMap: RecolorMap = { 

   Brim = Color3.fromRGB(255, 0, 0), -- We used to report an error here
   Top = Color3.fromRGB(255, 0, 0)

}

Accessing a property whose base expression was previously refined will now return the correct result. Linter improvementsPermalink

table.create(N, {}) will now produce a static analysis warning since the element is going to be shared for all table entries. Error reporting improvementsPermalink

When a type error involves a union (or an option), we now provide more context in the error message.

For instance, given the following code:

--!strict

type T = {x: number}

local x: T? = {w=4}

We now report the following:

Type 'x' could not be converted into 'T?' caused by: 

 None of the union options are compatible. For example: Table type 'x' not compatible with type 'T' because the former is missing field 'x'

Luau now gives up and reports an *unknown* type in far fewer cases when typechecking programs that have type errors. New APIsPermalink

We have brought in the coroutine.close function from Lua 5.4. It accepts a suspended coroutine and marks it as non-runnable. In Roblox, this can be useful in combination with task.defer to implement cancellation. REPL improvementsPermalink

The luau REPL application can be compiled from source or downloaded from releases page. It has grown some new features: 

   Added --interactive option to run the REPL after running the last script file.
   Allowed the compiler optimization level to be specified.
   Allowed methods to be tab completed
   Allowed methods on string instances to be completed
   Improved Luau REPL argument parsing and error reporting
   Input history is now saved/loaded

ThanksPermalink

A special thanks to all the fine folks who contributed PRs over the last few months! 

   Halalaluyafail3
   JohnnyMorganz
   Kampfkarren
   kunitoki
   MathematicalDessert
   metatablecat
   petrihakkinen
   rafa_br34
   Rerumu
   Slappy826
   SnowyShiro
   vladmarica
   xgladius

Contribution guide

Updated: January 27, 2022






Luau Recap: November 2021

November 29, 2021

Luau is our new language that you can read more about at https://luau-lang.org.

[Cross-posted to the Roblox Developer Forum.] Type packs in type aliasesPermalink

Type packs are the construct Luau uses to represent a sequence of types. We’ve had syntax for generic type packs for a while now, and it sees use in generic functions, but it hasn’t been available in type aliases. That has changed, and it is now syntactically legal to write the following type alias:

type X<A...> = () -> A... type Y = X<number, string>

We’ve also added support for explicit type packs. Previously, it was impossible to instantiate a generic with two or more type pack parameters, because it wasn’t clear where the first pack ended and the second one began. We have introduced a new syntax for this use case:

type Fn<P..., R...> = (P...) -> R... type X = Fn<(number, string), (string, number)>

For more information, check out the documentation or the RFC for this feature. Luau is open-source!Permalink

We announced this in early November but it deserves repeating: Luau is now an open-source project! You can use Luau outside of Roblox, subject to MIT License, and - importantly - we accept contributions.

Many changes contributed by community, both Roblox and external, have been merged since we’ve made Luau open source. Of note are two visible changes that shipped on Roblox platform: 

   The type error “Expected to return X values, but Y values are returned here” actually had X and Y swapped! This is now fixed.
   Luau compiler dutifully computed the length of the string when using # operator on a string literal; this is now fixed and #"foo" compiles to 3.

You might think that C++ is a scary language and you can’t contribute to Luau. If so, you’d be happy to know that the contents of https://luau-lang.org, where we host our documentation, is also hosted on GitHub in the same repository (https://github.com/Roblox/luau/tree/master/docs) and that we’d love the community to contribute improvements to documentation among other changes! For example see issues in this list that start with “Documentation”, but all other changes and additions to documentation are also welcome. Library improvementsPermalink

function bit32.countlz(n: number): number function bit32.countrz(n: number): number

Given a number, returns the number of preceding left or trailing right-hand bits that are 0.

See the RFC for these functions for more information. Type checking improvementsPermalink

We have enabled a rewrite of how Luau handles require tracing. This has two main effects: firstly, in strict mode, require statements that Luau can’t resolve will trigger type errors; secondly, Luau now understands the FindFirstAncestor method in require expressions.

Luau now warns when the index to table.move is 0, as this is non-idiomatic and performs poorly. If this behavior is intentional, wrap the index in parentheses to suppress the warning.

Luau now provides additional context in table and class type mismatch errors. Performance improvementsPermalink

We have enabled several changes that aim to avoid allocating a new closure object in cases where it’s not necessary to. This is helpful in cases where many closures are being allocated; in our benchmark suite, the two benchmarks that allocate a large number of closures improved by 15% and 5%, respectively.

When checking union types, we now try possibilities whose synthetic names match. This will speed up type checking unions in cases where synthetic names are populated.

We have also enabled an optimization that shares state in a hot path on the type checker. This will improve type checking performance.

The Luau VM now attempts to cache the length of tables’ array portion. This change showed a small performance improvement in benchmarks, and should speed up # expressions.

The Luau type checker now caches a specific category of table unification results. This can improve type checking performance significantly when the same set of types is used frequently.

When Luau is not retaining type graphs, the type checker now discards more of a module’s type surface after type checking it. This improves memory usage significantly. Bug fixesPermalink

We’ve fixed a bug where on ARM systems (mobile), packing negative numbers using unsigned formats in string.pack would produce the wrong result.

We’ve fixed an issue with type aliases that reuse generic type names that caused them to be instantiated incorrectly.

We’ve corrected a subtle bug that could cause free types to leak into a table type when a free table is bound to that table.

We’ve fixed an issue that could cause Luau to report an infinitely recursive type error when the type was not infinitely recursive.

Updated: November 29, 2021






Luau Goes Open-Source

November 3, 2021

When Roblox was created 15 years ago, we chose Lua as the scripting language. Lua was small, fast, easy to embed and learn and opened up enormous possibilities for our developers.

A lot in Roblox was built on Lua including hundreds of thousands of lines of internally-developed code that powers Roblox App and Roblox Studio to this day, and the millions of experiences that developers have created. For many of them, it was the first programming language they’ve learned.

A few years ago, we started looking into how we can evolve Lua to be even faster, have better ergonomics, make it easier to write robust code and to unlock an ecosystem of rich tooling—from better static analysis to IDE integrations.

This is how Luau was born.

Luau is a new language that started from Lua 5.1 and kept evolving while keeping backwards compatibility and preserving the original design goals: simplicity, performance, embeddability.

We’re incredibly grateful for the foundation that Lua has been—it’s been a joy to build on top of! So now we want to give back to the community at large.

Starting today, Luau is no longer an inseparable part of Roblox platform; it’s a separate, open-source language.

Luau is available at https://github.com/Roblox/luau and comes with the source code for the language runtime and all associated tooling: compiler, type checker, linter. The code is available to anyone, free of charge, under the terms of MIT License. We’re happy to accept contributions to the language, whether that’s documentation or source code.

The language evolution is driven by an RFC process that is also open to the public.

We are committed to improving Luau going forward—it remains a central piece of technology at Roblox. The team that works on the language keeps growing, and we have lots of ideas! The language will become even faster, even nicer to work with, even more powerful.

We can’t wait to see what we can build, together.

Updated: November 3, 2021






Luau Recap: October 2021

October 31, 2021

Luau is our new language that you can read more about at https://luau-lang.org.

[Cross-posted to the Roblox Developer Forum.] if-then-else expressionPermalink

In addition to supporting standard if statements, Luau adds support for if expressions. Syntactically, if-then-else expressions look very similar to if statements. However instead of conditionally executing blocks of code, if expressions conditionally evaluate expressions and return the value produced as a result. Also, unlike if statements, if expressions do not terminate with the end keyword.

Here is a simple example of an if-then-else expression:

local maxValue = if a > b then a else b

if-then-else expressions may occur in any place a regular expression is used. The if-then-else expression must match if <expr> then <expr> else <expr>; it can also contain an arbitrary number of elseif clauses, like if <expr> then <expr> elseif <expr> then <expr> else <expr>. Note that in either case, else is mandatory.

Here’s is an example demonstrating elseif:

local sign = if x < 0 then -1 elseif x > 0 then 1 else 0

Note: In Luau, the if-then-else expression is preferred vs the standard Lua idiom of writing a and b or c (which roughly simulates a ternary operator). However, the Lua idiom may return an unexpected result if b evaluates to false. The if-then-else expression will behave as expected in all situations. Library improvementsPermalink

New additions to the table library have arrived:

function table.freeze(t)

Given a non-frozen table, freezes it such that all subsequent attempts to modify the table or assign its metatable raise an error. If the input table is already frozen or has a protected metatable, the function raises an error; otherwise it returns the input table. Note that the table is frozen in-place and is not being copied. Additionally, only t is frozen, and keys/values/metatable of t don’t change their state and need to be frozen separately if desired.

function table.isfrozen(t): boolean

Returns true if and only if the input table is frozen. Typechecking improvementsPermalink

We continue work on our type constraint resolver and have multiple improvements this month.

We now resolve constraints that are created by or expressions. In the following example, by checking against multiple type alternatives, we learn that value is a union of those types:

--!strict local function f(x: any) 

   if type(x) == "number" or type(x) == "string" then
       local foo = x -- 'foo' type is known to be 'number | string' here
       -- ...
   end

end

Support for or constraints allowed us to handle additional scenarios with and and not expressions to reduce false positives after specific type guards.

And speaking of type guards, we now correctly handle sub-class relationships in those checks:

--!strict local function f(x: Part | Folder | string) 

   if typeof(x) == "Instance" then
       local foo = x -- 'foo' type is known to be 'Part | Folder' here
   else
       local bar = x -- 'bar' type is known to be 'string' here
   end

end

One more fix handles the a and b or c expression when ‘b’ depends on ‘a’:

--!strict function f(t: {x: number}?) 

   local a = t and t.x or 5 -- 'a' is a 'number', no false positive errors here

end

Of course, our new if-then-else expressions handle this case as well.

--!strict function f(t: {x: number}?) 

   local a = if t then t.x else 5 -- 'a' is a 'number', no false positive errors here

end

We have extended bidirectional typechecking that was announced last month to propagate types in additional statements and expressions.

--!strict function getSortFunction(): (number, number) -> boolean 

   return function(a, b) return a > b end -- a and b are now known to be 'number' here

end

local comp = getSortFunction()

comp = function(a, b) return a < b end -- a and b are now known to be 'number' here as well

We’ve also improved some of our messages with union types and optional types (unions types with nil).

When optional types are used incorrectly, you get better messages. For example:

--!strict function f(a: {number}?) 

   return a[1] -- "Value of type '{number}?' could be nil" instead of "'{number}?' is not a table'

end

When a property of a union type is accessed, but is missing from some of the options, we will report which options are not valid:

--!strict type A = { x: number, y: number } type B = { x: number } local a: A | B local b = a.y -- Key 'y' is missing from 'B' in the type 'A | B'

When we enabled generic functions last month, some users might have seen a strange error about generic functions not being compatible with regular ones.

This was caused by undefined behaviour of recursive types. We have now added a restriction on how generic type parameters can be used in recursive types: RFC: Recursive type restriction Performance improvementsPermalink

An improvement to the Stop-The-World (atomic in Lua terms) stage of the garbage collector was made to reduce time taken by that step by 4x factor. While this step only happens once during a GC cycle, it cannot be split into small parts and long times were visible as frame time spikes.

Table construction and resize was optimized further; as a result, many instances of table construction see 10-20% improvements for smaller tables on all platforms and 20%+ improvements on Windows.

Bytecode compiler has been optimized for giant table literals, resulting in 3x higher compilation throughput for certain files on AMD Zen architecture.

Coroutine resumption has been optimized and is now ~10% faster for coroutine-heavy code.

Array reads and writes are also now a bit faster resulting in 1-3% lift in array-heavy benchmarks.

Updated: October 31, 2021






Luau Recap: September 2021

September 30, 2021

Luau is our new language that you can read more about at https://luau-lang.org.

[Cross-posted to the Roblox Developer Forum.] Generic functionsPermalink

The big news this month is that generic functions are back!

Luau has always supported type inference for generic functions, for example:

type Point<X,Y> = { x: X, y: Y } function swap(p) 

 return { x = p.y, y = p.x }

end local p : Point<number, string> = swap({ x = "hi", y = 37 }) local q : Point<boolean, string> = swap({ x = "hi", y = true })

but up until now, there’s been no way to write the type of swap, since Luau didn’t have type parameters to functions (just regular old data parameters). Well, now you can:

function swap<X, Y>(p : Point<X, Y>): Point<Y, X> 

 return { x = p.y, y = p.x }

end

Generic functions can be used in function declarations, and function types too, for example

type Swapper = { swap : <X, Y>(Point<X, Y>) -> Point<Y, X> }

People may remember that back in April we announced generic functions, but then had to disable them. That was because DataBrain discovered a nasty interaction between typeof and generics, which meant that it was possible to write code that needed nested generic functions, which weren’t supported back then.

Well, now we do support nested generic functions, so you can write code like

function mkPoint(x) 

 return function(y)
   return { x = x, y = y }
 end

end

and have Luau infer a type where a generic function returns a generic function

function mkPoint<X>(x : X) : <Y>(Y) -> Point<X,Y> 

 return function<Y>(y : Y) : Point<X,Y>
   return { x = x, y = y }
 end

end

For people who like jargon, Luau now supports Rank N Types, where previously it only supported Rank 1 Types. Bidirectional typecheckingPermalink

Up until now, Luau has used bottom-up typechecking. For example, for a function call f(x) we first find the type of f (say it’s (T)->U) and the type for x (say it’s V), make sure that V is a subtype of T, so the type of f(x) is U.

This works in many cases, but has problems with examples like registering callback event handlers. In code like

part.Touched:Connect(function (other) ... end)

if we try to typecheck this bottom-up, we have a problem because we don’t know the type of other when we typecheck the body of the function.

What we want in this case is a mix of bottom-up and top-down typechecking. In this case, from the type of part.Touched:Connect we know that other must have type BasePart.

This mix of top-down and bottom-up typechecking is called bidirectional typechecking, and means that tools like type-directed autocomplete can provide better suggestions. Editor featuresPermalink

We have made some improvements to the Luau-powered autocomplete beta feature in Roblox Studio: 

   We no longer give autocomplete suggestions for client-only APIs in server-side scripts, or vice versa.
   For table literals with known shape, we provide autocomplete suggestions for properties.
   We provide autocomplete suggestions for Player.PlayerGui.
   Keywords such as then and else are autocompleted better.
   Autocompletion is disabled inside a comment span (a comment starting --[[).

Typechecking improvementsPermalink

In other typechecking news: 

   The Luau constraint resolver can now refine the operands of equality expressions.
   Luau type guard refinements now support more arbitrary cases, for instance typeof(foo) ~= "Instance" eliminates anything not a subclass of Instance.
   We fixed some crashes caused by use-after-free during type inference.
   We do a better job of tracking updates when script is moved inside the data model.
   We fixed one of the ways that recursive types could cause free types to leak.
   We improved the way that return statements interact with mutually recursive function declarations.
   We improved parser recovery from code which looks like a function call (but isn’t) such as

   local x = y
   (expr)[smth] = z

   We consistently report parse errors before type errors.
   We display more types as *unknown* rather than as an internal type name like error####.
   Luau now infers the result of Instance:Clone() much more accurately.

Performance improvementsPermalink 

   Vector3.new constructor has been optimized and is now ~2x faster
   A previously implemented optimization for table size prediction has been enhanced to predict final table size when setmetatable is used, such as local self = setmetatable({}, Klass)
   Method calls for user-specified objects have been optimized and are now 2-4% faster
   debug.traceback is now 1.5x faster, although debug.info is likely still superior for performance-conscious code
   Creating table literals with explicit numeric indices, such as { [1] = 42 }, is now noticeably faster, although list-style construction is still recommended.

Other improvementsPermalink 

   The existing ‘TableLiteral’ lint now flags cases when table literals have duplicate numeric indices, such as { [1] = 1, [1] = 2 }

Updated: September 30, 2021






Luau Recap: August 2021

August 31, 2021

Luau is our new language that you can read more about at https://luau-lang.org.

[Cross-posted to the Roblox Developer Forum.] Editor featuresPermalink

The Roblox Studio Luau-Powered Autocomplete & Language Features Beta that our team has been working on has finally been released! Be sure to check that out and leave your feedback for things we can improve.

To support that feature, a lot of work went into: 

   Improving fault-tolerant parser recovery scenarios
   Storing additional information in the AST, including comments, better location information and partial syntax data
   Tracking additional information about types and their fields, including tracking definition locations, function argument names, deprecation state and custom Roblox-specific tags
   Updating reflection information to provide more specific Instance types and correct previously missing or wrong type annotations
   Hybrid typechecking mode which tries to infer types even in scripts with no typechecking enabled
   Support for types that are attached to the DataModel tree elements to provide instance member information
   Placing limits to finish typechecking in a finite space/time
   Adding Autocomplete API for the Roblox Studio to get location-based entity information and appropriate suggestions
   Additional type inference engine improvements and fixes

While our work continues to respond to the feedback we receive, our team members are shifting focus to add generic functions, improve type refinements in conditionals, extend Parallel Luau, improve Lua VM performance and provide documentation. Typechecking improvementsPermalink

Type constraint resolver now remembers constraints placed on individual table fields.

This should fix false-positive errors reported after making sure the optional table field is present:

--!strict local t: {value: number?} = {value = 2}

if t.value then 

   local v: number = t.value -- ok

end

And it can also refine field type to a more specific one:

--!strict local t: {value: string|number} = {value = 2}

if type(t.value) == "number" then 

   return t.value * 2 -- ok

end

Like before, combining multiple conditions using ‘and’ and ‘not’ is also supported.

Constructing arrays with different values for optional/union types are now also supported for individual table fields and in functions call arguments:

--!strict type Foo = {x: number | string, b: number?}

local function foo(l: {Foo}) end

foo({ {x = 1234567}, {x = "hello"}, -- now ok })

type Bar = {a: {Foo}}

local foos: Bar = {a = { {x = 1234567}, {x = "hello", b = 2}, -- now ok }}

Finally, we have fixed an issue with Roblox class field access using indexing like part["Anchored"] = true. Linter improvementsPermalink

We have added a new linter check for duplicate local variable definitions.

It is created to find duplicate names in cases like these:

local function foo(a1, a2, a2) -- Function argument 'a2' already defined on column 24 local a1, a2, a2 = f() -- Variable 'a2' already defined on column 11

local bar = {} function bar:test(self) -- Function argument 'self' already defined implicitly

Our UnknownType linter warning was extended to check for correct class names passed into FindFirstChildOfClass, FindFirstChildWhichIsA, FindFirstAncestorOfClass and FindFirstAncestorWhichIsA functions. Performance improvementsPermalink

We have added an optimization to ‘table.unpack’ for 2x performance improvement.

We’ve also implemented an extra optimization for tables to predict required table capacity based on fields that are assigned to it in the code after construction. This can reduce the need to reallocate tables.

Variadic call performance was fine-tuned and is now ~10% faster.

Construction of array literals was optimized for a ~7% improvement.

Another optimization this month changes the location and rate of garbage collection invocations. We now try to avoid calling GC during the script execution and perform all the work in the GcJob part of the frame (it could be seen in the performance profiler). When possible, we can now skip that job in the frame completely, if we have some memory budget available. Other improvementsPermalink

For general stability improvements we fixed a crash when strange types like ‘nil??’ are used and when users have their own global functions named ‘require’.

Indexing a table with an incompatible type will now show squiggly error lines under the index instead of the whole expression, which was a bit misleading.

An issue with debug information that caused repeat ... until condition line to be skipped when stepping was fixed.

Type output was improved to replace display of types like ‘{<g405>(g405) -> g405}’ with ‘{<a>(a) -> a}’.

Updated: August 31, 2021






Luau Recap: July 2021

July 30, 2021

Luau is our new language that you can read more about at https://luau-lang.org. Our team was still busy working on upcoming Studio Beta feature for script editor, but we did fit in multiple typechecking improvements.

[Cross-posted to the Roblox Developer Forum.] Typechecking improvementsPermalink

A common complaint that we’ve received was a false-positive error when table with an optional or union element type is defined:

--!strict type Foo = {x: number | string} local foos: {Foo} = { 

   {x = 1234567},
   {x = "hello"} -- Type 'string' could not be converted into 'number'

}

This case is now handled and skipping optional fields is allowed as well:

--!strict type Foo = { 

   a: number,
   b: number?

} local foos: {Foo} = { 

   { a = 1 },
   { a = 2, b = 3 } -- now ok

}

Current fix only handles table element type in assignments, but we plan to extend that to function call arguments and individual table fields.

Like we’ve mentioned last time, we will continue working on our new type constraint resolver and this month it learned to handle more complex expressions (including type guards) inside assert conditions:

--!strict local part = script.Parent:WaitForChild("Part") assert(part:IsA("BasePart")) local basepart: BasePart = part -- no longer an error

And speaking of assertions, we applied a minor fix so that the type of the assert function correctly defines a second optional string? parameter.

We have also fixed the type of string.gmatch function reported by one of the community members. We know about issues in a few additional library functions and we’ll work to fix them as well.

Hopefully, you didn’t see ‘free type leak’ errors that underline your whole script, but some of you did and reported them to us. We read those reports and two additional cases have been fixed this month. We now track only a single one that should be fixed next month.

Another false positive error that was fixed involves tables with __call metatable function. We no longer report a type error when this method is invoked and we’ll also make sure that given arguments match the function definition:

--!strict local t = { x = 2 }

local x = setmetatable(t, { 

   __call = function(self, a: number)
       return a * self.x
   end

}) local a = x(2) -- no longer an error

Please note that while call operator on a table is now handled, function types in Luau are distinct from table types and you’ll still get an error if you try to assign this table to a variable of a function type. Linter improvementsPermalink

A new ‘TableOperations’ lint check was added that will detect common correctness or performance issues with table.insert and table.remove:

-- table.insert will insert the value before the last element, which is likely a bug; consider removing the second argument or wrap it in parentheses to silence table.insert(t, #t, 42)

-- table.insert will append the value to the table; consider removing the second argument for efficiency table.insert(t, #t + 1, 42)

-- table.insert uses index 0 but arrays are 1-based; did you mean 1 instead? table.insert(t, 0, 42)

-- table.remove uses index 0 but arrays are 1-based; did you mean 1 instead? table.remove(t, 0)

-- table.remove will remove the value before the last element, which is likely a bug; consider removing the second argument or wrap it in parentheses to silence table.remove(t, #t - 1)

-- table.insert may change behavior if the call returns more than one result; consider adding parentheses around second argument table.insert(t, string.find("hello", "h"))

Another new check is ‘DuplicateConditions’. The name speaks for itself, if statement chains with duplicate conditions and expressions containing and/or operations with redundant parts will now be detected:

if x then 

   -- ...

elseif not x then 

   -- ...

elseif x̳ then -- Condition has already been checked on line 1 

   -- ...

end

local success = a and a̳ -- Condition has already been checked on column 17

local good = (a or b) or a̳ -- Condition has already been checked on column 15

We’ve also fixed an incorrect lint warning when typeof is used to check for EnumItem. Editor featuresPermalink

An issue was fixed that prevented the debugger from displaying values inside Roblox callback functions when an error was reported inside of it. Behavior changesPermalink

table.insert will no longer move elements forward 1 spot when index is negative or 0.

This change also fixed a performance issue when table.insert was called with a large negative index.

The ‘TableOperations’ lint mentioned earlier will flag cases where insertion at index 0 is performed.

Updated: July 30, 2021






Luau Recap: June 2021

June 30, 2021

Luau is our new language that you can read more about at https://roblox.github.io/luau. Most of our team was busy working on improving Luau interaction with Roblox Studio for an upcoming feature this month, but we were able to add typechecking and performance improvements as well!

[Cross-posted to the Roblox Developer Forum.] Constraint ResolverPermalink

To improve type inference under conditional expressions and other dynamic type changes (like assignments) we have introduced a new constraint resolver framework into Luau type checker.

This framework allows us to handle more complex expressions that combine and/not operators and type guards.

Type guards support include expressions like: 

   if instance:IsA("ClassName") then
   if enum:IsA("EnumName") then
   if type(v) == "string" then

This framework is extensible and we have plans for future improvements with a == b/a ~= b equality constraints and handling of table field assignments.

It is now also possible to get better type information inside else blocks of an if statement.

A few examples to see the constraint resolver in action:

function say_hello(name: string?) 

   -- extra parentheses were enough to trip the old typechecker
   if (name) then 
       print("Hello " .. name .. "!")
   else
       print("Hello mysterious stranger!")
   end

end

function say_hello(name: string?, surname: string?) 

   -- but now we handle that and more complex expressions as well
   if not (name and surname) then
       print("Hello mysterious stranger!")
   else
       print("Hello " .. name .. " " .. surname .. "!")
   end

end

Please note that constraints are currently placed only on local and global variables. One of our goals is to include support for table members in the future. Typechecking improvementsPermalink

We have improved the way we handled module require calls. Previously, we had a simple pattern match on the local m = require(...) statement, but now we have replaced it with a general handling of the function call in any context.

Handling of union types in equality operators was fixed to remove incorrect error reports.

A new IsA method was introduced to EnumItem to check the type of a Roblox Enum. This is intended to replace the enumItem.EnumType == Enum.NormalId pattern in the code for a construct that allows our constraint resolver to infer better types.

Additional fixes include: 

   table.pack return type was fixed
   A limit was added for deeply nested code blocks to avoid a crash
   We have improved the type names that are presented in error messages and Roblox Studio
   Error recovery was added to field access of a table? type. While you add a check for nil, typechecking can continue with better type information in other expressions.
   We handled a few internal compiler errors and rare crashes

Editor featuresPermalink

If you have Luau-Powered Type Hover beta feature enabled in Roblox Studio, you will see more function argument names inside function type hovers. Behavior changesPermalink

We no longer allow referencing a function by name inside argument list of that function:

local function f(a: number, b: typeof(f)) -- 'f' is no longer visible here Performance improvementsPermalink

As always, we look for ways to improve performance of your scripts: 

   We have fixed memory use of Roblox Actor scripts in Parallel Luau beta feature
   Performance of table clone through table.move has been greatly improved
   Table length lookup has been optimized, which also brings improvement to table element insertion speed
   Built-in Vector3 type support that we mentioned in April is now enabled for everyone

Updated: June 30, 2021






Luau Recap: May 2021

May 31, 2021

Luau is our new language that you can read more about at https://roblox.github.io/luau. This month we have added a new small feature to the language and spent a lot of time improving our typechecker.

[Cross-posted to the Roblox Developer Forum.] Named function type argumentsPermalink

We’ve updated Luau syntax to support optional names of arguments inside function types. The syntax follows the same format as regular function argument declarations: (a: number, b: string)

Names can be provided in any place where function type is used, for example: 

   in type aliases:

   type MyCallbackType = (cost: number, name: string) -> string

   for variables:

   local cb: (amount: number) -> number
   local function foo(cb: (name: string) -> ())

Variadic arguments cannot have an extra name, they are already written as …: number.

These names are used for documentation purposes and we also plan to display them in Roblox Studio auto-complete and type hovers. They do not affect how the typechecking of Luau scripts is performed. Typechecking improvementsPermalink

Speaking of typechecking, we’ve implemented many improvements this month: 

   Typechecker will now visit bodies of all member functions, previously it didn’t check methods if the self type was unknown
   Made improvements to cyclic module import detection and error reporting
   Fixed incorrect error on modification of table intersection type fields
   When using an ‘or’ between a nillable type and a value, the resulting type is now inferred to be non-nil
   We have improved error messages that suggest to use ‘:’ for a method call
   Fixed order of types in type mismatch error that was sometimes reversed
   Fixed an issue with table.insert function signature
   Fixed a bug which caused spurious unknown global errors

We’ve also added new checks to our linter: 

   A new check will report uses of deprecated Roblox APIs
   Linter will now suggest replacing globals with locals in more cases
   New warning is generated if array loop starts or ends on index ‘0’, but the array is indexed from ‘1’
   FormatString lint will now check string patterns for find/match calls via : when object type is known to be a string

We also fixed one of the sources for “Free types leaked into this module’s public interface” error message and we are working to fix the remaining ones.

As usual, typechecking improvements will not break execution of your games even if new errors get reported. Editor featuresPermalink

We continue to improve our built-in support for auto-complete that will be used in future Roblox Studio updates and will make it easier to implement custom extensions for applications that support Language Server Protocol.

As part of this work we will improve the type information provided by Roblox APIs to match actual arguments and results. Behavior changesPermalink

When a relational comparison fails at runtime, the error message now specifies the comparison direction (e.g. attempt to compare nil <= number) Performance improvementsPermalink 

   Improved performance of table lookup with an index operator and a literal string: t["name"]
   Bytecode compilation is now ~5% faster which can improve server startup time for games with lots of scripts

Updated: May 31, 2021






Luau Recap: April 2021

April 30, 2021

Luau is our new language that you can read more about at https://roblox.github.io/luau. Another busy month in Luau with many performance improvements.

[Cross-posted to the Roblox Developer Forum.] Editor featuresPermalink

Luau implementation now provides an internal API for type-aware autocomplete suggestions.

Roblox Studio will be the first user of this API and we plan for a new beta feature to come soon in addition to existing Luau-powered beta features like Go To Declaration, Type Hovers and Script Function Filter (you should check those out!) Performance improvementsPermalink

Performance is a very important part of Luau implementation and we continue bringing in new performance optimizations: 

   We’ve finished the work on internal vector value type that will be used by Vector3 type in Roblox. Improvements of up to 10x can be seen for primitive operations and some of our heavy Vector3 benchmarks have seen 2-3x improvement. You can read more about this feature on Roblox Developer forums
   By optimizing the way string buffers are handled internally, we bring improvements to string operations including string.lower, string.upper, string.reverse, string.rep, table.concat and string concatenation operator ... Biggest improvements can be seen on large strings
   Improved performance of table.insert and table.remove. Operations in the middle of large arrays can be multiple times faster with this change
   Improved performance of internal table resize which brings additional 30% speedup for table.insert
   Improved performance of checks for missing table fields

Generic functionsPermalink

We had to temporarily disable generic function definitions last month after finding critical issues in the implementation.

While they are still not available, we are making steady progress on fixing those issues and making additional typechecking improvements to bring them back in. Debugger improvementsPermalink

Debugging is now supported for parallel Luau Actors in Roblox Studio.

Read more about the feature on Roblox Developer forums and try it out yourself. Behavior changesPermalink

Backwards compatibility is important for Luau, but sometimes a change is required to fix corner cases in the language / libraries or to improve performance. Even still, we try to keep impact of these changes to a minimum: 

   __eq tag method will always get called for table comparisons even when a table is compared to itself

Coming soon…Permalink 

   Better type refinements for statements under a condition using a new constraint resolver. Luau will now understand complex conditions combining and/not and type guards with more improvements to come

Updated: April 30, 2021






Luau Recap: March 2021

March 29, 2021

Luau is our new language that you can read more about at https://roblox.github.io/luau. It’s been a busy month in Luau!

[Cross-posted to the Roblox Developer Forum.] Typed variadicsPermalink

Luau supports variadic functions, meaning ones which can take a variable number of arguments (varargs!) but previously there was no way to specify their type. Now you can!

function f(x: string, ...: number) 

 print(x)
 print(...)

end f("hi") f("lo", 5, 27)

This function takes a string, plus as many numbers as you like, but if you try calling it with anything else, you’ll get a type error, for example f("oh", true) gives an error “Type boolean could not be converted into number”

Variadics can be used in function declarations, and function types, for example

type T = { 

 sum: (...number) -> number

} function f(x: T) 

 print(x.sum(1, 2, 3))

end

Generic functionsPermalink

WARNING Generic functions are currently disabled as we’re fixing some critical bugs. Typechecking improvementsPermalink

We’ve made various improvements to the Luau typechecker: 

   Check bodies of methods whose self has type any
   More precise types for debug.* methods
   Mutually dependent type aliases are now handled correctly

Performance improvementsPermalink

We are continuing to squeeze the performance out of all sorts of possible code; this is an ongoing process and we have many improvements in the pipeline, big and small. These are the changes that are already live: 

   Significantly optimized non-variadic function calls, improving performance by up to 10% on call-heavy benchmarks
   Improve performance of math.clamp, math.sign and math.round by 2.3x, 2x and 1.6x respectively
   Optimized coroutine.resume with ~10% gains on coroutine-heavy benchmarks
   Equality comparisons are now a bit faster when comparing to constants, including nil; this makes some benchmarks 2-3% faster
   Calls to builtin functions like math.abs or bit32.rrotate are now significantly faster in some cases, e.g. this makes SHA256 benchmark 25% faster
   rawset, rawget, rawequal and 2-argument table.insert are now 40-50% faster; notably, table.insert(t, v) is now faster than t[#t+1]=v

Note that we work off a set of benchmarks that we consider representative of the wide gamut of code that runs on Luau. If you have code that you think should be running faster, never hesitate to open a feature request / bug report on Roblox Developer Forum! Debugger improvementsPermalink

We continue to improve our Luau debugger and we have added a new feature to help with coroutine call debugging. The call stack that is being displayed while stopped inside a coroutine frame will display the chain of threads that have called it.

Before:

"Old debugger"

After:

"New debugger"

We have restored the ability to break on all errors inside the scripts. This is useful in cases where you need to track the location and state of an error that is triggered inside ‘pcall’. For example, when the error that’s triggered is not the one you expected.

"Break on all exceptions" Library changesPermalink 

   Added the debug.info function which allows retrieving information about stack frames or functions; similarly to debug.getinfo from Lua, this accepts an options string that must consist of characters slnfa; unlike Lua that returns a table, the function returns all requested values one after another to improve performance.

New logoPermalink

Luau now has a shiny new logo!

"New logo!" Coming soon…Permalink 

   Generic variadics!
   Native Vector3 math with dramatic performance improvements!
   Better tools for memory analysis!
   Better treatment of cyclic requires during type checking!
   Better type refinements including nil-ability checks, and/or and IsA!

Updated: March 29, 2021






Luau Recap: February 2021

March 1, 2021

Luau is our new language that you can read more about at https://roblox.github.io/luau. It’s been a busy few months in Luau!

[Cross-posted to the Roblox Developer Forum.] Infallible parserPermalink

Traditional compilers have focused on tasks that can be performed on complete programs, such as type-checking, static analysis and code generation. This is all good, but most programs under development are incomplete! They may have holes, statements that will be filled in later, and lines that are in the middle of being edited. If we’d like to provide support for developers while they are writing code, we need to provide tools for incomplete programs as well as complete ones.

The first step in this is an infallible parser, that always returns an Abstract Syntax Tree, no matter what input it is given. If the program is syntactically incorrect, there will also be some syntax errors, but the parser keeps going and tries to recover from those errors, rather than just giving up.

The Luau parser now recovers from errors, which means, for example, we can give hints about programs in an IDE.

A type error after a syntax error Type assertionsPermalink

The Luau type checker can’t know everything about your code, and sometimes it will produce type errors even when you know the code is correct. For example, sometimes the type checker can’t work out the intended types, and gives a message such as “Unknown type used… consider adding a type annotation”.

"Consider adding a type annotation"

Previously the only way to add an annotation was to put it on the declaration of the variable, but now you can put it on the use too. A use of variable x at type T can be written x :: T. For example the type any can be used almost anywhere, so a common usage of type assertions is to switch off the type system by writing x :: any.

"A type assertion y:any" Typechecking improvementsPermalink

We’ve made various improvements to the Luau typechecker: 

   We allow duplicate function definitions in non-strict mode.
   Better typechecking of and, (f or g)(), arrays with properties, and string:format().
   Improved typechecking of infinite loops.
   Better error reporting for function type mismatch, type aliases and cyclic types.

Performance improvementsPermalink

We are continuing to work on optimizing our VM and libraries to make sure idiomatic code keeps improving in performance. Most of these changes are motivated by our benchmark suite; while some improvements may seem small and insignificant, over time these compound and allow us to reach excellent performance. 

   Table key assignments as well as global assignments have been optimized to play nicer with modern CPUs, yielding ~2% speedup in some benchmarks
   Luau function calls are now ~3% faster in most cases; we also have more call optimizations coming up next month!
   Modulo operation (%) is now a bit faster on Windows, resulting in ~2% performance improvement on some benchmarks

"Benchmark vs Lua 5.3" Debugger improvementsPermalink

Our Luau VM implementation is focused on performance and provides a different API for implementation of debugger tools. But it does have its caveats and one of them was inability to debug coroutines (breakpoints/stepping).

The good news is that we have lifted that limitation and coroutines can now be debugged just like any regular function. This can especially help people who use Promise libraries that rely on coroutines internally.

Debugging a coroutine Library changesPermalink

table library now has a new method, clear, that removes all keys from the table but keeps the internal table capacity. When working with large arrays, this can be more efficient than assigning a table to {} - the performance gains are similar to that of using table.create instead of {} when you expect the number of elements to stay more or less the same. Note that large empty tables still take memory and are a bit slower for garbage collector to process, so use this with caution.

In addition to that we found a small bug in string.char implementation that allowed creating strings from out-of-range character codes (e.g. string.char(2000)); the problem has been fixed and these calls now correctly generate an error. Coming soon…Permalink 

   Generic function types will soon be allowed!

   function id<a>(x: a): a
     return x
   end

   Typed variadics will soon allow types to be given to functions with varying numbers of arguments!

   function sum(...: number): number
     local result = 0
     for i,v in ipairs({...}) do
         result += v
     end
     return result
   end

And there will be more!

Updated: March 1, 2021






Luau Type Checking Release

November 19, 2020

10 months ago, we’ve started upon the journey of helping Roblox scripters write robust code by introducing an early beta of type checking. We’ve received a lot of enthusiastic feedback and worked with the community on trying to make sure critical issues are addressed, usability is improved and the type system is ready for prime time.

Today I’m incredibly excited to announce that the first release of Luau type checking is officially released! Thanks a lot to @Apakovtac, @EthicalRobot, @fun_enthusiast, @machinamentum, @mrow_pizza and @zeuxcg!

[Originally posted on the Roblox Developer Forum.] What is type checking?Permalink

When Luau code runs, every value has a certain type at runtime - a kind of value it stores. It could be a number, a string, a table, a Roblox Instance or one of many others. Thing is, some operations work on some types but don’t work on others!

Consider this:

local p = Instance.new("Part") p.Positio = Vector3.new(1,2,3)

Is this code correct? No - there’s a typo. The way you get to find this typo is by running your code and eventually seeing an error message. Type checker tries to analyze your code before running, by assigning a type to each value based on what we know about how that value was produced, or based on the type you’ve explicitly told us using a new syntax extension, and can produce an error ahead of time:

"Positio not found in class Part"

This can require some effort up front, especially if you use strict mode, but it can save you valuable time in the future. It can be especially valuable if you have a large complex code base you need to maintain for years, as is the case with many top Roblox games. How do I use type checking?Permalink

A very important feature of Luau type checking you need to know about is that it has three modes: 

   nocheck, where we don’t type check the script in question.
   nonstrict, where we type check the script but try to be lenient to allow commonly seen patterns even if they may violate type safety
   strict, where we try to make sure that every single line of code you write is correct, and every value has a known type.

The modes can be selected per script by writing a comment at the top of the script that starts with --!, e.g. --!strict.

As of this release, the default mode is nocheck. This means by default you actually won’t see the type checking produce feedback on your code! We had to use nocheck by default because we aren’t fully ready to unleash nonstrict mode on unsuspecting users - we need to do a bit more work to make sure that most cases where we tell you that something is wrong are cases where yes, something is actually wrong.

However we highly encourage trying at least non-strict mode on your codebase. You can do this by opting into a different default via a Studio beta:

"Studio option"

This beta only changes the default mode. Another way to change the mode is to prepend a --! comment to the script - you can do this manually for now, but if anyone in the community wants to release a plugin that does it automatically on selected scripts (+ descendants), that would be swell!

If you really want your code to be rock solid, we recommend trying out strict mode. Strict mode will require you to use type annotations. What are type annotations and how do I use them?Permalink

Glad you asked! (please pretend you did) Type annotations are a way to tell the type checker what the type of a variable is. Consider this code in strict mode:

function add(x, y) 

   return x + y

end

Is this code correct? Well, that depends. add(2, 3) will work just fine. add(Vector3.new(1, 2, 3), Vector3.new(4, 5, 6)) will work as well. But add({}, nil) probably isn’t a good idea.

In strict mode, we will insist that the type checker knows the type of all variables, and you’ll need to help the type checker occasionally - by adding types after variable names separated by ::

function add(x: number, y: number) 

   return x + y

end

If you want to tell the type checker “assume this value can be anything and I will take responsibility”, you can use any type which will permit any value of any type.

If you want to learn more about the type annotation syntax, you should read this documentation on syntax. We also have a somewhat more complete guide to type checking than this post can provide, that goes into more details on table types, OOP, Roblox classes and enums, interaction with require and other topics - read it if you’re curious!. What happens when I get a type error?Permalink

One concept that’s very important to understand is that right now type errors do not influence whether the code will run or not.

If you have a type error, this means that our type checker thinks your code has a bug, or doesn’t have enough information to prove the code works fine. But if you really want to forge ahead and run the code - you should feel free to do so!

This means that you can gradually convert your code to strict mode by adding type annotations and have the code runnable at all times even if it has type errors.

This also means that it’s safe to publish scripts even if type checker is not fully happy with them - type issues won’t affect script behavior on server/client, they are only displayed in Studio. Do I have to re-learn Lua now?!?Permalink

This is a question we get often! The answer is “no”.

The way the type system is designed is that it’s completely optional, and you can use as many or as few types as you’d like in your code.

In non-strict mode, types are meant as a lightweight helper - if your code is likely wrong, we’re going to tell you about it, and it’s up to you on whether to fix the issue, or even disable the type checker on a given problematic file if you really don’t feel like dealing with this.

In strict mode, types are meant as a power user tool - they will require more time to develop your code, but they will give you a safety net, where changing code will be much less likely to trigger errors at runtime. Is there a performance difference?Permalink

Right now type annotations are ignored by our bytecode compiler; this means that performance of the code you write doesn’t actually depend on whether you use strict, nonstrict or nocheck modes or if you have type annotations.

This is likely going to change! We have plans for using the type information to generate better bytecode in certain cases, and types are going to be instrumental to just-in-time compilation, something that we’re going to invest time into next year as well.

Today, however, there’s no difference - type information is completely elided when the bytecode is built, so there is zero runtime impact one way or another. What is next for types?Permalink

This is the first full release of type checking, but it’s by far the last one. We have a lot more ground to cover. Here’s a few things that we’re excited about that will come next: 

   Making nonstrict mode better to the point where we can enable it as a default for all Roblox scripts

   Adding several features to make strict mode more powerful/friendly, such as typed variadics, type ascription and better generics support

   Improving type refinements for type/typeof and nil checks

   Making it possible to view the type of a variable in Studio

   Reworking autocomplete to use type information instead of the current system

If you have any feedback on the type system, please don’t hesitate to share it here or in dedicated bug report threads. We’re always happy to fix corner cases that we’ve missed, fix stability issues if they are discovered, improve documentation when it’s not clear or improve error messages when they are hard to understand.

Updated: November 19, 2020






Luau Recap: October 2020

October 30, 2020

Luau is our new language that you can read more about at https://roblox.github.io/luau; we’ve been so busy working on the current projects that we didn’t do an update in September, so let’s look at changes that happened since August!

Many people work on these improvements, with the team slowly growing - thanks @Apakovtac, @EthicalRobot, @fun_enthusiast, @machinamentum, @mrow_pizza and @zeuxcg!

[Originally posted on the Roblox Developer Forum.] Types are very closePermalink

We’ve been in beta for a while now, but we’re steadily marching towards getting the first release of the type checker, what we call “types v0”, out of the door. It turns out that we’ve substantially underestimated the effort required to make the type system robust, strike the balance between “correct” and “usable” and give quality diagnostics in the event we do find issues with your code 🙂

Because of this, we’re changing the original plans for the release a bit. We’re actively working on a host of changes that we consider to be part of the “v0” effort, and when they are all finished - which should happen next month, fingers crossed - we’re going to be out of beta!

However, by default, on scripts with no annotations, we won’t actually activate type checking. You would have to opt into the type checking by using --!nonstrict or --!strict, at the top of each script. We are also going to open the second beta, “All scripts use non-strict mode by default” or something along these lines.

This is important because we found that our non-strict mode still needs some more work to be more tolerant to some code that occurs commonly in Roblox and is correct, but doesn’t type-check. We’re going to evaluate what changes specifically are required to make this happen, but we didn’t want the extra risk of a flood of reports about issues reported in existing code to shift the release date in an unpredictable fashion.

To that end, we’ve been working on Lots and Lots and Lots and Lots and Lots of changes to finish the first stage. Some of these changes are already live and some are rolling out; the amount of changes is so large that I can’t possibly list the up-to-date status on each one as these recaps are synthesized by the human who is writing this on a Friday night, so here’s just a raw list of changes that may or may not have been enabled: 

   Strict mode is now picky about passing extra arguments to functions, even though they are discarded silently at runtime, as this can hide bugs
   The error message about using a : vs . during type checking is now much more precise
   Recursive type annotations shouldn’t crash the type checker now, and we limit the recursion and iteration depth during type checking in a few cases in general in an effort to make sure type checker always returns in finite time
   Binary relational operators (< et al) are now stricter about the argument types and infer the argument types better
   Function argument and return types are now correctly contra- and co-variant; if this looks like gibberish to you, trust me - it’s for the best!
   Fixed a few problems with indexing unions of tables with matching key types
   Fixed issues with tracing types across modules (via require) in non-strict mode
   Error messages for long table types are now trimmed to make the output look nicer
   Improve the interaction between table types of unknown shape ({ [string]: X }) and table types of known shape.
   Fix some issues with type checking table assignments
   Fix some issues with variance of table fields
   Improve the legibility of type errors during function calls - errors now point at specific arguments that are incorrect, and mismatch in argument count should clearly highlight the problem
   Fix types for many builtins including ipairs, table.create, Color3.fromHSV, and a few others
   Fix missing callbacks for some instance types like OnInvoke for bindables (I think this one is currently disabled while we’re fixing a semi-related bug, but should be enabled soon!)
   Rework the rules under which globals are okay to use in non-strict mode to mostly permit valid scripts to type-check; strict mode will continue to frown upon the use of global variables
   Fix a problem with the beta where two scripts with identical names would share the set of errors/warnings, resulting in confusing error highlights for code that doesn’t exist
   Improve the legibility of type errors when indexing a table without a given key
   Improve the parsing error when trying to return a tuple; function f(): string, number is invalid since the type list should be parenthesized because of how our type grammar is currently structured
   Type checker now clearly reports cases where it finds a cyclic dependency between two modules
   Type errors should now be correctly sorted in the Script Analysis widget
   Error messages on mismatches between numbers of values in return statements should now be cleaner, as well as the associated type mismatch errors
   Improve error messages for comparison operators
   Flag attempts to require a non-module script during type checking
   Fix some cases where a type/typeof guard could be misled into inferring a non-sensible type
   Increase the strictness of return type checks in strict mode - functions now must conform to the specified type signature, whereas before we’d allow a function to return no values even in strict mode
   Improve the duplicate definition errors to specify the line of the first definition
   Increase the strictness of binary operators in strict mode to enforce the presence of the given operator as a built-in or as part of the metatable, to make sure that strict mode doesn’t infer types when it can’t guarantee correctness
   Improve the type errors for cyclic types to make them more readable
   Make type checker more friendly by rewording a lot of error messages
   Fix a few crashes in the type checker (although a couple more remain - working on them!)
   … I think that’s it?
   …edit ah, of course I forgot one thing - different enums that are part of the Roblox API now have distinct types and you can refer to the types by name e.g. Enum.Material; this should go live next week though. If you want to pretend that you’ve read and understood the entire list above, just know that we’ve worked on making sure strict mode is more reliably reporting type errors and doesn’t infer types incorrectly, on making sure non-strict mode is more forgiving for code that is probably valid, and on making the type errors more specific, easier to understand, and correct.

Type syntax changesPermalink

There’s only two small changes here this time around - the type syntax is now completely stable at this point, and any existing type annotation will continue parsing indefinitely. We of course reserve the right to add new syntax that’s backwards compatible :slight_smile:

On that note, one of the small changes is that we’ve finally removed support for fat arrows (=>); we’ve previously announced that this would happen and that thin arrows (->) are the future, and had warnings issued on the legacy syntax for a while. Now it’s gone.

On a positive note, we’ve added a shorter syntax for array-like table types. Whereas before you had to use a longer { [number]: string } syntax to declare an array-like table that holds strings, or had to define an Array type in every. single. module. you. ever. write. ever., now you can simply say {string}! This syntax is clean, in line with the value syntax for Lua table literals, and also was chosen by other research projects to add type annotations to Lua.

(if you’re a monster that uses mixed tables, you’ll have to continue using the longer syntax e.g. { [number]: string, n: number }) Library changesPermalink

There’s only a few small tweaks here this time around on the functionality front: 

   utf8.charpattern is now exactly equal to the version from Lua 5.3; this is now possible because we support \0 in patterns, and was suggested by a user on devforum. We do listen!
   string.pack now errors out early when the format specifier is Way Too Large. This was reported on dev forum and subsequently fixed. Note that trying to generate a Moderately Large String (like, 100 MB instead of 100 GB) will still succeed but may take longer than we’d like - we have a plan to accelerate operations on large strings substantially in the coming months.

Performance improvementsPermalink

We were super focused on other things so this is very short this time around. We have a lot of ideas here but they are waiting for us to finish some other large projects! 

   Method calls on strings via : are now ~10% faster than before. We still recommend using fully-qualified calls from string library such as string.foo(str), but extra performance never hurts!
   Speaking of string methods, string.sub is now ~20% faster than before with the help of voodoo magic.

Miscellaneous fixesPermalink

There were a few small fixes that didn’t land into any specific category that I wanted to highlight: 

   In some rare cases, debug information on conditions inside loops have been fixed to stop debugger from incorrectly suggesting that the current line is inside a branch that wasn’t taken. As usual, if you ever see debugger misbehaving, please file bugs on this!
   Code following assert(false) is now treated as an unreachable destination from the linting and type checking point of view, similarly to error calls.
   Linting support for various format strings has been greatly improved based on fantastic feedback from @Halalaluyafail3 (thanks!).

Ok, phew, that’s what I get for skipping a month again. Please don’t hesitate to report bugs or suggestions, here or via separate posts. Due to our usual end-of-year code freeze there’s going to be one more recap at the end of the year where we will look back at 2020 and take a small peek into the future.

Updated: October 30, 2020






Luau Recap August 2020

August 11, 2020

As everyone knows by now, Luau is our new language stack that you can read more about at https://roblox.github.io/luau and the month following June is August so let’s talk about changes, big and small, that happened since June!

Many people work on these improvements, with the team slowly growing - thanks @Apakovtac, @EthicalRobot, @fun_enthusiast, @mrow_pizza and @zeuxcg!

[Originally posted on the Roblox Developer Forum.] Type annotations are safe to use in production!Permalink

When we started the Luau type checking beta, we’ve had a big warning sign in the post saying to not publish the type-annotated scripts to your production games which some of you did anyway. This was because we didn’t want to commit to specific syntax for types, and were afraid that changing the syntax would break your games.

This restriction is lifted now. All scripts with type annotations that parse & execute will continue to parse & execute forever. Crucially, for this to be true you must not be using old fat arrow syntax for functions, which we warned you about for about a month now:

Fat arrow deprecated

… and must not be using the __meta property which no longer holds special meaning and we now warn you about that:

meta deprecated

Part of the syntax finalization also involved changing the precedence on some type annotations and adding support for parentheses; notably, you can now mix unions and intersections if you know what that means ((A & B) | C is valid type syntax). Some complex type annotations changed their structure because of this - previously (number) -> string & (string) -> string was a correct way to declare an intersection of two function types, but now to keep it parsing the same way you need to put each function type in parentheses: ((number) -> string) & ((string) -> string).

Type checking is not out of beta yet - we still have some work to do on the type checker itself. The items on our list before going out of beta right now include: 

   Better type checking for unary/binary operators
   Improving error messages to make type errors more clear
   Fixing a few remaining crashes for complex scripts
   Fixing conflation of warnings/errors between different scripts with the same path in the tree
   Improving type checking of globals in nonstrict mode (strict mode will continue to frown upon globals)

Of course this doesn’t mark the end of work on the feature - after type checking goes out of beta we plan to continue working on both syntax and semantics, but that list currently represents the work we believe we have left to do in the first phase - please let us know if there are other significant issues you are seeing with beta beyond future feature requests! Format string analysisPermalink

A few standard functions in Luau are using format strings to dictate the behavior of the code. There’s string.format for building strings, string.gmatch for pattern matching, string.gsub’s replacement string, string.pack binary format specification and os.date date formatting.

In all of these cases, it’s important to get the format strings right - typos in the format string can result in unpredictable behavior at runtime including errors. To help with that, we now have a new lint rule that parses the format strings and validates them according to the expected format.

String format

Right now this support is limited to direct library calls (string.format("%.2f", ...) and literal strings used in these calls - we may lift some of these limitations later to include e.g. support for constant locals.

Additionally, if you have type checking beta enabled, string.format will now validate the argument types according to the format string to help you get your %ds and %ses right.

String format Improvements to string. libraryPermalink

We’ve upgraded the Luau string library to follow Lua 5.3 implementation; specifically: 

   string.pack/string.packsize/string.unpack are available for your byte packing needs
   string.gmatch and other pattern matching functions now support %g and \0 in patterns

This change also [inadvertently] makes string.gsub validation rules for replacement string stricter - previously % followed by a non-digit character was silently accepted in a replacement string, but now it generates an error. This accidentally broke our own localization script Purchase Prompt broken in some games (% character in title)), but we got no other reports, and this in retrospect is a good change as it makes future extensions to string replacement safe… It was impossible for us to roll the change back and due to a long release window because of an internal company holiday we decided to keep the change as is, although we’ll try to be more careful in the future.

On a happier note, string.pack may seem daunting but is pretty easy to use to pack binary data to reduce your network traffic (note that binary strings aren’t safe to use in DataStores currently); I’ve posted an example in the release notes thread Release Notes for 441 that allows you to pack a simple character state in 16 bytes like this:

local characterStateFormat = "fffbbbB"

local characterState = string.pack(characterStateFormat, 

   posx, posy, posz, dirx * 127, diry * 127, dirz * 127, health)

And unpack it like this after network transmission:

local posx, posy, posz, dirx, diry, dirz, health = 

   string.unpack(characterStateFormat, characterState)

dirx /= 127 diry /= 127 dirz /= 127

Assorted fixesPermalink

As usual we fixed a few small problems discovered through testing. We now have an automated process that generates random Luau code in semi-intelligent ways to try to break different parts of our system, and a few fixes this time are a direct result of that. 

   Fix line debug information for multi-line function calls to make sure errors for code like foo.Bar(...) are generated in the appropriate location when foo is nil
   Fix debug information for constant upvalues; this fixes some bugs with watching local variables from the nested functions during debugging
   Fix an off-by-one range check in string.find for init argument that could result in reading uninitialized memory
   Fix type confusion for table.move target table argument that could result in reading or writing arbitrary memory
   Fix type confusion for debug.getinfo in some circumstances (we don’t currently expose getinfo but have plans to do so in the future)
   Improve out of memory behavior for large string allocations in string.rep and some other functions like table.concat to handle these conditions more gracefully
   Fix a regression with os.time from last update, where it erroneously reverted to Lua 5.x behavior of treating the time as a local time. Luau version (intentionally) deviates from this by treating the input table as UTC, which matches os.time() behavior with no arguments.

Performance improvementsPermalink

Only two changes in this category here this time around; some larger scale performance / memory improvements are still pending implementation. 

   Constant locals are now completely eliminated in cases when debugging is not available (so on server/client), making some scripts ~1-2% faster
   Make script compilation ~5% faster by tuning the compiler analysis and code generation more carefully Oh, also math.round is now a thing which didn’t fit into any category above.

Updated: August 11, 2020






Luau Recap: June 2020

June 20, 2020

… otherwise known as “This Month in Luau” I guess? You know the drill by now. We’ll talk about exciting things that happened to Luau - our new language stack.

anxiously glances at FIB3 thread that casts a huge shadow on this announcement, but hopefully somebody will read this

Many people work on these improvements; thanks @Apakovtac, @EthicalRobot, @fun_enthusiast, @zeuxcg!

[Originally posted on the Roblox Developer Forum.] We have a website!Permalink

Many developers told us on many occasions that as much as they love the recaps, it’s hard to know what the state of the language or libraries is if the only way to find out is to read through all updates. What’s the syntax extensions that Luau supports now? How do I use type checking? What’s the status of from Lua 5.x?

Well, you can find all of this out here now: https://roblox.github.io/luau/

Please let us know if this documentation can be improved - what are you missing, what could be improved. For now to maximize change velocity this documentation is separate from DevHub; it’s also meant as an external resource for people who don’t really use the language but are curious about the differences and novelties.

Also, _VERSION now returns “Luau” because we definitely aren’t using Lua 5.1 anymore. Compound assignmentsPermalink

A long-standing feature request for Lua is compound assignments. Somehow Lua never got this feature, but Luau now implements +=, -=, *=, /=, %=, ^= and ..= operators. We decided to implement them because they are absolutely ubiquitous among most frequently used programming languages, both those with C descent and those with different lineage (Ruby, Python). They result in code that’s easier to read and harder to make mistakes in.

We do not implement ++ and --. These aren’t universally adopted, -- conflicts with comment syntax and they are arguably not as intuitively obvious. We trust everyone to type a few extra characters for += 1 without too much trouble.

Two important semantical notes are that the expressions on the left hand side are only evaluated once, so for example table[makeIndex()] += 1 only runs makeIndex once, and that compound assignments still call all the usual metamethod (__add et al, and __index/__newindex) when necessary - you don’t need to change any data structures to work with these.

There’s no noticeable performance improvement from these operators (nor does using them carry a cost) - use them when they make sense for readability. Nicer error messagesPermalink

Good errors are critical to be able to use Luau easily. We’ve spent some time to improve the quality of error messages during parsing and runtime execution: 

   In runtime type errors, we now often use the “Roblox” type name instead of plain userdata, e.g. math.abs(v) now says number expected, got Vector3
   When arguments are just missing, we now explicitly say that they are missing in libraries like math/table; the old message was slightly more confusing
   string.format in some cases produced error messages that confused missing arguments for incorrect types, which has been fixed
   When a builtin function such as math.abs fails, we now add the function name to the error message. This is something that used to happen in Lua, then we lost this in Luau because Luau removes a very fragile mechanism that supported that, but we now have a new, robust way to report this so you can have the function name back! The message looks like this now: invalid argument #1 to 'abs' (number expected, got nil)
   In compile-time type errors, we now can identify the case when the field was mistyped with a wrong case (ha), and tell you to use the correct case instead.
   When you forget an end statement, we now try to be more helpful and point you to the problematic statement instead of telling you that the end is missing at the very end of the program. This one is using indentation as a heuristic so it doesn’t always work perfectly.
   We now have slightly more helpful messages for cases when you forget parentheses after a function call

   We now have slightly more helpful messages for some cases when you accidentally use ( ... ) instead of { ... } to create a table literal Additionally two places had very lax error checking that made the code more fragile, and we fixed those:
   xpcall now fails immediately when the error function argument is not a function; it used to work up until you get an error, and failed at that point, which made it hard to find these bugs
   tostring now enforces the return type of the result to be a string - previously __tostring could return a non-string result, which worked fine up until you tried to do something like passing the resulting value to string.format for %s. Now tostring will fail early. Our next focus here is better error messages during type checking - please let us know if there are other errors you find confusing and we could improve!

Type checker improvementsPermalink

We’re getting closer and closer to be able to move out of beta. A big focus this month was on fixing all critical bugs in the type checker - it now should never hang or crash Studio during type checking, which took a bit of work to iron out all the problems.

Notably, typing function string.length no longer crashes Studio (although why you’d do that is unclear), and Very Large Scripts With Tons Of Nested Statements And Expressions should be stable as well.

We’ve also cleaned up the type information for builtin libraries to make it even more precise, including a few small fixes to string/math functions, and a much more precise coroutine library type information. For the latter we’ve introduced a primitive type thread, which is what coroutine library works with. Linter improvementsPermalink

Linter is the component that produces warnings about scripts; it’s otherwise known as “Static Analysis” in Studio, although that is now serving as a place where we show type errors as well.

Most of the changes here this month are internal as they concern warnings that aren’t yet enabled in Studio (the web site linked above documents all warnings including ones that aren’t active yet but may become active), but once notable feature is that you can now opt out of individual warnings on a script-by-script basis by adding a –!nolint comment to the top of the script. For example, if you really REALLY REALLY like the Game global, you can add this to the top of the script:

--!nolint DeprecatedGlobal

Or, if you basically just want us to not issue any warnings ever, I guess you can add this:

--!nocheck --!nolint

and live happily ignorant of all possible errors up until you run your code. (please don’t do that) os. enhancementsPermalink

Our overall goal is to try to be reasonably compatible with Lua 5.x in terms of library functions we expose. This doesn’t always work - in some cases we have to remove library features for sandboxing reasons, and in others the library functions don’t make sense in context of Roblox. However, some of these decisions can be revised later. In particular, when we re-added os. library to Roblox, we limited it to os.date, os.time and os.difftime (although why difftime is a thing isn’t clear), omitting os.clock and restricting inputs to os.date to return a table with date components, whereas Lua 5.x supports format strings.

Well, this changes today. os.clock is now available if you need a high-precision time for benchmarking, and os.date can now return formatted date using Lua 5.x format string that you can read about here https://www.lua.org/pil/22.1.html (we support all these specifiers: aAbBcdHIjmMpSUwWxXyYzZ).

While os.date() is hopefully welcome, os.clock may raise some eyebrows - aren’t there enough timing functions in Roblox already? Well, this is nice if you are trying to port code from Lua 5.x to Luau, and there’s this

Oblig. xkcd

But really, most existing Roblox timing functions are… problematic. 

   time() returns the total amount of time the game has been running simulation for, it’s monotonic and has reasonable precision. It’s fine - you can use it to update internal gameplay systems without too much trouble. It should’ve been called “tick” perhaps but that ship has sailed.
   elapsedTime and its close cousin ElapsedTime, are telling you “how much time has elapsed since the current instance of Roblox was started.”. While technically true, this isn’t actually useful because on mobile the “start” time here can be days in the past. It’s also inadequate for performance measurements as on Windows, it has a 1ms resolution which isn’t really enough for anything interesting. We’re going to deprecate this in the future.
   tick() sounds perfect - it has a high resolution (usually around 1 microsecond), and a well-defined baseline - it counts since UNIX epoch! Or, well, it actually doesn’t. On Windows, it returns you a variant of the UNIX timestamp in local time zone. In addition, it can be off by 1 second from the actual, real UNIX timestamp, and might have other idiosyncrasies on non-Windows platforms. We’re going to deprecate this in the future

So, if you need a UNIX timestamp, you should use os.time(). You get a stable baseline (from 1970’s) and 1s resolution. If you need to measure performance, you should use os.clock(). You don’t get a stable baseline, but you get ~1us resolution. If you need to do anything else, you should probably use time(). Performance optimizationsPermalink

As you can never have too much performance, we’re continuing to work on performance! We’re starting to look into making Vector3 faster and improving the garbage collector, with some small changes already shipping, but overall it’s a long way out so here are the things that did get visibly better: 

   A few string. methods, notably string.byte and string.char, were optimized to make it easier to write performant deserialization code. string.byte is now ~4x faster than before for small numbers of returned characters. For optimization to be effective, it’s important to call the function directly ( string.byte(foo, 5) ) instead of using method calls ( foo:byte(5) )
   Optimize coroutine resumption, making some code that is heavily reliant on coroutine. library ~10% faster. We have plans to improve this further, watch this space.
   Optimize typeof() to run ~6x faster. It used to be that type() was much faster than typeof() but they now should be more or less comparable.
   Some secret internal optimizations make some scripts a few percent faster
   The memory allocator used in Luau was rewritten using a new, more efficient, implementation. There might be more changes here in the future to save some memory, but for now this makes some allocation-intensive benchmarks ~15% faster.
   Using tables with keys that are not strings or numbers is a fair bit more efficient now (most commonly comes up when Instance is used as a key in a hash table), on par with using strings.

Also we found a bug with some of our optimizations (which delayed the string. performance improvement above, but also could affect some math. calls) where in some complex functions you would see valid calls to math. etc. breaking with non-sensical errors such as “expected number, got table” - this has been fixed! Memory optimizationsPermalink

As with performance, our goal here is simple - the more efficient internal Luau structures can become, the less memory will Lua heap take. This is great for both memory consumption, and for garbage collection performance as the collector needs to traverse less data. There’s a few exciting changes in this area this month: 

   Non-array-like tables now take 20% less space. This doesn’t affect arrays but can be observed on object-like tables, both big and small. This is great because some of you are using a lot of large tables apparently, since this resulted in very visible reduction in overall Lua heap sizes across all games.
   Function objects now take up to 30% less space. This isn’t as impactful since typically function objects are not created very frequently and/or don’t live for very long, but it’s nice nonetheless.
   New allocator mentioned in the previous section can save up to 5-6% of Lua heap memory as well, although these gains are highly dependent on the workload, and we usually see savings in the 1-2% range.

And that’s it! Till next time. As usual let us know if you have questions, suggestions or bug reports.

Updated: June 20, 2020






Luau Recap: May 2020

May 18, 2020

Luau (lowercase u, “l-wow”) is an umbrella initiative to improve our language stack - the syntax, compiler, virtual machine, builtin Lua libraries, type checker, linter (known as Script Analysis in Studio), and more related components. We continuously develop the language and runtime to improve performance, robustness and quality of life. Here we will talk about all things that happened since the update in March!

[Originally posted on the Roblox Developer Forum.] New function type annotation syntaxPermalink

As noted in the previous update, the function type annotation syntax now uses : on function definitions and -> on standalone function types:

type FooFunction = (number, number) -> number

function foo(a: number, b: number): number 

   return a + b

end

This was done to make our syntax more consistent with other modern languages, and is easier to read in type context compared to our old =>.

This change is now live; the old syntax is still accepted but it will start producing warnings at some point and will be removed eventually. Number of locals in each function is now limited to 200Permalink

As detailed in Upcoming change to (correctly) limit the local count to 200 (which is now live), when we first shipped Luau we accidentally set the local limit to 255 instead of 200. This resulted in confusing error messages and code that was using close to 250 locals was very fragile as it could easily break due to minor codegen changes in our compiler.

This was fixed, and now we’re correctly applying limits of 200 locals, 200 upvalues and 255 registers (per function) - and emit proper error messages pointing to the right place in the code when either limit is exceeded.

This is technically a breaking change but scripts with >200 locals didn’t work in our old VM and we felt like we had to make this change to ensure long-term stability. Require handling improvements in type checker + export typePermalink

We’re continuing to flesh out the type checker support for modules. As part of this, we overhauled the require path tracing - type checker is now much better at correctly recognizing (statically) which module you’re trying to require, including support for game:GetService.

Additionally, up until now we have been automatically exporting all type aliases declared in the module (via type X = Y); requiring the module via local Foo = require(path) made these types available under Foo. namespace.

This is different from the explicit handling of module entries, that must be added to the table returned from the ModuleScript. This was highlighted as a concern, and to fix this we’ve introduced export type syntax.

Now the only types that are available after require are types that are declared with export type X = Y. If you declare a type without exporting it, it’s available inside the module, but the type alias can’t be used outside of the module. That allows to cleanly separate the public API (types and functions exposed through the module interface) from implementation details (local functions etc.). Improve type checker robustnessPermalink

As we’re moving closer to enabling type checking for everyone to use (no ETA at the moment), we’re making sure that the type checker is as robust as possible.

This includes never crashing and always computing the type information in a reasonable time frame, even on obscure scripts like this one:

type ( ... ) ( ) ; ( ... ) ( - - ... ) ( - ... ) type = ( ... ) ; ( ... ) ( ) ( ... ) ; ( ... ) ""

To that end we’ve implemented a few changes, most of them being live, that fix crashes and unbounded recursion/iteration issues. This work is ongoing, as we’re fixing issues we encounter in the testing process. Better types for Lua and Roblox builtin APIsPermalink

In addition to improving the internals of the type checker, we’re still working on making sure that the builtin APIs have correct type information exposed to the type checker.

In the last few weeks we’ve done a major audit and overhaul of that type information. We used to have many builtin methods “stubbed” to have a very generic type like any or (...) -> any, and while we still have a few omissions we’re much closer to full type coverage.

One notable exception here is the coroutine. library which we didn’t get to fully covering, so the types for many of the functions there are imprecise.

If you find cases where builtin Roblox APIs have omitted or imprecise type information, please let us know by commenting on this thread or filing a bug report.

The full set of types we expose as of today is listed here for inquisitive minds: https://gist.github.com/zeux/d169c1416c0c65bb88d3a3248582cd13 Removal of __gc from the VMPermalink

A bug with continue and local variables was reported to us a few weeks ago; the bug was initially believed to be benign but it was possible to turn this bug into a security vulnerability by getting access to __gc implementation for builtin Roblox objects. After fixing the bug itself (the turnaround time on the bug fix was about 20 hours from the bug report), we decided to make sure that future bugs like this don’t compromise the security of the VM by removing __gc.

__gc is a metamethod that Lua 5.1 supports on userdata, and future versions of Lua extend to all tables; it runs when the object is ready to be garbage collected, and the primary use of that is to let the userdata objects implemented in C to do memory cleanup. This mechanism has several problems: 

   __gc is invoked by the garbage collector without context of the original thread. Because of how our sandboxing works this means that this code runs at highest permission level, which is why __gc for newproxy-created userdata was disabled in Roblox a long time ago (10 years?)
   __gc for builtin userdata objects puts the object into non-determinate state; due to how Lua handles __gc in weak keys (see https://www.lua.org/manual/5.2/manual.html#2.5.2), these objects can be observed by external code. This has caused crashes in some Roblox code in the past; we changed this behavior at some point last year.
   Because __gc for builtin objects puts the object into non-determinate state, calling it on the same object again, or calling any other methods on the object can result in crashes or vulnerabilities where the attacker gains access to arbitrarily mutating the process memory from a Lua script. We normally don’t expose __gc because the metatables of builtin objects are locked but if it accidentally gets exposed the results are pretty catastrophic.
   Because __gc can result in object resurrection (if a custom Lua method adds the object back to the reachable set), during garbage collection the collector has to traverse the set of userdatas twice - once, to run __gc and a second time to mark the survivors.

For all these reasons, we decided that the __gc mechanism just doesn’t pull its weight, and completely removed it from the VM - builtin userdata objects don’t use it for memory reclamation anymore, and naturally declaring __gc on custom userdata objects still does nothing.

Aside from making sure we’re protected against these kinds of vulnerabilities in the future, this makes garbage collection ~25% faster. Memory and performance improvementsPermalink

It’s probably not a surprise at this point but we’re never fully satisfied with the level of performance we get. From a language implementation point of view, any performance improvements we can make without changing the semantics are great, since they automatically result in Lua code running faster. To that end, here’s a few changes we’ve implemented recently: 

   A few string. methods, notably string.byte and string.char, were optimized to make it easier to write performant deserialization code. string.byte is now ~4x faster than before for small numbers of returned characters. For optimization to be effective, it’s important to call the function directly (string.byte(foo, 5)) instead of using method calls (foo:byte(5)). This had to be disabled due to a rare bug in some cases, this optimization will come back in a couple of weeks.
   table.unpack was carefully tuned for a few common cases, making it ~15% faster; unpack and table.unpack now share implementations (and the function objects are equal to each other).
   While we already had a very efficient parser, one long standing bottleneck in identifier parsing was fixed, making script compilation ~5% faster across the board, which can slightly benefit server startup times.
   Some builtin APIs that use floating point numbers as arguments, such as various Vector3 constructors and operators, are now a tiny bit faster.
   All string objects are now 8 bytes smaller on 64-bit platforms, which isn’t a huge deal but can save a few megabytes of Lua heap in some games.
   Debug information is using a special compact format that results in ~3.2x smaller line tables, which ends up making function bytecode up to ~1.5x smaller overall. This can be important for games with a lot of scripts.
   Garbage collector heap size accounting was cleaned up and made more accurate, which in some cases makes Lua heap ~10% smaller; the gains highly depend on the workload.

Library changesPermalink

The standard library doesn’t see a lot of changes at this point, but we did have a couple of small fixes here: 

   coroutine.wrap and coroutine.create now support C functions. This was the only API that treated Lua and C functions differently, and now it doesn’t.
   require silently skipped errors in module scripts that occurred after the module scripts yielding at least once; this was a regression from earlier work on yieldable pcall and has been fixed.

As usual, if you have questions, comments, or any other feedback on these changes, feel free to share it in this thread or create separate posts for bug reports.

Updated: May 18, 2020






Luau Recap: February 2020

February 25, 2020

We continue to iterate on our language stack, working on many features for type checking, performance, and quality of life. Some of them come with announcements, some come with release notes, and some just ship - here we will talk about all things that happened since November last year.

A lot of people work on these improvements; thanks @Apakovtac, @EthicalRobot, @fun_enthusiast, @xyzzyismagic, @zeuxcg!

[Originally posted to the Roblox Developer Forum.]

We were originally intending to ship the beta last year but had to delay it due to last minute bugs. However, it’s now live as a beta option on production! Go here to learn more:

EDIT: Please DO NOT publish places with type annotations just yet as they will not work on production! This is why it’s a beta 🙂 However, please continue to experiment in Studio and give us feedback. We are reading everything and will be fixing reported bugs and discussing syntax / semantics issues some people brought up. Hello! We’ve been quietly working on building a type checker for Lua for quite some time now. It is now far enough along that we’d really like to hear what…

We’re continuing to iterate on the feedback we have received here. Something that will happen next is that we will enable type annotations on live server/clients - which will mean that you will be able to publish source code with type annotations without breaking your games. We still have work to do on the non-strict and strict mode type checking before the feature can move out of beta though, in particular we’ve implemented support for require statement and that should ship next week 🤞

We also fixed a few bugs in the type definitions for built-in functions/API and the type checker itself: 

   table.concat was accidentally treating the arguments as required
   string.byte and string.find now have a correct precise type
   typeof comparisons in if condition incorrectly propagated the inferred type into elseif branches

We are also making the type checker more ergonomic and more correct. Two changes I want to call out are: 

   Type aliases declared with type X = Y are now co-recursive, meaning that they can refer to each other, e.g.

type array<T> = { [number]: T }

type Wheel = { radius: number, car: Car } type Car = { wheels: array<Wheel> } 

   We now support type intersections (A & B) in addition to type unions (A | B). Intersections are critical to modeling overloaded functions correctly - while Lua as a language doesn’t support function overloads, we have various APIs that have complex overloaded semantics - one of them that people who tried the beta had problems with was UDim2.new - and it turns out that to correctly specify the type, we had to add support for intersections. This isn’t really intended as a feature that is used often in scripts developers write, but it’s important for internal use.

Debugger (beta)Permalink

When we shipped the original version of the VM last year, we didn’t have the debugger fully working. Debugger relies on low-level implementation of the old VM that we decided to remove from the new VM - as such we had to make a new low-level debugging engine.

This is now live under the Luau VM beta feature, see this post for details.

If you use the debugger at all, please enable the beta feature and try it out - we want to fix all the bugs we can find, and this is blocking us enabling the new VM everywhere.

(a quick aside: today the new VM is enabled on all servers and all clients, and it’s enabled in Studio “edit” mode for plugins - but not in Studio play modes, and full debugger support is what prevents us from doing so) LanguagePermalink

This section is short and sweet this time around: 

   You can now use continue statement in for/while/repeat loops. :tada:

Please note that we only support this in the new VM, so you have to be enrolled in Luau VM beta to be able to use it in Studio. It will work in game regardless of the beta setting as per above. PerformancePermalink

While we have some really forward looking ideas around multi-threading and native code compilation that we’re starting to explore, we also continue to improve performance across the board based on our existing performance backlog and your feedback.

In particular, there are several memory and performance optimizations that shipped in the last few months: 

   Checking for truth (if foo or foo and bar) is now a bit faster, giving 2-3% performance improvements on some benchmarks
   table.create (with value argument) and table.pack have been reimplemented and are ~1.5x faster than before
   Internal mechanism for filling arrays has been made faster as well, which makes Terrain:ReadVoxels ~10% faster
   Catching engine-generated errors with pcall/xpcall is now ~1.5x faster (this only affects performance of calls that generated errors)
   String objects now take 8 bytes less memory per object (and in an upcoming change we’ll save a further 4 bytes)
   Capturing local variables that are never assigned to in closures is now much faster, takes much less memory and generates much less GC pressure. This can make closure creation up to 2x faster, and improves some Roact benchmarks by 10%. This is live in Studio and will ship everywhere else shortly.
   The performance of various for loops (numeric & ipairs) on Windows regressed after a VS2017 upgrade; this regression has been fixed, making all types of loops perform roughly equally. VS2017 upgrade also improved Luau performance on Windows by ~10% across the board.
   Lua function calls have been optimized a bit more, gaining an extra 10% of performance in call-heavy benchmarks on Windows.
   Variadic table constructors weren’t compiled very efficiently, resulting in surprisingly low performance of constructs like {...}. Fixing that made {...} ~3x faster for a typical number of variadic arguments.

DiagnosticsPermalink

We spent some time to improve error messages in various layers of the stack based on the reports from community. Specifically: 

   The static analysis warning about incorrect bounds for numeric for loops is now putting squigglies in the right place.
   Fixed false positive static analysis warnings about unreachable code inside repeat…until loops in certain cases.
   Multiline table construction expressions have a more precise line information now which helps in debugging since callstacks are now easier to understand
   Incomplete statements (e.g. foo) now produce a more easily understandable parsing error
   In some cases when calling the method with a . instead of :, we emitted a confusing error message at runtime (e.g. humanoid.LoadAnimation(animation)). We now properly emit the error message asking the user if : was intended.
   The legacy global ypcall is now flagged as deprecated by script analysis
   If you use a Unicode symbol in your source program outside of comments or string literals, we now produce a much more clear message, for example:

   local pi = 3․13 -- spoiler alert: this is not a dot!

   produces Unexpected Unicode character: U+2024. Did you mean '.'?

LoadLibrary removalPermalink

Last but not least, let’s all press F for LoadLibrary.

It was fun while it lasted, but supporting it caused us a lot of pain over the years and prevented some forward-looking changes to the VM. We don’t like removing APIs from the platform, but in this case it was necessary. Thanks to the passionate feedback from the community we adjusted our initial rollout plans to be less aggressive and batch-processed a lot of gear items that used this function to stop using this function. The update is in effect and LoadLibrary is no more.

As usual, if you have any feedback about any of these updates, suggestions, bug reports, etc., post them in this thread or (preferably for bugs) as separate posts in the bug report category.

Updated: February 25, 2020






Luau Type Checking Beta

January 16, 2020

Hello!

We’ve been quietly working on building a type checker for Lua for quite some time now. It is now far enough along that we’d really like to hear what you think about it.

I am very happy to offer a beta test into the second half of the Luau effort.

[Originally posted on the Roblox Developer Forum.] Beta TestPermalink

First, a word of caution: In this test, we are changing the syntax of Lua. We are pretty sure that we’ve mostly gotten things right, but part of the reason we’re calling this a beta is that, if we learn that we’ve made a mistake, we’re going to go back and fix it even if it breaks compatibility.

Please try it out and tell us what you think, but be aware that this is not necessarily our final form. 🙂

Beta testers can try it out by enabling the “Enable New Lua Script Analysis” beta feature in Roblox Studio. OverviewPermalink

Luau is an ahead-of-time typechecking system that sits atop ordinary Lua code. It does not (yet) feed into the runtime system; it behaves like a super powerful lint tool to help you find bugs in your code quickly.

It is also what we call a gradual type system. This means that you can choose to add type annotations in some parts of your code but not others. Two ModesPermalink

Luau runs in one of two modes: strict, and nonstrict. Nonstrict ModePermalink

Nonstrict mode is intended to be as helpful as possible for programs that are written without type annotations. We want to report whatever we can without reporting an error in reasonable Lua code. 

   If a local variable does not have a type annotation and it is not initially assigned a table, its type is any
   Unannotated function parameters have type any
   We do not check the number of values returned by a function
   Passing too few or too many arguments to a function is ok

Strict ModePermalink

Strict mode is expected to be more useful for more complex programs, but as a side effect, programs may need a bit of adjustment to pass without any errors. 

   The types of local variables, function parameters, and return types are deduced from how they are used
   Errors are produced if a function returns an inconsistent number of parameters, or if it is passed the wrong number of arguments

Strict mode is not enabled by default. To turn it on, you need to add a special comment to the top of your source file.

--!strict

New syntaxPermalink

You can write type annotations in 5 places: 

   After a local variable
   After a function parameter
   After a function declaration (to declare the function’s return type)
   In a type alias, and
   After an expression using the new as keyword.

local foo: number = 55

function is_empty(param: string) => boolean 

   return 0 == param:len()

end

type Point = {x: number, y: number}

local baz = quux as number

Type syntaxPermalink Primitive typesPermalink

nil, number, string, and boolean anyPermalink

The special type any signifies that Luau shouldn’t try to track the type at all. You can do anything with an any. TablesPermalink

Table types are surrounded by curly braces. Within the braces, you write a list of name: type pairs:

type Point = {x: number, y: number}

Table types can also have indexers. This is how you describe a table that is used like a hash table or an array.

type StringArray = {[number]: string}

type StringNumberMap = {[string]: number}

FunctionsPermalink

Function types use a => to separate the argument types from the return types.

type Callback = (string) => number

If a function returns more than one value, put parens around them all.

type MyFunction = (string) => (boolean, number)

UnionsPermalink

You can use a | symbol to indicate an “or” combination between two types. Use this when a value can have different types as the program runs.

function ordinals(limit) 

   local i = 0
   return function() => number | nil
       if i < limit then
           local t = i
           i = i + 1
           return t
       else
           return nil
       end
   end

end

OptionsPermalink

It’s pretty commonplace to have optional data, so there is extra syntax for describing a union between a type and nil. Just put a ? on the end. Function arguments that can be nil are understood to be optional.

function foo(x: number, y: string?) end

foo(5, 'five') -- ok foo(5) -- ok foo(5, 4) -- not ok

Type InferencePermalink

If you don’t write a type annotation, Luau will try to figure out what it is.

--!strict local Counter = {count=0}

function Counter:incr() 

   self.count = 1
   return self.count

end

print(Counter:incr()) -- ok print(Counter.incr()) -- Error! print(Counter.amount) -- Error!

Future PlansPermalink

This is just the first step!

We’re excited about a whole bunch of stuff: 

   Nonstrict mode is way more permissive than we’d like
   Generics!
   Editor integration

Updated: January 16, 2020






Luau Recap: November 2019

November 11, 2019

A few months ago, we’ve released our new Lua implementation, Luau (Faster Lua VM Released) and made it the default for most platforms and configurations. Since then we’ve shipped many smaller changes that improved performance and expanded the usability of the VM. Many of them have been noted in release notes but some haven’t, so here’s a recap of everything that has happened in the Lua land since September!

[Originally posted to the Roblox Developer Forum.] Debugger betaPermalink

When we launched the new VM, we did it without the full debugger support. The reason for this is that the new VM is substantially different and the old implementation of the debugger (that relied on line hooks) just doesn’t work.

We had to rebuild the low level implementation of the debugger from scratch - this is a tricky problem and it took time! We are excited to share a beta preview of this with you today.

To use this, simply make sure that you’re enrolled in the new Lua VM beta:

Enable New Lua VM

After this you can use the debugger as usual. If you see any bugs, please feel free to report them! Performance improvementsPermalink 

   The for loop optimization that specializes pairs/ipairs now works for localized versions of these globals as well, as well as next, table expressions
   a^k expressions are now faster for some trivial values of k such as 2 and 0.5
   Calling methods and accessing properties on deeply nested Roblox objects is now significantly faster than it used to be (~2x faster for objects that have an 8-deep nesting) - the cost is now independent of the hierarchy depth.
   Accessing .X/.Y/.Z properties on Vector types is now ~30% faster
   On Windows and Xbox, we’ve tuned our interpreter to be ~5-6% faster on Lua-intensive code
   For a set of builtin functions, we now support very quickly calling them from VM via a new fastcall mechanism.

Fastcall requires the function call to be present in source as a global or localized global access (e.g. either math.max(x, 1) or max(x, 1) where local max = math.max). This can be substantially faster than normal calls, e.g. this makes SHA256 benchmark ~1.7x faster. We are currently optimizing calls to bit32, math libraries and additionally assert and type. Also, just like other global-based optimizations, this one is disabled if you use getfenv/setfenv. Lua library extensionsPermalink

We’ve implemented most library features available in later versions of upstream Lua, including: 

   table.pack and table.unpack from Lua 5.2 (the latter is same as global unpack, the former helps by storing the true argument count in .n field)
   table.move from Lua 5.3 (useful for copying data between arrays)
   coroutine.isyieldable from Lua 5.3
   math.log now accepts a second optional argument (as seen in Lua 5.2) for the logarithm base

We’ve also introduced two new functions in the table library: 

   table.create(count, value) can create an array-like table quickly
   table.find(table, value [, init]) can quickly find the numeric index of the element in the table

Autocomplete support for table.create/table.find will ship next week Lua syntax extensionsPermalink

We’ve started taking a look at improving the Lua syntax. To that end, we’ve incorporated a few changes from later versions of Lua into the literal syntax: 

   String literals now support \z (skip whitespace), \x (hexadecimal byte) and \u (Unicode codepoint) escape sequences

and implemented a few extra changes: 

   Number literals now support binary literals, e.g. 0b010101
   Number literals now support underscores anywhere in the literal for easier digit grouping, e.g. 1_000_000

Note that the literal extensions aren’t currently supported in syntax highlighter in Studio but this will be amended soon. Error messagesPermalink

Error messages are slowly getting a bit of love. We’ve improved some runtime errors to be nicer, in particular: 

   When indexing operation fails, we now specify the key name or type, e.g. “attempt to index foo with ‘Health’”
   When arithmetic operations fails, we now specify the type of arithmetic operation, e.g. “attempt to perform arithmetic (add) on table and number”

We’ve also improved some parse errors to look nicer by providing extra context - for example, if you forget parentheses after function name in a function declaration, we will now say Expected '(' when parsing function, got 'local'.

We are looking into some reports of misplaced line numbers on errors in multi-line expressions but this will only ship later. Correctness fixesPermalink

There are always a few corner cases that we miss - a new Lua implementation is by necessity subtly different in a few places. Our goal is to find and correct as many of these issues as possible. In particular, we’ve: 

   Fixed some cases where we wouldn’t preserve negative zero (-0)
   Fixed cases where getfenv(0) wouldn’t properly deoptimize access to builtin globals
   Fixed cases where calling a function with >255 parameters would overflow the stack
   Fixed errors with very very very long scripts and control flow around large blocks (thousands of lines of code in a single if/for statement)
   Fixed cases where in Studio on Windows, constant-time comparisons with NaNs didn’t behave properly (0/0==1)

Also, the upvalue limit in the new VM has been raised to 200 from 60; the limit in Lua 5.2 is 255 but we decided for now to match the local limit. Script analysisPermalink

Along with the compiler and virtual machine, we’ve implemented a new linting framework on top of Luau which is similar to our old script analysis code but is richer. In particular, we support a few more checks that are enabled by default: 

   Unreachable code warning, for cases where function provably doesn’t reach a specific point, such as redundant return after a set of if/else statements where every branch returns or errors.
   Unknown type warning, which was emitted before for Instance.new/GetService/IsA calls, is now also emitted when the result of type/typeof is compared to a string literal
   We now recognize and flag mistaken attempts to iterate downwards with a for loop (such as for i=9,1 or for i=#t,1 as well as cases where numeric for loop doesn’t reach the stated target (for i=1,4.5)
   We now detect and flag cases where in assignment expressions variables are implicitly initialized with nil or values are dropped during assignment
   “Statement spans multiple lines” warning now does not trigger on idiomatic constructs involving setting up locals in a block (local name do ... name = value ... end)

We also have implemented a few more warnings for common style/correctness issues but they aren’t enabled yet - we’re looking into ways for us to enable them without too much user impact: 

   Local variables that shadow other local variables / global variables
   Local variables that are assigned but never used
   Implicit returns, where functions that explicitly return values in some codepaths can reach the end of the function and implicitly return no values (which is error-prone)

Future plansPermalink

There’s a fair bit of performance improvements that we haven’t gotten to yet that are on the roadmap - this includes general VM optimizations (faster function calls, faster conditional checks, faster error handling including pcall) and some library optimizations (in particular, Vector3 math performance improvements). And we’re starting to look into some exciting ways for us to make performance even better in the future.

Also we’re still working on the type system! It’s starting to take shape and we should have something ready for you by the end of the year, but you’ll learn about it in a separate post :smiley:

As always don’t hesitate to reach out if you have any issues or have any suggestions for improvements.

Updated: November 11, 2019




Retrieved from ‘https://wiki.gideros.rocks/index.php?title=Latest_Luau&oldid=23886’