What a Rust closure turns into

On June 11, 2014, Niko Matsakis opened a pull request against Rust’s design documents, its RFCs, titled “Unboxed closures”. Rust then had two ways to write a function value on the spot. A closure, with a type written like |int| -> int, borrowed the variables it used from the code around it. A proc took them by value, so it could outlive that code, for example as the body of a new task. The RFC says there “was sometimes a lack of clarity about which would be the appropriate choice”, and set out to “unify closures, procs, and traits”. It proposed removing proc and was merged on July 30, 2014. On keeping proc around, its verdict was “Frankly, the main objection to this is that nobody likes the proc keyword.”

That RFC, and three more written over the next two years, explain this program and one more thing you can do with it:

1
fn main() {
2
    let add_one = |x| x + 1;
3
    println!("{}", add_one(5));
4
}

It prints 6. All output below is from Rust 1.95 nightly.

A closure is a struct

The RFC’s answer to “what is a closure?” was that it is nothing special:

Closures are always translated into a fresh struct type with one field per upvar.

An upvar is a variable from the surrounding code that the closure uses. A closure that reads a local n gets a struct with a field holding n, or a reference to it. Since edition 2021 a closure can capture a single field such as p.x instead of all of p, but the idea is the same. “Fresh” means every closure expression gets its own type, one nobody else shares. Today’s Rust Reference puts it as “a unique, anonymous type that cannot be written out”.

add_one uses only its own parameter, so its struct has no fields. Add println!("{}", std::mem::size_of_val(&add_one)); to the program and it prints 0. The variable add_one holds a value that takes no memory at all.

A struct can’t be called, though. So what does add_one(5) mean?

A call is a trait method

Rust already let a type decide what + means for it, by implementing a trait called Add. The same RFC did that for the call parentheses. In its words, “function calls become an overridable operator”, and a(b, c, d) becomes a call to one of three trait methods.

There are three because a closure can need different access to its own fields. One that only reads them can be called through a shared reference, as often as you like. One that changes them, say a counter it bumps on each call, needs a mutable reference. One that hands a captured value away can run only once, because after that the value is gone. The RFC wrote the three like this:

Fn::call(&mut a, (b, c, d))           // today FnMut::call_mut
FnShare::call_share(&a, (b, c, d))    // today Fn::call
FnOnce::call_once(a, (b, c, d))       // still FnOnce::call_once

Today’s standard library spells them differently: the mutable one is FnMut, and the shared one has the plain name Fn. The arguments travel as one tuple, (b, c, d), so a single trait can cover calls with any number of arguments. The RFC considered a more general feature for that, generics that take any number of type parameters, and preferred the tuple as “a simpler model overall”.

The Reference says a closure “which does not mutate or move out of any captured variables implements Fn”. add_one captures nothing, so it implements Fn, and rustc compiles line 3’s call as:

<{closure} as Fn<(i32,)>>::call(&add_one, (5,))

Read it as “the call method from the closure type’s Fn implementation”. (5,) is a tuple with one element; the trailing comma is what makes it a tuple and not just 5 in parentheses. The closure is passed as &add_one, borrowed and not used up, so it could be called again on the next line. Written out by hand, the closure is roughly this. Stable Rust doesn’t let you implement the Fn traits yourself, so treat it as a sketch of what the compiler generates:

1
struct AddOne;                     // no fields: nothing captured
2

3
impl Fn<(i32,)> for AddOne {       // rustc treats the closure as if it had this, plus FnMut and FnOnce
4
    fn call(&self, (x,): (i32,)) -> i32 {
5
        x + 1
6
    }
7
}

That call names i32. The source never does, so where did it come from?

Where the i32 comes from

|x| x + 1 on its own gives x no type. The call add_one(5) ties x to the type of the literal 5, but that doesn’t settle anything either. A literal can carry its type as a suffix, as in 5u8, and this one doesn’t, so rustc only knows it is some integer. println! doesn’t narrow it: printing works for every integer type.

This exact situation was the subject of RFC 212, which Matsakis opened on August 26, 2014. The RFC tells the history. Early Rust gave a literal without a suffix the type int, so counting with unsigned integers meant writing 0u and 1u. Rust then added inference for literals and kept int as the fallback for when inference found nothing. Then the team measured, “found that in real world code this fallback was rarely used”, and removed it. The measurements had left out tests and other small programs, and those were where it hurt. The RFC’s example, shortened here, looks a lot like this program:

let mut c = 0;
loop {
    ...;
    c += 1;
}
println!("{}", c); // Does not constrain type of `c`

The RFC was merged on September 3, 2014, bringing back the old fallback, int, the pointer-sized integer of the time. A later revision, opened by a contributor named tbu- in November 2014 and merged that December, changed it to i32, giving among its reasons that i32 is “faster than 64-bit integers in arithmetic today” and takes half the memory of one. The text now reads “Integer literals whose type is unconstrained will default to i32”. So x is i32 because nothing in the program asks for anything else.

The fallback runs only once rustc has finished type-checking main and nothing has picked a type. It’s tempting to read line 3 as the place where x becomes i32, and to conclude that adding a call add_one(5u8) on the next line couldn’t compile. It does compile. With that line added, the u8 is the only thing in the program that picks a type, so rustc types the whole closure as u8. Its typed dump of the closure, which writes every value’s type out as an as cast, reads:

|x| ((x as u8) + (1 as u8) as u8)

A closure still gets one signature. The error comes when two calls pick different ones, as in add_one(5i32) followed by add_one(5u8). The start of rustc’s message:

error[E0308]: mismatched types
 --> main.rs:4:28
  |
4 |     println!("{}", add_one(5u8));
  |                    ------- ^^^ expected `i32`, found `u8`
  |                    |
  |                    arguments to this function are incorrect
  |
note: expected because the closure was earlier called with an argument of type `i32`

With x an i32, there is one more question about line 2: what does x + 1 do when x is already the largest i32?

What + does at the edge

Rust once defined its integer types as wrapping around on overflow, as C does for unsigned integers: one past the largest value comes back around to the smallest. RFC 560, dated June 30, 2014, written by Gábor Lehel and later edited by Matsakis, and merged in February 2015, gave its reason: overflow “is known to be a common source of error in other languages”, and even without memory-safety problems “it is still possible for overflow to lead to bugs”. It changed overflow into “a program error (but not undefined behavior in the C sense)”. In C, signed overflow is undefined behavior, which means the compiler may assume it never happens and the program can then do anything. Rust’s overflow does one of two defined things. The RFC left compilers free to choose when to check, with one requirement:

Implementations are required to at least check dynamically when debug_assert! assertions are enabled.

For +, it spells out the two outcomes: “When checking is enabled this will panic. When checking is disabled this will two’s complement wrap.”

Debug assertions are on in a debug build, so there rustc compiles the closure’s body to an addition that also reports whether it overflowed, followed by a check. As ordinary Rust:

1
fn call(&self, (x,): (i32,)) -> i32 {
2
    let (sum, overflowed) = x.overflowing_add(1);
3
    if overflowed {
4
        panic!("attempt to add with overflow");
5
    }
6
    sum
7
}

Change line 3 to add_one(i32::MAX) and that check fires:

rustc main.rs && ./main
thread 'main' (2718556) panicked at main.rs:2:23:
attempt to add with overflow
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Optimized builds drop the check. rustc enables overflow checks only when debug assertions are on, and those are on by default only when nothing is optimized. Cargo’s release profile turns overflow checks off too. The same program built with rustc -O prints -2147483648, the smallest i32. Built that way, at optimization level 3, the closure’s body is a single plain addition with no check. For the original add_one(5), rustc’s own optimizer goes further. It inlines the closure, pasting its body into main in place of the call, and then folds 5 + 1 into the constant 6, so no addition is left to run.

It knows what to paste in because the closure’s type names exactly one piece of code. That raises a question: a closure that captures nothing is code and no data, which is what a plain function is. Can it be used as one?

When a plain function will do

In 2016 it still couldn’t. Rust has a type for a pointer to a function, written fn(i32) -> i32, and only a function defined ahead of time could become one. RFC 1558, dated March 25, 2016, opens with the complaint: “Currently in Rust, it is impossible to bind anything but a pre-defined function as a function pointer.” Its example is an array of small handlers. Every element of an array must have the same type, and every closure has its own, so each handler had to be written out first as a named function. The RFC calls its fix “admittedly, a convenience-motivated feature”.

It also covers the workaround. An array of &dyn Fn values, references to closures whose concrete types are no longer known, works, but each call goes through “an extra layer of indirection”, a lookup of which code to run. And code in C that takes a callback needs a plain function pointer. Of the workaround, the RFC says “it is impossible to use this syntax when dealing with FFI”, Rust’s interface for calling code in other languages. A closure that captures nothing has no state to carry, so the RFC proposed letting it convert to a function pointer automatically, which Rust calls a coercion. Rust 1.19.0 shipped that on July 20, 2017. Its release notes say “Non capturing closures can now be coerced into fns”.

So this compiles, and rustc inserts the conversion at the let:

1
fn main() {
2
    let add_one = |x| x + 1;
3
    let f: fn(i32) -> i32 = add_one;   // closure converted to a function pointer here
4
    println!("{} {}", add_one(5), f(5));
5
}

Here the annotation on f picks i32 for x, so this version doesn’t need the fallback. add_one is still usable after the conversion because a closure that captures nothing can be copied, like an integer. The conversion copies it and leaves the original in place.

The two values aren’t the same thing. add_one takes 0 bytes: its type already says which code to run, so there is nothing to store. f takes 8 bytes, because it holds the address of that code, and a call through f jumps to whatever address is stored there.

That matters to the optimizer. To turn add_one(5) into 6, it only has to look at the type. To do the same for f(5), it first has to work out which address f holds. In a program this small it can, and with -O both calls end up as 6. When a pointer could hold any of several functions, it can’t. RFC 114 wanted closures to keep that choice with the programmer, and listed among its goals “Give control over inlining to users”.

The conversion only works while rustc still knows the closure’s own type. RFC 1558 notes that “once explicitly assigned to an Fn trait, the closure can no longer be coerced into fn, even if it has no captures”. So once a closure is stored as a &dyn Fn(i32) -> i32, where all rustc knows is “something callable”, it can’t become a fn(i32) -> i32 anymore.