fitzgen / bumpalo

A fast bump allocation arena for Rust
https://docs.rs/bumpalo
Apache License 2.0
1.34k stars 109 forks source link

Modifies RawVec `reserve` fn structure to improve inlining #239

Closed zslayton closed 4 months ago

zslayton commented 4 months ago

This PR reorganizes RawVec::reserve_internal into a family of methods that allow the compiler to make better inlining decisions.

fallible_reserve_internal is a wrapper around reserve_internal that performs the "do we already have enough space?" check:

if self.cap().wrapping_sub(used_cap) >= needed_extra_cap {
    return Ok(());
}

This portion of the code (which produces a very small amount of assembly) is always inlined. Only if this check fails (which is relatively rare) does it then call reserve_internal_or_error, which is not inlined.

The infallible_reserve_internal/reserve_internal_or_panic method pair is similar to the above, but also prevents the code needed to detect an error and panic accordingly from being inlined.

Collectively, this allows the compiler to inline Vec::push and Vec::extend* in more places by transitively shrinking the amount of assembly each produces.

You can see the impact in the benchmarks introduced in #235:

image