Reuse scopes from primitive computed objects

[dphfox]

Right now with scopes, all computed objects construct and hold on to an innerScope which is cleaned up on recomputation. However, a large number of computed objects only ever do primitive computations, and do not need a scope to track any cleanup tasks. This means these objects allocate and hold onto an empty table for no reason.

Ideally, we would know ahead of time whether we need to allocate a scope for the computation to work. However, the only way to know whether a computation needs to allocate a scope, is to run the computation with a scope and see if the scope is empty afterwards. In other words, we would need time travel to predict the 'right' number of allocations to make.

However, through some clever reuse, it should be possible to, on average, only allocate one scope per non-primitive computation.

The idea is simple in principle; imagine a global variable which holds an empty table at all times. When a computation needs to be run, the metatable of that empty table is set to the metatable of the outer scope of the computed. Then, the computation is run with that empty table acting as the scope; the computation sees a fully-formed scope which it can add items to if needed. Finally, the computed checks to see if anything was added; if so, the global variable is refreshed with a newly-allocated empty table, and if not, the computed releases the empty table back to the global variable, and drops all of its own references to it. This means, aside from one initial allocation, the only time a new scope is allocated is when a computation needs to add stuff to a scope.

This could have vast memory savings, as most code bases deal with a larger number of primitive computed objects, compared to non-primitive objects, with more complex code bases trending towards an even larger imbalance.

We could reuse this trick in other parts of Fusion, though we'd need to be careful we don't end up holding more things in memory at the end. Remember that this isn't strictly free; we're just trading the cost of having one empty table always held in memory, with the cost of having one empty table allocated per computation.

[dphfox]

Potentially, this could be rolled in with the existing deriveScope() method, so we automatically enable this optimisation everywhere, including in user code.

We would need to be careful about how this works. When reusing a scope, we need to ensure it's immediately taken out of the global 'pool'. The only time a scope can be reused is when it is explicitly 'given back'.

It's unclear whether the 'giving back' API should be public or not. For simplicity, we'll start with this being internal-only. If we do expose a user API for this, we'll need to think about how to avoid doing more work ensuring they pass in a give-back-able table, compared to the work the memory allocator would do in the absence of this "optimisation"

[dphfox]

This should not be rolled into doCleanup. While Fusion internally does not reuse anything passed to doCleanup (it's treated like a black hole, so to speak), there is nothing that prevents user code from continuing to use a scope passed to doCleanup. Users might choose to do this if they're trying to implement their own pooling, for example, which means we can't depend on doCleanup implicitly releasing scopes for pooling.

Instead, it would be more ideal to implement this 'releasing into the pool' behaviour in a separate function that makes the semantics clear. I would suggest combining it with an 'emptiness check' so it doesn't duplicate that check when being used in common cases such as that of computed object inner scopes.

It might look something like this:

local function reuseScopeIfEmpty(
    scope: Scope<T>
): Scope<T>?
    if next(scope) == nil then
        sendToGlobalPoolOfScopes(scope)
        return nil
    else
        return scope
    end
end

deriveScope could look like this:

local function deriveScope<T>(
    existing: Types.Scope<T>
): Types.Scope<T>
    local derived = getScopeFromGlobalPool() or {}
    return setmetatable(derived, getmetatable(existing)) :: any
end

From inside Computed, it'd look like this:

local innerScope = deriveScope(outerScope)
local function use<T>(target: Types.CanBeState<T>): T
    -- snip
end
    local ok, newValue = xpcall(self._processor, parseError, use, innerScope)
    if ok then
        -- snip
        if self._innerScope ~= nil then
            doCleanup(self._innerScope)
            self._innerScope = nil
        end
        self._innerScope = reuseScopeIfEmpty(innerScope)

[dphfox]

Implemented as part of #273 but will need to test further to decide on an ideal pool size.