Statically-Enforced Dynamic Memory Management (SEDMM)

Since Vinum's early beginnings, it was de-facto in-aware of memory management at scale. Since 0.2, it provided a destroy public function so that the users could be able to destroy their created objects.

However, this API was in fact so low-level, that you could say its just an reincarnation of your cute little free() function in languages like C++. This of course meant that it also inherited the same foot guns as free(), or at least, in some sort of way. This meant that you need to pair one object construction call with exactly one object destruction call, and Modern Science had found that developers suck at this.

A better solution for this is perhaps to introduce a scoping behavior, or perhaps tying an object's existence to a defined process. A process that would just act as a container for objects that need to be cleaned up when the process is needed to be killed.

Of course, there is more to this, and I will add more info to this issue as I get new insights and ideas.

Generally speaking, all values we create in Luau and Roblox are tied to a specific scope, or to be technically specific, process. For example, all instances you create in the DataModel are tied to Roblox's process/scope. That is, while you will certainly have memory leaks from not cleaning up instances at all, they still adhere to a specific scope.

As you might have guessed, the reason of these memory leaks is that the scope these said instances adhere to, is practically the root scope. So to fix this, we need to implement scoping primitives in our code. However, as an additional requirement, we need to also make it very explicit, and also dependent on Luau's type checking. This way, you will always be required to create an object that adheres to a scope defined and controlled by you.

The solution for this is to preferably expose a scope/process global function that returns a cleanup table with a metatable that points at a table of constructors, something like this:

local scope = process()

local Health = scope:Hold(100) -- processor omitted for clarity.

And Hold's constructor would look something like this:

local function Hold(cleanupTable, value, processor)
   local self = {...}
   table.insert(cleanupTable, self)

   return self
end

And to kill a process/scope, we could always expose a kill function that cleans the same type of data your good old Maids clean, in addition for support for calling a custom destroy function suited for cleaning up state objects. (to be clear, it will not be functionally same as the current public destroy function).

Having this solution implemented will greatly help with memory management in Vinum, as in contrast to previous versions, cleanup code will now only bother about cleaning up processes/scopes, not individual objects! This works, because for example,its easier for your brain to remember Kill this scope when *this* instance gets destroyed than it to remember Destroy all of **those** individual objects when this instance gets destroyed.

This being said, there are more rules about this scoping behavior that will affect how you use Vinum though I'd imagine not by a lot.

Firstly, Scopes are Long-Living Mortals

All scopes you create will all have strong references held by Vinum. While this can look like a bad idea at first glance, it's actually very useful. This is particularly because scopes/processes need to be cleaned up manually, either in a declarative manner in a library like Fusion {[onCleanup] = function() kill(yourScope) end}, or in an imperative one.

Another great side-benefit from this, is you got the ability to efficiently inspect what scopes are currently alive at the moment, which opens up for Debuggers and other alike tools.

Of course, these scopes will eventually get cleaned up, but that will probably be when all memory is getting free'd anyway.

Secondly, Scopes have Children

This rule's existence can be always tied to current state object destruction behavior. When you destroy a state object, all of its dependents will get cleaned up as well. This is generally because if you just cleaned up the root state object, all the other child state object will either just sit there causing memory leaks, or potentially errors if they have a still-alive dependency.

But to be incredibly honest, this rule is more of an idiom rather than an actual rule that can be enforced statically. Though, that's an implementation detail you will never need to worry about as a user of the library as preferably, Vinum will automatically mark a state object's scope a child of another when the said state object depends on a state object of the other scope. This means that you shouldn't get any error related to that at all.

Another related issue is the non-existence of a standard interface of destruction code. That is, Instances and custom luau objects adopt object::Destroy, while classes like Connections adopt connection::Disconnect, and other data structures have completely unique destruction semantics.

Of course, hard-coded support for all of these common classes is manageable, but only so until you got to support 5+ types. While support for function calling might open up for custom support, it doesn't enforce clear user intent and will ultimately lead for more code complication so that it can work nicely.

Destruction Traits

The solution I found for this is to introduce some sort of a destruction trait, where in implementation-specifics, just a table with an object field that points at the value, and a destroy field that points at a defined destructor function. Technically speaking, you wouldn't be creating these tables yourself, as two public functions would be exposed just for that.

Those two are called Owned and Borrowed. As the name implies, they are functions that return a destruction trait that either specify a value need to be destructed (the value is owned by the current user agent), or need to be left alone (the value is borrowed by the current user agent).

For example:

local Owned = Vinum.makeOwned(function(value)
    print(value)
end)
local Borrowed = Vinum.Borrowed

local function your_processor(old, new)
   return old ~= new
end

Compute(function(...)
   return Borrowed(workspace.Baseplate)
end, your_processor)

Compute(function(...)
   return Owned(Instance.new("Part"))
end, your_processor)

Few observations can be found from this:

An Owned can be constructed from a function that specifies its destructor function.
Both Owned and Borrowed accept a value as an argument..
Processors now aren't bothered with cleanup semantics.
After every udate, Compute calls the destroy field of the previous destruction trait.
Owned implies that a value must be destructed after you are finished with it.
Borrowed implies that a value must be not destructed after you are finished with it.

Using those observations, we could derive some rules and idioms:

Owned can't accept Destruction Traits

This should makes sense really, as whether the incoming trait is created by a Borrowed or an Owned, it does still means that the wrapped value already has an owner. As such, Owned should error when met with a case like this.

It could be said that when you give Owned a trait, it is implied you are giving ownership to the said Owned over the said trait; however, Vinum remains authoritative of all traits, due to the possibility for trait recycling. This should be the expected, as Destruction Traits are nothing but traits at the end of the day. Not an entire self-aware object in a way, instead, it's just a trait that defines some destruction behavior.

Also, even if we decided against trait recycling, it still makes sense to not Owned any trait, as they are garbage collectable, and not expected to have a reference held to them (ex. Processors receive the unwrapped value rather than their traits).

This also means that ideally, you shouldn't be working with Traits if you aren't the agent that will clean up the mess, instead, you should always unwrap them simply by indexing their object field.

Borrowed unwrap incoming Destruction Traits

This is also similar to the previous rule in some way, as while Borrowed is allowed to unwrap any trait, and absorb their value, it's not allowed to borrow another trait as it could introduce nesting and a false perception that traits can be borrowed (ex. imagine a Compute's processor getting a trait instead of its value because of nesting..).

All manually-managed values must be wrapped in a Destruction Trait

Technically speaking, not all values need destruction. Simple singular values like strings and numbers don't need destruction at all. So in a way, we need to make it obvious to Luau's typechecker when to warn when there isn't a trait getting passed to an agent.

vinum-team / Vinum