MMTk integration desired

[stephenrkell]

For a long time I've wanted high-performance garbage-collected heaps to be available and integrated with liballocs -- especially as a foundation for multi-language interop. It is looking like MMTk could provide a great way to achieve this.

There is an open call for liballocs-related projects within MMTk here. This issue is the converse call to the liballocs audience (such as it is).

One initial set of goals is roughly this:

• Any MMTk-based allocator/GC/heap implementation should be queryable via liballocs
• If there are ways in which MMTk and liballocs overlap, they should do so harmoniously (deliberately vague term)

We can add more goals later, especially about what should perform well.

I'll add notes below about what I manage to get done, or just to understand, during early investigations of MMTk and liballocs.

[stephenrkell]

One possible overlap is the approach to classifying pointers at the top level. In MMTk there is a 4MB notion of "chunks" that is used to keep track of memory-mapping-granularity divisions of memory. In liballocs we use the pageindex, which has finer granularity (one page, as we'd expect) but a similar purpose... and behind the pageindex are big_allocations of course. Maybe we'll find that we can use one in place of the other (probably the pageindex in place of an chunk-associative structure, but that's not 100% clear yet).

[stephenrkell]

A more extreme idea is this: can we implement the MMTk memory manager interface using liballocs? This would mean that any memory known to liballocs would be queryable or manipulable via MMTk's interface, i.e. backing onto liballocs's base- and meta-level APIs. Is this useful?

Like liballocs, MMTk encapsulates memory management with a mixture of base-level actions on memory (free) / objects (allocated), and somewhat-meta-level queries on objects (closed) / addresses or address ranges (open).

[stephenrkell]

Flipping it around, we can also ask: given a Plan (in MMTk-speak), or possibly a Builder or MMTK (not sure), can we generate a struct allocator (in liballocs-speak)?

[stephenrkell]

Now I understand a tiny bit more about MMTk, here are some thoughts.

• In a given instance of MMTk, the heap bounds are fixed at start-up. (Can this be relaxed? Can liballocs help? "We'd like to be able to tolerate mmap failure.")
• It's not currently supported for MMTk to have multiple instances in one process. (Can this be relaxed? Can liballocs help?)
• Therefore initially a single mmtk_allocator is not a silly idea. One per plan or per policy would be more sensible longer-term.
• I think liballocs-aware configurations should always enable VO [valid object] bits.
• How can size information be recovered from language implementation? I think this must already be catered for in VMBinding somehow.
• Can we extend VMBinding so that type information can be passed out of the language implementation? Doesn't MMTk need something like this already for pointer maps?
• If type descriptors are themselves GC'd, it makes it challenging for them to participate in the usual uniqtype protocols. But there will be a crude way forward e.g. just by pinning them. In any case we probably want to proxy the per-VM descriptors by uniqtypes that point to them, say.