Using a request/query based allocator

[JelteF]

The main reason why I'm interested in this is for building Postgres extensions. Postgres has a request based allocator and I was wondering how that could/would fit with the Alloc API.

To expand a bit more on the behaviour of the allocator:

• palloc is used to allocate memory (with same arguments as malloc)
• memory is automatically freed at some point depending on the memory context
• you can change the memory context by calling MemoryContextSwitchTo with a specific memory context handle
• there's memory contexts for the current row, query, transaction or connection. These memory contexts are freed after you are done with a row, query, transaction or connection.
• using the allocated memory after its context has been cleared results in undefined behaviour (because of use after free)

So my main question is: How could you use the allocator API to work with this? Would you need an Allocator for each memorycontext or something?

[SimonSapin]

You’d implement the Alloc / AllocRef trait for a type (possibly literally a &'a Foo reference) that has a lifetime parameter that borrows some value that represents a memory context. Make AllocRef::dealloc a no-op, and document that it’s OK not to call it for this particular allocator (although e.g. Vec<T, &'a MyAllocator> would still call it). When the memory context goes out of scope, free memory in bulk. The borrow checker ensures that by that point nothing is borrowing it anymore, so undefined behavior is prevented.

That said, if you want to write code that directly allocates memory with a palloc-like API and not use the standard library’s collection types, I’d recommend making your own allocator API (that might not even have a dealloc method), or use an existing library (there are a few, the keyword is “arena”) without waiting on this WG to stabilize something in the standard library.

[JelteF]

That's a good idea to use the borrow checker for the memory context. Because indeed the Vec would have to store a reference to the allocator to to be able to deallocate.

So to clear two things up:

• I would want to use standard library types like Vec.
• you can actually free memory in the postgres allocator manually as well with pfree