Generic Fail-able, Pool and Tag Aware Allocators

[jxy-s]

Development in the Windows Kernel requires allocations from different pools and the capability to tag allocations is an invaluable tool for debugging/triage.

Today this repository looks to recommend a global allocator and one that allocates exclusively from non-paged pool with a hard-coded pool tag.

https://github.com/microsoft/windows-drivers-rs/blob/cd1fd23f000c64ea244b5cbfc33d64acb1104b8b/crates/wdk-alloc/src/lib.rs#L35-L45

Rust crates for the Windows Kernel probably shouldn't provide a global kernel allocator. Else everyone using these crates may subtly end up allocating memory using identical tags. This will make debugging and triage a nightmare in the long term. Additionally I'll note, the non-paged pools are far more limited resource that the paged pools. I recognize that the safer option when implementing a global allocator is to force it into non-paged memory, since not doing so has the potential to introduce other issues. However resource exhaustion is far more likely when forcing all allocations into non-paged pools.

For Rust to be a first-class citizen in the Windows Kernel. The language must support generic fail-able allocators. And the crates for the Windows Kernel should expose and support appropriate allocators that are capable of specifying pool types and tags.

[wmmc88]

Users of these crates are free to consume them without the alloc feature, and define their own allocators with their own tags and pool.

Rust requires a global allocator to be defined in order to use the types in the alloc module in the standard library(ex. alloc::vec). But this doesn't force all allocations to use that allocator, the user is still free to allocate themselves via the rust binding to ExAllocatePool2.

Ideally, we would like to support alloc types with different allocator "settings"(ie. pool and tag) each time, but this is currently not something that is supported in stable rust. In unstable rust, there are currently two approaches to provide alloc types with custom allocators: the allocator-api proposal (which some libraries implement on stable via allocator_api2), and a Storage-backed allocator proposal.

[jxy-s]

Rust requires a global allocator to be defined in order to use the types in the alloc module in the standard library(ex. alloc::vec). But this doesn't force all allocations to use that allocator, the user is still free to allocate themselves via the rust binding to ExAllocatePool2.

I think this ties back to https://github.com/microsoft/windows-drivers-rs/issues/6 and https://github.com/microsoft/windows-drivers-rs/issues/6#issuecomment-1732809169. alloc::vec will abort the "program" if it can't allocate. I think that's an unacceptable outcome for the kernel. If the kernel can't allocate memory for some request it shouldn't bugcheck.

Users of these crates are free to consume them without the alloc feature, and define their own allocators with their own tags and pool.

As was said elegantly in the comment on the other issue, "I think it would be beneficial to consider options with how to make this a bit safer for developers to avoid some mishaps that are likely going to be very easy to make (e.g. accidentally using the wrong memory allocator)."