The first two functions are used only for big allocations and therefore don't need to be instrumented with KFENCE.
The last function (there also exists its node-agnostic version, slab_alloc(), that calls slab_alloc_node()) actually performs all allocations we are interested in.
For kmalloc() allocations the exact allocation size is never passed to slab_alloc_node(), which makes it impossible to precisely put kmalloc()-allocated objects at the end of the page.
To handle this properly, we can:
either pass the size requested by kmalloc() to slab_alloc_node() (we can use the upper bits in _RET_IP_ to store it)
or insert KFENCE hooks into allocation functions in a way that:
there is at least one hook on every path from an API function to slab_alloc_node() at a place where size is available;
if there is more than one such hook, only one should be executed;
The first option looks a bit hackier, but will work for both naive and stealing implementations. The second one will require additional trickery to support kmalloc() in the stealing implementation.
For the second option, there is no basis API subset that can be instrumented with exactly one hook, e.g. because kmalloc() may call either __kmalloc() or slab_alloc_node() (which means we need a hook in kmalloc()), whereas kmalloc_array() may call either kmalloc() or __kmalloc() (which means we need a hook in __kmalloc() as well).
We can use a special GFP flag to tell the underlying functions we've run KFENCE for this allocation already.
We need to ensure naive implementation calls our runtime exactly once.
Every allocation API function from
include/linux/slab.h
ends up calling one of the following three functions to allocate memory:The first two functions are used only for big allocations and therefore don't need to be instrumented with KFENCE. The last function (there also exists its node-agnostic version,
slab_alloc()
, that callsslab_alloc_node()
) actually performs all allocations we are interested in.For
kmalloc()
allocations the exact allocation size is never passed toslab_alloc_node()
, which makes it impossible to precisely putkmalloc()
-allocated objects at the end of the page. To handle this properly, we can:kmalloc()
toslab_alloc_node()
(we can use the upper bits in_RET_IP_
to store it)slab_alloc_node()
at a place where size is available;The first option looks a bit hackier, but will work for both naive and stealing implementations. The second one will require additional trickery to support
kmalloc()
in the stealing implementation.For the second option, there is no basis API subset that can be instrumented with exactly one hook, e.g. because
kmalloc()
may call either__kmalloc()
orslab_alloc_node()
(which means we need a hook inkmalloc()
), whereaskmalloc_array()
may call eitherkmalloc()
or__kmalloc()
(which means we need a hook in__kmalloc()
as well).We can use a special GFP flag to tell the underlying functions we've run KFENCE for this allocation already.