Open jrhemstad opened 6 months ago
@harrism You might be interested in this
Is there a long-term plan to pull more of the concrete implementations from rmm into CCCL? That seems like the best way to broaden adoption and usage of these allocators and would satisfy some of the new features mentioned above IIRC.
@vyasr yes I believe we want to pull some of the foundational features into cccl. Definitely not all but some
Is there a long-term plan to pull more of the concrete implementations from rmm into CCCL? That seems like the best way to broaden adoption and usage of these allocators and would satisfy some of the new features mentioned above IIRC.
Yes, that is what we mean by "Concrete types that satisfy the resource
and async_resource
concepts"
our RFE:
deallocate/deallocate_async
functions should accept const void*
to skip const_cast<T*>()
on the user sidecuda::mr::*
functions in device codeClarify (or fix) the expected behavior of allocate()
deallocate()
for async_resource
_async()
version of the API and add the stream to allocate/deallocate
Clarify (or fix) the expected behavior of allocate() deallocate() for async_resource
Can you elaborate on what you mean? allocate()
and deallocate()
are expected to always be synchronous.
Can you elaborate on what you mean? allocate() and deallocate() are expected to always be synchronous.
Yes, but what is their purpose if the code uses an async_resource
with _async()
API. They look redundant and confusing in this case
Yes, but what is their purpose if the code uses an async_resource with _async() API. They look redundant and confusing in this case
The thinking is that the async_resource
concept is a strict superset of the resource
concept. This way, if you have an async_resource
object, you can still conveniently pass it to a function that expects a resource
.
ok, I didn't interpret async_resource
as a superset of the resource concept
. In this case, can we please just clarify this point on the doc?
cuda::mr
is intended to be the future of heterogenous memory allocation in CUDA C++. It is inspired heavily by lessons learned in RMM and our experience with thedevice_memory_resource*
and friends.cuda::mr
does not seek to replace RMM, but instead distill and standardize the best parts of RMM into a more central location. Furthermore, RMM is already in the process of rebasing on top of using thecuda::mr
interface.What we have today is the
cuda/memory_resource
header that provides[async_]resource
concepts[async_]resource_ref
polymorphic typeIn essence, this just provides the top-level interface for memory allocation and defining properties of the allocated memory.
Questions we'll need to answer along the way: