search

rapidsai / dask-cuda

Utilities for Dask and CUDA interactions
https://docs.rapids.ai/api/dask-cuda/stable/
Apache License 2.0
281 stars 89 forks source link

When to use `rmm_async` #602

Open charlesbluca opened 3 years ago

charlesbluca commented 3 years ago

With #566, we added the ability to use RMM's asynchronous allocator for cluster setup, which is mutually exclusive with RMM pools and managed memory. I am wondering if there are any specific benefits for using this, or use cases where it would be important? These could potentially be added somewhere in docs if they are prominent enough.

cc @pentschev

pentschev commented 3 years ago

cc @beckernick who originally asked about this feature.

beckernick commented 3 years ago

The async memory resource will not currently perform as well as the default pool resource. However, this feature has one core benefit relative to a standard RMM pool: it makes it easier to share GPU memory with libraries that don't expose a pluggable external memory interface (while still getting performance gains relative to standard cudaMalloc).

The standard pool resource will exhaust the available memory as needed and never free it. The async resource will free it.

pentschev commented 3 years ago

Seems like the use case where this is helpful is when using external libraries at the moment. Do you happen to have any such examples @beckernick ? Perhaps we could document that's the intended use case and, if possible, add an example of the intended usage.