search

rapidsai / cuml

cuML - RAPIDS Machine Learning Library
https://docs.rapids.ai/api/cuml/stable/
Apache License 2.0
4.2k stars 525 forks source link

[FEA] Chunked pairwise distance #2644

Open tfeher opened 4 years ago

tfeher commented 4 years ago

Is your feature request related to a problem? Please describe. cuML provides pairwise distance metrics https://github.com/rapidsai/cuml/pull/2502

For large datasets GPU memory can becomes a limitation, and chunked pairwise distances would be useful.

Describe the solution you'd like Provide chunked pairwise distance metrics in cuML, like https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise_distances_chunked.html

Additional context Tagging @ChristofHenkel to provide more context.

teju85 commented 4 years ago

JFYI, dbscan has a very crude implementation of this chunked (aka batched) approach here. We'll probably have to generalize this inside our C++ distance metrics and then expose it in the python layer.

Tagging @cjnolet and @JohnZed

github-actions[bot] commented 3 years ago

This issue has been marked rotten due to no recent activity in the past 90d. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.

github-actions[bot] commented 3 years ago

This issue has been marked stale due to no recent activity in the past 30d. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be marked rotten if there is no activity in the next 60d.

github-actions[bot] commented 3 years ago

This issue has been labeled inactive-90d due to no recent activity in the past 90 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.

valdo404 commented 1 year ago

Hi mates, any news from this issue ? I'm trying to score a KDE estimate on a big grid, and it seems not to work because of the lack of memory on the GPU.