Open gclen opened 4 years ago
Sadly you need the low_memory=True
option. All this will be fixed properly in 0.5 when pynndescent becomes an explicit dependency. In the meantime the FAQ should have some notes on this: https://umap-learn.readthedocs.io/en/latest/faq.html#i-ran-out-of-memory-help but I would welcome suggestions for better more obvious places to document this. It is largely an issue with sparse data and angular metrics.
Using UMAP on a small dataset (20 newsgroups), ran my machine of memory (56GB of RAM). However, when I installed pynndescent, this issue went away. I had installed UMAP via
pip install umap-learn --pre
and the code to reproduce it is