search

CannyLab / tsne-cuda

GPU Accelerated t-SNE for CUDA with Python bindings
BSD 3-Clause "New" or "Revised" License
1.81k stars 130 forks source link

fail with 'conda install -c CannyLab -c pytorch tsnecuda' #103

Closed GlennCGL closed 2 years ago

GlennCGL commented 3 years ago

cuda version is 10.1, and when I install with -c CannyLab, faiss and fairss-gpu installed will be crashed. Only when I reinstall faiss-gpu with 'conda install faiss-gpu -c pytorch', problem solved. However, tsnecuda will be unable to use. When on machine with cuda version 9.0, 'conda install tsnecuda cuda90 -c pytorch' is OK without crashing problem.

lfvillavicencio commented 3 years ago

also, a lot of problems with WSL2 installation. should try a guide

DavidMChan commented 3 years ago

We don't officially support WSL - if you can get WSL working, then we'd love an updated contribution to the readme. Conda's support for GPU-based libraries is pretty rough, and we don't have the development/packaging experience that Facebook and Google have for multiple platforms, so by far the best way to consistently ensure that the code works on your machine is to build from source every time.

Itr seems like there's a version conflict with the versions of faiss-gpu/faiss and those with tsnecuda. The exact error logs would be really helpful when debugging this.

lfvillavicencio commented 3 years ago

as far as I seem, the conda installation of faiss-gpu gives you the version 1.7.5. But building your great package need the version 1.6.5

h-vetinari commented 3 years ago

Conda's support for GPU-based libraries is pretty rough, [...]

Hey there; conda support for GPU-based packages has gotten much better in recent times, actually nvidia itself is distributing lots of its binaries through conda-forge. The one wrinkle is that there are no GPUs in the CI provided by azure, so while the packages can be tested in general, the code paths that depend on loading some GPU drivers etc. won't work (though it's possible to test locally by downloading an artefact from a PR, for example).

[...] and we don't have the development/packaging experience that Facebook and Google have for multiple platforms,

Facebook has done quite a lot on they pytorch side, but I've done basically all of the work for packaging faiss in conda-forge (including a lot of original debugging to get CUDA on windows to run). Google has also not done much beyond providing huge wheels for tensorflow, but the situation in conda-forge has improved recently as well.

As I mentioned on another issue already, I have a draft to add tsnecuda to conda-forge, then all those version conflicts should be a thing of the past (though it would be good to eventually become compatible with faiss 1.7)

DavidMChan commented 3 years ago

@h-vetinari - I really appreciate this! Since I'm just one AI PhD student (and this was originally just a class project), I really appreciate the help with packaging. I wasn't really directly referring to facebook/google individually, just pointing out that I'm one person with not so much experience in multi-platform release trying to make this tool available.

Towards FAISS 1.7 compatibility, see #98

DavidMChan commented 2 years ago

Closed as stale.