DmitryUlyanov / Multicore-TSNE

Parallel t-SNE implementation with Python and Torch wrappers.
Other
1.89k stars 228 forks source link

Using single core even when n_jobs=4 is used #53

Open MayukhSobo opened 6 years ago

MayukhSobo commented 6 years ago

I am seeing

Performing t-SNE using 1 cores.
Using no_dims = 2, perplexity = 30.000000, and theta = 0.500000
Computing input similarities...
Building tree...

this verbose message even when my n_jobs=4

jeffThompson commented 5 years ago

I see the same issue on Mac, installed with pip.

Attempting to run on 8 cores, but says it's only running on 1.

jeffThompson commented 5 years ago

Not a great solution, but in the short term this fork has Mac-specific install instructions that worked for me: https://github.com/sg-s/Multicore-TSNE

sg-s commented 5 years ago

ouch -- i just nuked that repo, thinking this would be OK and now i find this bug. fml

update: repo restored

jwittenbach commented 5 years ago

I had this problem on my Mac as well. After reading the notes form @sg-s (huge help!), I was able to come up with a relatively simple fix.

  1. install a more updated version of the C/C++ compilers using homebrew: brew install gcc
  2. find the executables -- for me it installed gcc-8 and g++-8 to /usr/local/bin/ (found via which gcc-<version>)
  3. set them as your default C and C++ compilers: export CC=/path/to/gcc-8; export CXX=/path/to/g++-8
  4. clone the Multicore-TSNE repo and pip install . from the top level

It's possible that you only need to set he correct C++ compiler (and not the C compiler), but I didn't test trying to make it simpler.

ohld commented 5 years ago

Just faced the same problem and the @jwittenbach solution helped! Thanks!

chm90 commented 4 years ago

The solution by @jwittenbach also worked for me. The below formulation the command of point 3. might be easier to use export CC=$(which gcc-8); export CXX=$(which g++-8).

vishalmhjn commented 4 years ago

@jwittenbach these settings fail to create the .so file in the installation module.