DmitryUlyanov / Multicore-TSNE

Parallel t-SNE implementation with Python and Torch wrappers.
Other
1.89k stars 228 forks source link

Memory Allocation Fail (Big Data) #70

Open cipheLab opened 5 years ago

cipheLab commented 5 years ago

Hello,

I use Multicore-opt-TSNE on a big data, 24 000 000 events and 18 parameters on a ubuntu server with 40 core, and 500G Ram with this command line :

python2 MulticoreTSNE/run/run_optsne.py --optsne --data Data.csv --outfile Data_tsne.csv --n_threads 40 --perp 50

and i have this error : Memory allocation failed!

Can you tell me if i change something in my commande line (in my parameters) i have more luck to run my job or if you know the setup necessary to run a Multicore-Tsne on thi big data ?

Best regards.

Quentin Barbier