issues
search
vu-minh
/
tsne_auto_perp_constraints
Auto select perplexity for tsne with user constraints
0
stars
0
forks
source link
Setup script to run t-SNE on server
#6
Closed
vu-minh
closed
6 years ago
vu-minh
commented
6 years ago
code should be run in parallel (using python multiprocessing with
joblib
)
how to store the embedding result (pickle with
joblib.dump
)
vu-minh
commented
6 years ago
Find a way to use C++ or multi-core implementation.
Current issue: when run in parallel, each process calculate pairwise distances matrix, that lead to
OutOfMemory
error.
We can try:
[ ] Reduce number of parallel jobs
[ ] Using C++ implementation and still run job in parallel
[ ] Using multi-cores implementation and run in sequential
joblib
)joblib.dump
)