[FEA] T-SNE support to embed out-of-sample points

[cjnolet]

UMAP's ability to embed out-of-sample data points has allowed us to create a fairly simple Dask wrapper for distributed inference, by embedding a subset of the training data on a single GPU and then broadcasting the trained model to Dask workers to perform inference in parallel. We have found that as long as the training dataset is a representative sample of the neighborhoods in the larger population, the distributed embedding is showing even higher overall trustworthiness on some datasets.

If we were able to support out-of-sample embeddings in T-SNE, we should be able to support distributed inference in a similar manner to UMAP. However, this feature has been shown in past research to not be quite as straightforward to implement. It would still be worth investigating the feasibility of this feature for cuML's T-SNE- while distributing the training is going to require distributing the entire optimization stage. Though possible, it's going to have a high communication overhead.

Van Der Maaten created Parametric TSNE, which should be able to accomplish this, but requires the use of deep neural networks: https://lvdmaaten.github.io/publications/papers/AISTATS_2009.pdf. Also relevant is this issue on Van Der Maaten's Barne's Hut Github.

Additional literature:

• http://www.iro.umontreal.ca/~lisa/pointeurs/tr1238.pdf
• https://arxiv.org/pdf/1902.07958.pdf
• https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2012-25.pdf
• https://arxiv.org/pdf/1710.05128.pdf
• https://www.semanticscholar.org/paper/Parametric-nonlinear-dimensionality-reduction-using-Gisbrecht-Schulz/b1a1fbf2ffb922fbfc6c4d0565925497a82d6b85

[zbjornson]

I think #861 can be merged into this issue.

[github-actions[bot]]

This issue has been marked rotten due to no recent activity in the past 90d. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.

[github-actions[bot]]

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.