Hyperparameters' definition domain

[Laurae2]

Hello,

I've looked thoroughly the paper discussing about largeVis since it is available in arxiv (since more than 5 months now), and I am still wondering about the following:

• How many total hyperparameters are there from scratch? From what I am seeing, there are 10 of them (k, n_trees, tree_threshold, max_iter, distance_method, perplexity, M, gamma, alpha, rho), assuming the output dimension is fixed by the user - there are 6 if excluding the Stochastic Gradient Descent part
• What are the definition domains of each hyperparameter? The question is not existant for M/gamma/alpha/rho (they are obvious), but what about k / n_trees / tree_threshold / max_iter / distance_method?

I am assuming currently that, from a matrix MAT before transposition:

• k = [1, nrow(MAT)]
• n_trees = [1, Inf]
• tree_threshold = [1, nrow(MAT)], where the suggested is ncol(MAT)
• max_iter = [1, Inf]
• distance_method = Euclidean or Cosine
• perplexity = [1, nrow(MAT)] unlike t-SNE where it is [1, floor(nrow(MAT)/3)]

And if using Windows, k*nrow(MAT) < ~4 billion (2^32) else error during projectKNNs (larger than arma sparse matrix max capacity) or even before.

Are my assumptions correct or did I miss something?

[elbamos]

Yes, that's correct.

But -- and this is discussed in the benchmarks vignette -- n_trees, tree_threshold, and max_iter are really three different ways of attacking the same problem, of finding the best approximation of nearest neighbors in the least time with the least RAM. So when you adjust those hyperparameters, usually what you are doing is trading one off against others.

One of the things I like about largeVis is that its a pragmatic algorithm. Being pragmatic, max_iters can be safely set at < 3 (usually, I leave it at 1). The choice of n_trees and tree_threshold is a choice of how long you want to wait before you get your result. And the balance between n_trees and tree_threshold is based on how much RAM you have.

Does this help?

[elbamos]

@Laurae2 If I've answered your question, I'd appreciate if you would close this issue. Thanks!

[Laurae2]

Yep you answered my question!

Closing this issue.