maxentile
commented
9 years ago Joel Tropp's work provides very good background for this:
- Chapter 6 of Intro to Matrix Concentration inequalities: http://arxiv.org/pdf/1501.01571v1.pdf
- Randomized matrix factorization: http://epubs.siam.org/doi/abs/10.1137/090771806
Also Alex Smola is teaching a course on machine learning this term, and one of the lectures covers kernel approximation: https://www.youtube.com/watch?v=KutLsqMyqUY
Also I hadn't realized how similar the objective functions for tICA and CCA are-- I think that time-structured ICA reduces to Canonical Correlation Analysis, i.e. CCA(X,Y) where Y is a t-lagged view of X is very similar to tICA(X, lagtime=t). A couple experiments yielded identical solutions up to rotation. If the solutions are actually equivalent, then we could immediately write a linear-time approximation for the currently cubic-time kernel tICA algorithm, which would be nice.
Looks pretty interesting and simple to implement, and in section 5.4 they also use it for fast autoencoder training
Paper: http://arxiv.org/pdf/1402.0119v2.pdf Talk: http://techtalks.tv/talks/randomized-nonlinear-component-analysis/60874/ Code: https://github.com/lopezpaz/randomized_nonlinear_component_analysis