Eigenvalues

[toth12]

Hello,

I have a state space with 23 states and when training a Markov chain by using Pyemma's MSM class (http://emma-project.org/latest/api/generated/pyemma.msm.MSM.html?highlight=eigen#pyemma.msm.MSM.eigenvalues), I want to keep only ten eigenvectors with the highest eigenvalues. My understanding is that after the following steps, I should have only 10 states in the model. But that is not the case.

transition_matrix.shape (23,) mm = msm.MSM(transition_matrix,neig=10) mm.pi.shape (23,)

My questions:

1. In my Pyemma do you use eigenvalues for dimensionality reduction, i.e to keep only those states that have high eigenvalues?
2. If so, how to implement it?

[clonker]

Hi, the neig parameter is mostly used for performance reasons when working with very large sparse matrices - not so much for dimensionality (in terms of number of states) reduction. For the reducing the number of states in your state space I would recommend a spectral clustering method like PCCA+. The eigenvalues themselves are not directly related to a single state but rather to kinetic processes described by the model. They, e.g., can give information on implied relaxation times (in pyemma through the timescales property). What exactly are you trying to achieve by setting eigenvalues to zero?

[toth12]

@clonker many thanks for the answer; I misunderstood the neig parameter but now I understand what it is used for. Thanks for pointing to PCCA+, I will look at that. I was trying to set the neig parameter to 10 (not to zero) because I wrongly believed that I can do dimensionality reduction in this way. Thanks for the answer, please close this issue!

[clonker]

Sure, you're welcome