CalculateImpliedTimeScale failed with "k must be between 1 and ndim(A)-1"

[Serilin]

Hi, everyone I'm working with my system using msmbuilder2.8.2. The cluster process has succeed with the commond " msmb Cluster rmsd hybrid -d 0.045 -l 50" . But the command "CalculateImpliedTimeScales -l 1,25 -i 1 -o Data/ImpliedTimeScales.dat" failed, telling the following {'assignments': 'Data/Assignments.h5', 'eigvals': 10, 'interval': 1, 'lagtime': '1,25', 'notrim': False, 'output': 'Data/ImpliedTimescales.dat', 'procs': 1, 'quiet': False, 'symmetrize': 'MLE'} 09:04:17 - Getting 10 eigenvalues (timescales) for each lagtime... 09:04:17 - Building MSMs at the following lag times: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25] 09:04:17 - Calculating implied timescales at lagtime 1 09:04:17 - Selected component 58 with population 0.080616 09:04:18 - BFGS likelihood maximization terminated after 17 function calls. Initial and final log likelihoods: -79.490250, -78.928102. 09:04:18 - You cannot calculate 11 eigenvectors from a 5 x 5 matrix. 09:04:18 - Instead, calculating 5 eigenvectors. 09:04:18 - Calculating implied timescales at lagtime 8 09:04:18 - Selected component 72 with population 0.056490 Traceback (most recent call last): File "/home/tim/anaconda/bin/msmb", line 9, in load_entry_point('msmbuilder==2.8.2', 'console_scripts', 'msmb')() File "/home/tim/anaconda/lib/python2.7/site-packages/msmbuilder-2.8.2-py2.7.egg/msmbuilder/scripts/msmb.py", line 51, in entry_point getattr(scripts, args.subparser_name).entry_point() File "/home/tim/anaconda/lib/python2.7/site-packages/msmbuilder-2.8.2-py2.7.egg/msmbuilder/scripts/CalculateImpliedTimescales.py", line 85, in entry_point (not args.notrim), args.symmetrize, args.procs) File "/home/tim/anaconda/lib/python2.7/site-packages/msmbuilder-2.8.2-py2.7.egg/msmbuilder/scripts/CalculateImpliedTimescales.py", line 67, in run trimming=trimming, symmetrize=symmetrize, n_procs=nProc) File "/home/tim/anaconda/lib/python2.7/site-packages/msmbuilder-2.8.2-py2.7.egg/msmbuilder/msm_analysis.py", line 237, in get_implied_timescales lags = result.get(999999) File "/home/tim/anaconda/lib/python2.7/multiprocessing/pool.py", line 554, in get raise self._value ValueError: k must be between 1 and ndim(A)-1 09:04:18 - BFGS likelihood maximization terminated after 13 function calls. Initial and final log likelihoods: -82.038017, -75.611886. 09:04:18 - You cannot calculate 11 eigenvectors from a 3 x 3 matrix. 09:04:18 - Instead, calculating 3 eigenvectors.

I has tried the cluster with ’-d 0.2” and ‘-d 0.1“, and both failed. I don't know why.

I noted that when I ran the alanine dipeptide tutorial of msmbuilder2.8, "msmb CalculateImpliedTimescales -l 1,25 -i 1 -o Data/ImpliedTimescales.dat" gave out the 07:17:18 - Calculating implied timescales at lagtime 24 07:17:18 - Selected component 0 with population 1.000000 07:17:19 - BFGS likelihood maximization terminated after 157 function calls. Initial and final log likelihoods: -163578.487091, -163577.602213. 07:17:19 - Calculating implied timescales at lagtime 25 07:17:19 - Selected component 0 with population 1.000000 07:17:20 - BFGS likelihood maximization terminated after 185 function calls. Initial and final log likelihoods: -163322.252035, -163321.354107. 07:17:20 - Abnormal termination of BFGS likelihood maximization. Error code 2 07:17:20 - Saved output to Data/ImpliedTimescales.dat This tutorial is successful in my computer, so why my system was wrong ? Anyone can give some suggestion?

ps My trajectory has not solvent and was saved every 5 ps for total 200 ns. How should I define the number of min and max lagtime? Is it alway from 1 to 25?

[mpharrigan]

The distance cutoff in the clustering step is likely too strict. This means it is probably making way too many clusters and a really disconnected transition matrix.

09:04:18 - Selected component 72 with population 0.056490
Traceback (most recent call last):

This line means that 95% of your data is being thrown away. You're then left with too few states, and the sparse eigensolver can't get the eigenvalues.

Instead of using the -d option, try specifying the number of clusters directly with the -k flag.

[Serilin]

Mpharrigan, Thanks for your answer. According to your suggetion, we have tested to produce 200, 100 and 50 clusters with -k flag, respectively, and the cluster process has no problem. However, the same question stil appears when coming to "CalculateImpliedTimeScales -l 1,25 -i 1 -o Data/ImpliedTimeScales.dat".

{'assignments': 'Data/Assignments.h5', 'eigvals': 10, 'interval': 1, 'lagtime': '1,25', 'notrim': False, 'output': 'Data/ImpliedTimescales.dat', 'procs': 1, 'quiet': False, 'symmetrize': 'MLE'} 01:24:02 - Getting 10 eigenvalues (timescales) for each lagtime... 01:24:02 - Building MSMs at the following lag times: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25] 01:24:02 - Calculating implied timescales at lagtime 1 01:24:02 - Selected component 0 with population 0.700740 01:24:02 - BFGS likelihood maximization terminated after 24 function calls. Initial and final log likelihoods: -347.627745, -347.247757. 01:24:02 - You cannot calculate 11 eigenvectors from a 6 x 6 matrix. 01:24:02 - Instead, calculating 6 eigenvectors. 01:24:02 - Calculating implied timescales at lagtime 8 01:24:02 - Selected component 0 with population 0.701723 Traceback (most recent call last): File "/home/tim/anaconda/bin/msmb", line 9, in load_entry_point('msmbuilder==2.8.2', 'console_scripts', 'msmb')() File "/home/tim/anaconda/lib/python2.7/site-packages/msmbuilder-2.8.2-py2.7.egg/msmbuilder/scripts/msmb.py", line 51, in entry_point getattr(scripts, args.subparser_name).entry_point() File "/home/tim/anaconda/lib/python2.7/site-packages/msmbuilder-2.8.2-py2.7.egg/msmbuilder/scripts/CalculateImpliedTimescales.py", line 85, in entry_point (not args.notrim), args.symmetrize, args.procs) File "/home/tim/anaconda/lib/python2.7/site-packages/msmbuilder-2.8.2-py2.7.egg/msmbuilder/scripts/CalculateImpliedTimescales.py", line 67, in run trimming=trimming, symmetrize=symmetrize, n_procs=nProc) File "/home/tim/anaconda/lib/python2.7/site-packages/msmbuilder-2.8.2-py2.7.egg/msmbuilder/msm_analysis.py", line 237, in get_implied_timescales lags = result.get(999999) File "/home/tim/anaconda/lib/python2.7/multiprocessing/pool.py", line 554, in get raise self._value ValueError: k must be between 1 and ndim(A)-1

It's always prompt " k must be between 1 and ndim(A)-1", and I'm confused.

Seriline

[mpharrigan]

Hi Seriline, It still looks like you don't end up with enough states

01:24:02 - Calculating implied timescales at lagtime 1
01:24:02 - Selected component 0 with population 0.700740
01:24:02 - You cannot calculate 11 eigenvectors from a 6 x 6 matrix.
01:24:02 - Instead, calculating 6 eigenvectors.

At lagtime=1, you're trimming 30% of your data (which is not great, but doesn't necessarily lead to a problem)... but after the trimming step you are only left with 6 states. It crashes at lagtime=8, which means at that lagtime you are probably left with < ~3 states

This could be because all of your sampling is concentrated in a small region of conformational space. Perhaps @rmcgibbo or @schwancr could give their opinion, but it looks like you just need more sampling.

Matt

[Serilin]

Matt, Appreciate for your quick reply. Actually, the 200 ns trajectory was a single long trajectory. That is, it comes from one original conformation. Does it mean that this trajectory isn't proper for MSM analysis, right ? If not, is there anyway to increase the samping? Seriline