improved benchmarking

[franknoe]

This PR improves efficiency by correcting some mistakes in the switch for sparse treatment.

@marscher and @fabian-paul : please run python variational/estimators/tests/benchmark_moments.py again on Linux and Windows and share the resulting output text. All "speed-up" entries should be around 1 or greater.

If successful, we can start using the covariance calculation in TICA. The way to do this is as follows:

• copy the modules estimators/moments.py, estimators/running_moments.py, the code in estimators/covar_c, and the respective tests under estimators/tests into a subpackage of pyemma.coordinates (temporary solution until the API is stable)
• In TICA, initialize a RunningCovar estimator with estimators.running_covar(xx=True, xy=True, yy=False, remove_mean=True, symmetrize=True, nsave=nsave)
• In PCA, initialize a RunningCovar estimator with estimators.running_covar(xx=True, xy=False, yy=False, remove_mean=True, symmetrize=False, nsave=nsave)
• In both cases, nsave should be set to ceil(log2(nchunks)), where nchunks is the total number of chunks needed across all the data.
• Chunksizes should be set to 5000 at least, better 10000.
• For each chunk, simply call add(X) (PCA) or add(X, Y) (TICA), where X is the data chunk and Y is the time-lagged chunk. Make sure that X and Y are always corresponding, i.e. they must have the same length (possibly Y needs to be truncated), and if there is no Y, then there is no X either.
• The TICA option force_ev_le_one becomes obsolete (we can deprecate it, but it will have no effect anymore). We will now always run TICA if that flag were set True (not that this is a change in API behavior, but it's right to do it this way).
• Please keep this in a separate pyemma branch. Before creating a PR we should benchmark this branch against the devel branch or last release using real data to make sure that nothing was introduced. I suggest using (1) the BPTI data, and (2) Contact trajectories from Nuria's Barnase-Barstar (she is still generating that, but it will be a nice test set to see if we can exploit any sparsity in the data). Once the benchmarking is fine, we can create a PR.

[fabian-paul]

Windows results: https://gist.github.com/fabian-paul/755a2f2a5780f41b96d5

[franknoe]

Great. No red flags.

Am 25/11/15 um 11:30 schrieb fabian-paul:

Windows results: https://gist.github.com/fabian-paul/755a2f2a5780f41b96d5

— Reply to this email directly or view it on GitHub https://github.com/markovmodel/variational/pull/23#issuecomment-159564780.

---

Prof. Dr. Frank Noe Head of Computational Molecular Biology group Freie Universitaet Berlin

Phone: (+49) (0)30 838 75354 Web: research.franknoe.de

Mail: Arnimallee 6, 14195 Berlin, Germany

[marscher]

On allegro there are also no more slowdowns https://gist.github.com/marscher/7c0f88f44548a8abb779

[marscher]

nsave parameter has to be at least 2, otherwise one will get an index error while adding chunks:

  File "/home/marscher/workspace/pyemma/pyemma/coordinates/transform/pca.py", line 179, in _param_add_data
    self._covar.add(X)
  File "/home/marscher/workspace/pyemma/pyemma/coordinates/estimators/covar/running_moments.py", line 219, in add
    self.storage_XX.store(Moments(w, s_X, s_X, C_XX))
  File "/home/marscher/workspace/pyemma/pyemma/coordinates/estimators/covar/running_moments.py", line 130, in store
    self.storage[-1].combine(moments, mean_free=self.remove_mean)
IndexError: list index out of range

[franknoe]

ah, this is a bug. Please make an issue and assign it to me.

Am 25/11/15 um 15:56 schrieb Martin K. Scherer:

nsave parameter has to be at least 2, otherwise one will get an index error while adding chunks:

File"/home/marscher/workspace/pyemma/pyemma/coordinates/transform/pca.py", line179,in _param_add_data self._covar.add(X) File"/home/marscher/workspace/pyemma/pyemma/coordinates/estimators/covar/running_moments.py", line219,in add self.storage_XX.store(Moments(w, s_X, s_X, C_XX)) File"/home/marscher/workspace/pyemma/pyemma/coordinates/estimators/covar/running_moments.py", line130,in store self.storage[-1].combine(moments,mean_free=self.remove_mean) IndexError:list index out ofrange

— Reply to this email directly or view it on GitHub https://github.com/markovmodel/variational/pull/23#issuecomment-159632080.

---

Prof. Dr. Frank Noe Head of Computational Molecular Biology group Freie Universitaet Berlin

Phone: (+49) (0)30 838 75354 Web: research.franknoe.de

Mail: Arnimallee 6, 14195 Berlin, Germany

[marscher]

I've already started to integrate the code into PCA and TICA. I'll copy the bugfixes over as soon as it is merged here.

[marscher]

Is it possible to provide the means? We introduced this feature for PCA/TICA but it seems this is now obsolete.

[franknoe]

Not yet, but easy to do. Create an issue.

Am 25/11/15 um 16:03 schrieb Martin K. Scherer:

Is it possible to provide the means? We introduced this feature for PCA/TICA but it seems this is now obsolete.

— Reply to this email directly or view it on GitHub https://github.com/markovmodel/variational/pull/23#issuecomment-159634262.

---

Prof. Dr. Frank Noe Head of Computational Molecular Biology group Freie Universitaet Berlin

Phone: (+49) (0)30 838 75354 Web: research.franknoe.de

Mail: Arnimallee 6, 14195 Berlin, Germany