problem with array dimension broadcasting running `msm-pipeline`

[sonyahanson]

Not quite sure of the cause of this.

getting output of TICA:   0% (   2/1377) [                       ] eta 8:39:12 \Traceback (most recent call last):
  File "pipeline.py", line 190, in <module>
    run_pipeline(fnames, project_name = project_name)
  File "pipeline.py", line 103, in run_pipeline
    Y = tica.get_output()
  File "/cbio/jclab/home/hansons/opt/anaconda/lib/python2.7/site-packages/pyEMMA-2.2.1-py2.7-linux-x86_64.egg/pyemma/coordinates/transform/transformer.py", line 205, in get_output
    return super(StreamingTransformer, self).get_output(dimensions, stride, skip, chunk)
  File "/cbio/jclab/home/hansons/opt/anaconda/lib/python2.7/site-packages/pyEMMA-2.2.1-py2.7-linux-x86_64.egg/pyemma/coordinates/data/_base/iterable.py", line 154, in get_output
    trajs[itraj][it.pos:it.pos + L, :] = chunk[:, dimensions]
ValueError: could not broadcast input array from shape (5236,1348) into shape (5196,1348)

Note that I had to make a change to the end of pipeline.py from the current master, but haven't committed it yet because I wanted to make sure it worked. I find it unlikely these are related, but maybe...:

if __name__ == '__main__':
    import sys
    path_to_trajs = sys.argv[1]
    if len(sys.argv) > 2:
        project_name = sys.argv[2]
    else:
        project_name = 'abl'

[maxentile]

Thanks for pointing this out!

Looks like something confusing happened inside PyEMMA's iterable get_output() method, where the chunk size was not equal to L: trajs[itraj][it.pos:it.pos + L, :] = chunk[:, dimensions] --> could not broadcast input array from shape (5236,1348) into shape (5196,1348).

Do you still have this issue in PyEMMA 2.2.2? Looks like L is set to len(chunk), which I would assume prevents this from happening... https://github.com/markovmodel/PyEMMA/blob/devel/pyemma/coordinates/data/_base/iterable.py#L220

[sonyahanson]

Ahhh. Thanks! Will try PyEMMA 2.2.2 and see if that fixes it.

[sonyahanson]

Resolved. Thanks!