Iterate ptv_is trajectories instead of reading entire list at once.

[yosefm]

Default for trajectories() and trajectories_ptvis() is still to read everything to memory, but now you can use iter_trajectories_ptvis() or trajectories(..., iter_allowed=True) to get an iterator. Then when you iterate over trajectories in a for loop, they are read only when needed and forgotten immediately after. See this used in save_particles_table().

Also in: finally, a test for reading ptv_is files.

[alexlib]

In order to run nosetests I had to add the tests/ to the directory on line 18 in test_io.py

self.tmpl = "tests/testing_fodder/ptvis/ptv_is.%d"

Patch is here:

http://pastebin.com/YKfdEwXV

[alexlib]

the tmp.py fails on the line that uses forward difference, when the traj_len is not defined. it's possible to default the traj_min_len = 2

The patch is here:

http://pastebin.com/y6cRVG2b

and fixed in my branch.

[yosefm]

Alex, what is tmp.py? At any rate, since we don't do the forward-difference thing for xuap files, a better solution would be to default it to None, then select based on the format. I'll do a commit implementing this in a sec.

[yosefm]

Ok, please test the two new commits and if everything is ok, I'll merge.

[alexlib]

I added this failing test to tests_io.py the problem is with reading the xuap files as there we estimate acceleration, and then the traj_min_length shall not be None, but at least 2

diff --git a/tests/test_io.py b/tests/test_io.py index c37ab94..c3de99a 100644 --- a/tests/test_io.py +++ b/tests/test_io.py @@ -49,4 +49,15 @@ class TestPtvis(unittest.TestCase): nptest.assert_array_almost_equal(trj.accel(), correct.accel()) nptest.assert_array_almost_equal(trj.time(), correct.time()) self.failUnlessEqual(trj.trajid(), correct.trajid())

• def test_iter_trajectories_ptvis_xuap(self):
• inName = './data/particles/xuap.%d'
• trjs = io.trajectories_ptvis(inName, xuap = True, traj_min_len = 2)
• self.failUnlessEqual(len(trjs), 332)

• here it fails due to the same problem with traj_min_len

• trjs = io.trajectories_ptvis(inName, xuap = True, traj_min_len = None)
• self.failUnlessEqual(len(trjs), 332)

[alexlib]

The following patch resolves the problem with the xuap files:

diff --git a/flowtracks/io.py b/flowtracks/io.py index 31759fe..9e87d91 100644 --- a/flowtracks/io.py +++ b/flowtracks/io.py @@ -247,7 +247,7 @@ def iter_trajectories_ptvis(fname, first=None, last=None, frate=1., xuap=False, ('pos_int', '3f8'), ('vel', '3f8'), ('acc', '3f8')]) skip = 0 count_base = 1

• def_tr_len = 1
• def_tr_len = 2 else: fmt = np.dtype([('prev', 'i4'), ('next', 'i4'), ('pos', '3f8')]) skip = 1

[alexlib]

created a separate pull request from my branch to your branch.

[yosefm]

Ok, I see that the problem was with the acceleration calculation, so xuap also needs 2 frames min. Merged with your fix. Thanks for the test!