search

OpenPTV / postptv

Post processing routines for analysing PTV data.
GNU General Public License v3.0
14 stars 13 forks source link

Ptvis memory consumption #2

Closed yosefm closed 10 years ago

yosefm commented 10 years ago

Speed up, and more importantly, greatly reduce the memory footprint of reading ptv_is files, with two measures:

  1. Hold only the active trajectories in memory at any given frame (minor, but needed for 2).
  2. Allow on-line short trajectory discarding.
alexlib commented 10 years ago

Reading Ron's single files using the

trajects = trajectories_ptvis('/Users/alex/Documents/OpenPTV/ronshnapp/python_turbulence/Tornado_PTV_Data/100_smooth.txt')

fails with

IndexError: list index out of range

or similar errors (on different files). The files I tested are from Ron, the example file is here:

https://dl.dropboxusercontent.com/u/5266698/100_smooth.txt

and this file creates error in a different place:

https://dl.dropboxusercontent.com/u/5266698/40_rough.txt

yosefm commented 10 years ago
  1. On the first one you didn't remove the first line (particle count) on the frames.
  2. Does the second one only fail with this PR applied, or does it fail without this PR too?

I'll explore more soon.

alexlib commented 10 years ago

40_rough.txt fails also on the initial commit

45713e7de455eb71b38f817e6e5baf6f984de7be

yosefm commented 10 years ago

Merging this, as we found that the bug is not in this PR but before, so it will be addressed separately.