DeepLearnPhysics / larcv3

Third version of larcv. This is a complete replacement for larcv2.
MIT License
11 stars 6 forks source link

Reopening Group at every get_data call #8

Closed coreyjadams closed 5 years ago

coreyjadams commented 5 years ago

In this line in IO manager:

https://github.com/DeepLearnPhysics/larcv3/blob/0663596d3a0544e56b289b3d779e4fae544d96a6/src/larcv3/core/dataformat/IOManager.cxx#L745

The group is re-opened at every call to get_data. This is a pretty big overhead for reading large batches of data, since this small function gets called many many times.

This needs to be modified to maintain a list of open groups and pass them to the deserialization as needed.

marcodeltutto commented 5 years ago

15 should fix this. A list of opened groups is stored in a map. Only new groups are opened.