Closed marcodeltutto closed 5 years ago
The distributed_larcv_interface now has 3 options to read the data:
This option can be specified in the distributed_larcv_interface constructor:
I have updated this branch with the threadio tests and hdf5-fix branch, if all tests pass I will merge.
All tests have passed. I am going to merge into develop.
It should be noted that there are not tests available yet for the distributed IO interface. I will open an issue to work on this.
This branch includes changes that allow running ThreadProcessor on several MPI processes reading different entries in the file. The start entry can now be specified explicitly, and also how many entries have to be skipped between two consecutive readings.
The distributed_larcv_interface now allows the option "read_from_all_ranks" in the constructor. If this is set to true, an instance of larcv_threadio (and so of ThreadProcessor) will be present in all ranks. The start entry of every rank is set to (rank minibatch_size), and the number of entries to skip between to readings is set to (size minibatch_size), where rank is the rank number, and size is the number of ranks.