search

art-framework-suite / art

The implementation of the art physics event processing framework
Other
2 stars 7 forks source link

New event mixing mode #109

Closed knoepfel closed 4 months ago

knoepfel commented 2 years ago

This issue has been migrated from https://cdcvs.fnal.gov/redmine/issues/25876 (FNAL account required) Originally created by @gaponenko on 2021-05-26 05:09:54

Hello,

In art/Framework/IO/ProductMix/MixHelper.h I see several "random" read modes for event mixing, as well as a sequential. Random modes offer better statistical properties for some use cases, but impose a larger I/O overhead compared to the sequential mode. There are other use cases where we want to re-sample events from an input file in a uniform way, and it does not matter if sequential "main" events use sequential or "random" events from the secondary file. For such cases the sequential mode is an obvious choice. However a user has to return a proper random value from the eventsToSkip() call to make sure all events from a secondary file are used by different jobs. Otherwise the sequential mode will case each job to take the same events from the beginning of the file while leaving events at the end unused. That eventsToSkip() return value has to be uniformly sampled between 0 and (numSecondaryEvents-1). (I assume wrapFiles:true.) However the number of events in the secondary file is not exposed to the user, as far as I can see. Instead of adding an extra interface, I suggest to add another file reading mode to support the "almost sequential" use case. The new mode should start at a random event in a secondary file, then behaves like the sequential mode.

Andrei

knoepfel commented 9 months ago

Superseded by https://github.com/art-framework-suite/art/issues/138.

knoepfel commented 4 months ago

Reopening issue based on conversation with @gaponenko on April 17, 2024.

knoepfel commented 4 months ago

This issue has been resolved with commits https://github.com/art-framework-suite/art/commit/9a405d1cdfd5e2ed170e5a25f6b61e82b328c48e and https://github.com/art-framework-suite/art/commit/5b500a6a3da02ea31220de988066822aac804ff9. The new mixing mode is called randomOffset, and it uses the (perhaps default) seed value specified for the mix filter.

This feature will be included as part of art 3.15.

knoepfel commented 4 months ago

Released as part of art 3.15.