search

art-framework-suite / art

The implementation of the art physics event processing framework
Other
2 stars 7 forks source link

Modules after custom input source not recognizing -n flag #114

Closed calcuttj closed 2 years ago

calcuttj commented 2 years ago

Describe the bug In dunetpc, we've written a custom input source that decodes headers in an HDF5 file. This source creates an art Event for each header in the file and stores information regarding the file name and how to access the data within the file. Downstream modules use this to access and decode data from within the HDF5 file. We'd like to use the -n flag to only process that many events using the downstream modules, however this does not appear to work.

Below is output from lar, using the source + downstream module to process the files. Shown in bold is output showing multiple events are being processed by downstream modules.

VDColdboxFiles$ lar -c vdcoldbox_raw_dataprep.fcl np02_bde_coldbox_run011910_0005_20211028T151327.hdf5 -n 1 ... ToolBasedRawDigitPrepService:beginEvent: Begin processing run 11910-1 event 806 with 5 tools. DataPrepByApaModule::produce: Raw data read status: 0 DataPrepByApaModule::produce: Allowed tick count range is (8151, 8232) ToolBasedRawDigitPrepService:prepare: Processing 1792 channels with 5 tools. ToolBasedRawDigitPrepService:prepare: Running tool digitReader ToolBasedRawDigitPrepService:prepare: Running tool adcSampleFiller ToolBasedRawDigitPrepService:prepare: Running tool vdcbb_adcChannelPedestalPlotter ToolBasedRawDigitPrepService:prepare: Running tool vdcbb_adcChannelRawRmsPlotter ToolBasedRawDigitPrepService:prepare: Running tool adcKeepAllSignalFinder ToolBasedRawDigitPrepService:endEvent: End processing run 11910-1 event 806 with 5 tools. 05-Nov-2021 09:19:52 CDT Opened output file with pattern "rawinput_test.root" ToolBasedRawDigitPrepService:beginEvent: Begin processing run 11910-1 event 807 with 5 tools. DataPrepByApaModule::produce: Raw data read status: 0 DataPrepByApaModule::produce: Allowed tick count range is (8151, 8232) ToolBasedRawDigitPrepService:prepare: Processing 1792 channels with 5 tools. ToolBasedRawDigitPrepService:prepare: Running tool digitReader ToolBasedRawDigitPrepService:prepare: Running tool adcSampleFiller ToolBasedRawDigitPrepService:prepare: Running tool vdcbb_adcChannelPedestalPlotter ToolBasedRawDigitPrepService:prepare: Running tool vdcbb_adcChannelRawRmsPlotter ToolBasedRawDigitPrepService:prepare: Running tool adcKeepAllSignalFinder ToolBasedRawDigitPrepService:endEvent: End processing run 11910-1 event 807 with 5 tools. ToolBasedRawDigitPrepService:beginEvent: Begin processing run 11910-1 event 808 with 5 tools. DataPrepByApaModule::produce: Raw data read status: 0 DataPrepByApaModule::produce: Allowed tick count range is (8151, 8232) ToolBasedRawDigitPrepService:prepare: Processing 1792 channels with 5 tools. ToolBasedRawDigitPrepService:prepare: Running tool digitReader ToolBasedRawDigitPrepService:prepare: Running tool adcSampleFiller ToolBasedRawDigitPrepService:prepare: Running tool vdcbb_adcChannelPedestalPlotter ToolBasedRawDigitPrepService:prepare: Running tool vdcbb_adcChannelRawRmsPlotter ToolBasedRawDigitPrepService:prepare: Running tool adcKeepAllSignalFinder ToolBasedRawDigitPrepService:endEvent: End processing run 11910-1 event 808 with 5 tools.

To Reproduce To reproduce this:

Please let me know any other information I can give regarding this issue

Thanks -Jake

calcuttj commented 2 years ago

Here's the fcl file that we run: https://cdcvs.fnal.gov/redmine/projects/dunetpc/repository/revisions/v09_35_00/entry/dune/VDColdbox/RawDecoding/vdcoldbox_raw_dataprep.fcl

knoepfel commented 2 years ago

Thank you for the detailed instructions, @calcuttj. We have reproduced the error and implemented a fix with commit https://github.com/art-framework-suite/art/commit/488685f86c44c6aa440b2cd2c944b1e0b99c81d3. Incidentally, the Source template was always designed to support the maxEvents parameter (or -n program option), but the ability was unintentionally removed at some point.

We anticipate including this fix in art 3.10, which should be available in the next couple weeks.

calcuttj commented 2 years ago

Great thanks for the quick help!