search

art-framework-suite / art

The implementation of the art physics event processing framework
Other
2 stars 7 forks source link

Guidance writing Custom Input Source with --nskip/skipEvents enabled #144

Closed calcuttj closed 5 months ago

calcuttj commented 6 months ago

Hello, I'm attempting write a new input source that operates on a custom NTuple/ROOT Tree (not artroot) file. I'd like to be able to use the --nskip CLI flag and/or skipEvents fhicl parameter when using this. However I can't seem to find much documentation on how to actually do this. I would greatly appreciate guidance on this

Please see this feature branch in the DUNE software on my attempts to do this. I have the input source working up to, but not including the skip events portion and have an analyzer module to demonstrate

Specifically I try to implement skipEvents here thinking it would automagically work but that doesn't seem to be the case. See below, where I try to use --nskip 2 and thus art event should start at 2 (my thinking was that I'd increment the event n times in that steering class before iterating through the events)

Utils$ lar -c run_h4_test.fcl /pnfs/dune/persistent/dunepro/beam_data/simulation/G4beamlineVersion3.06/v34b_1GeV_G4beamlineVersion3.06/H4_v34b_1GeV_-27.7_10M_1.root --nskip 2
%MSG-i MF_INIT_OK:  Early 08-Mar-2024 16:15:02 CST JobSetup
Messagelogger initialization complete.
%MSG

RooFit v3.60 -- Developed by Wouter Verkerke and David Kirkby 
                Copyright (C) 2000-2013 NIKHEF, University of California & Stanford University
                All rights reserved, please read http://roofit.sourceforge.net/license.txt

Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v09_85_00d00/gdml/protodunehd_v6_refactored.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 1148
Info in <TGeoManager::CloseGeometry>: 25608 nodes/ 5535 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
%MSG-i GeometryCore:  Early 08-Mar-2024 16:15:04 CST JobSetup
New detector geometry loaded from 
    /cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v09_85_00d00/gdml/protodunehd_v6_refactored.gdml
    /cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v09_85_00d00/gdml/protodunehd_v6_refactored_nowires.gdml

%MSG
%MSG-i GeometryCore:  Early 08-Mar-2024 16:15:04 CST JobSetup
Sorting volumes...
%MSG
%MSG-i DuneApaChannelMapAlg:  Early 08-Mar-2024 16:15:04 CST JobSetup
Initializing channel map...
%MSG
Cryostat 0:
  10240 total channels
  4 APAs
  For all identical APA:
    Number of channels per APA = 2560
    U channels per APA = 800
    V channels per APA = 800
    Z channels per APA = 960
    Pitch in U Plane = 0.4669
    Pitch in V Plane = 0.4669
    Pitch in Z Plane = 0.4792
%MSG-i SimpleChannelStatusService:  Early 08-Mar-2024 16:15:04 CST JobSetup
Loaded from configuration:
  - 0 bad channels
  - 0 noisy channels
  - largest channel ID: 10239, largest present: 10239
%MSG
%MSG-i setupProvider<DetectorPropertiesStandard>:  Early  08-Mar-2024 16:15:04 CST JobSetup
Asked to ignore 1 keys: 'InheritNumberTimeSamples'
%MSG
%MSG-i PhotonVisibilityService:  Early 08-Mar-2024 16:15:04 CST JobSetup
PhotonVisbilityService initializing
%MSG
%MSG-i SignalShapingServiceDUNE:  Early 08-Mar-2024 16:15:04 CST JobSetup
Getting Filter from .fcl file
%MSG
%MSG-i SignalShapingServiceDUNE:  Early 08-Mar-2024 16:15:04 CST JobSetup
 using the field response provided from a .root file 
%MSG
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
%MSG-i H4Beam:  OpenFile 08-Mar-2024 16:15:04 CST BeforeEvents H4BeamInput_source.cc:13
Opening Beam Input File /pnfs/dune/persistent/dunepro/beam_data/simulation/G4beamlineVersion3.06/v34b_1GeV_G4beamlineVersion3.06/H4_v34b_1GeV_-27.7_10M_1.root
%MSG
Opening /pnfs/dune/persistent/dunepro/beam_data/simulation/G4beamlineVersion3.06/v34b_1GeV_G4beamlineVersion3.06/H4_v34b_1GeV_-27.7_10M_1.root
0x9f8bf10
Getting run 129 128 1
Front face tree: 0xa242590
Found 96951 potential events
- Candidate event 391 trigger particle of type 211 not at the front face... searching for children
  - Found child with PDG = -13
- Candidate event 735 trigger particle of type 211 not at the front face... searching for children
  - Found child with PDG = -13
- Candidate event 756 trigger particle of type 211 not at the front face... searching for children
  - Found child with PDG = -13
- Candidate event 801 trigger particle of type 2212 not at the front face... searching for children
  - Found child with PDG = 2112
  - Found child with PDG = 2212
- Candidate event 1079 trigger particle of type 211 not at the front face... searching for children
  - Found child with PDG = 2112
  - Found child with PDG = 2212
  - Found child with PDG = 2212
  - Found child with PDG = 2112
  - Found child with PDG = 2112
  - Found child with PDG = 2212
  - Found child with PDG = 2112
  - Found child with PDG = 211
- Candidate event 1189 trigger particle of type 2212 not at the front face... searching for children
  - Found child with PDG = 2112
  - Found child with PDG = 2212
- Candidate event 1229 trigger particle of type 2212 not at the front face... searching for children
  - Found child with PDG = 211
  - Found child with PDG = 2112
  - Found child with PDG = 2112
Proto trigger list has 1434 events
Issue found with event 2073090. Removing it from the trigger list - 1433 remain
Issue found with event 5201514. Removing it from the trigger list - 1432 remain
Issue found with event 5289155. Removing it from the trigger list - 1431 remain
Issue found with event 5647434. Removing it from the trigger list - 1430 remain
Issue found with event 5725990. Removing it from the trigger list - 1429 remain
Issue found with event 7959797. Removing it from the trigger list - 1428 remain
Issue found with event 8954637. Removing it from the trigger list - 1427 remain
Issue found with event 9515190. Removing it from the trigger list - 1426 remain
Issue found with event 9701955. Removing it from the trigger list - 1425 remain
Issue found with event 9736152. Removing it from the trigger list - 1424 remain
 - Finished adding information from VirtualDetector/TOF1
 - Finished adding information from VirtualDetector/BPROF1
 - Finished adding information from VirtualDetector/BPROF2
 - Finished adding information from VirtualDetector/BPROF3
 - Finished adding information from VirtualDetector/BPROFEXT
 - Finished adding information from VirtualDetector/BPROF4
Final trigger list has 1424 events
%MSG-i H4Beam:  OpenFile 08-Mar-2024 16:15:14 CST BeforeEvents H4BeamInput_source.cc:20
Has events: 1424
%MSG
Art event: 1 1 0
BFS: 1
Art event: 1 1 1
BFS: 2
Art event: 1 1 2
BFS: 3
Art event: 1 1 3
BFS: 4
Art event: 1 1 4
BFS: 5
Art event: 1 1 5
BFS: 6
Art event: 1 1 6
BFS: 7
Art event: 1 1 7
BFS: 8
Art event: 1 1 8
BFS: 9
Art event: 1 1 9
BFS: 10
knoepfel commented 5 months ago

@calcuttj, the art Source<T> template does not automatically support the skipEvents behavior. However, art makes sure that you have access to the skipEvents configuration parameter in the fhicl::ParameterSet in the H4BeamInputDetail constructor. With that, you can implement the skip-event functionality directly in readFile or readNext (wherever is appropriate).

Does that make sense?

calcuttj commented 5 months ago

Yes it does, thanks Kyle I think I have it working

I'll do some testing to make sure and will close/resolve this when ready