AIDASoft / DD4hep

Detector Description Toolkit for High Energy Physics
http://dd4hep.cern.ch
GNU Lesser General Public License v3.0
47 stars 93 forks source link

Streaming Hepmc3 ROOT input from XRootD is broken in 01-29 #1281

Closed veprbl closed 4 weeks ago

veprbl commented 1 month ago

Check duplicate issues.

Goal

Stream big files without downloading them locally

Operating System and Version

doesn't matter

compiler

doesn't matter

ROOT Version

doesn't matter

DD4hep Version

01-29

Reproducer

ddsim   --runType batch  --filter.tracker edep0   -v WARNING   --numberOfEvents 100   --compactFile $DETECTOR_PATH/epic.xml   --inputFiles root://dtn-eic.jlab.org//work/eic2/EPIC/EVGEN/DIS/NC/10x100/minQ2=1/pythia8NCDIS_10x100_minQ2=1_beamEffects_xAngle=-0.025_hiDiv_1.hepmc3.tree.root   --outputFile sim_output/epic/backgrounds/EPIC/EVGEN/DIS/NC/10x100/minQ2=1/pythia8NCDIS_10x100_minQ2=1_beamEffects_xAngle=-0.025_hiDiv_1.edm4hep.root

gives

ERROR: The inputfile 'root://dtn-eic.jlab.org//work/eic2/EPIC/EVGEN/DIS/NC/10x100/minQ2=1/pythia8NCDIS_10x100_minQ2=1_beamEffects_xAngle=-0.025_hiDiv_1.hepmc3.tree.root' does not exist

Additional context

The likely cause for regression is in https://github.com/AIDASoft/DD4hep/commit/da34149cb0af4e873e275d3b6b9cb0523ffe8889 (#1247)

andresailer commented 1 month ago

Just not check when the URL starts with anything but / or is there some xrd stat call we could make instead?

wdconinc commented 1 month ago

xrdfs stat, e.g.

xrdfs dtn-eic.jlab.org stat /work/eic2/EPIC/EVGEN/BACKGROUNDS/BEAMGAS/proton/pythia8.306-1.0/100GeV/pythia8.306-1.0_ProtonBeamGas_100GeV_run001.hepmc3.tree.root
andresailer commented 1 month ago

Thanks!

I will try with the python API https://xrootd.slac.stanford.edu/doc/doxygen/5.6.4/python/modules/client/file.html#XRootD.client.File.stat

veprbl commented 1 month ago

Can we forego check for XRootD or even all URLs completely? It makes no sense to impose a separate mechanism that doesn't have the same behaviors as TFile, that will lead to failures in some other edge cases.

andresailer commented 4 weeks ago

Please see #1283.