search

key4hep / EDM4hep

Generic event data model for HEP collider experiments
https://cern.ch/edm4hep
Apache License 2.0
25 stars 36 forks source link

FCCAnalyses samples #228

Open kjvbrt opened 1 year ago

kjvbrt commented 1 year ago

Hi,

I've tested the samples we use for the studies with the several tools which might be used to understand what is saved in those files. Most important one being podio-dump others are good to have. The test was done on CentOS7 machine and files are stored on EOS.

y - runs as expected x - crash/problem

podio-dump

stable nightlies
spring2021 x1 x1
winter2023 y y

edm4hep2json

stable nightlies
spring2021 x2 x4
winter2023 x3 y

collInfo

From FCC AuxTools.

stable nightlies
spring2021 y y
winter2023 y y

Test files

Errors

  1. RecoParticleRefData warning + segmentation violation 
    
TClass::Init:0: RuntimeWarning: no dictionary for class edm4hep::RecoParticleRefData is available
PODIO: Reconstructing CollectionTypeInfo branch from other sources in file: '/eos/experiment/fcc/ee/generat
ion/DelphesEvents/spring2021/IDEA/p8_ee_ZH_ecm240/events_101027117.root'
*** Break *** segmentation violation

. . .

9 0x00007fa8cd20de16 in TClass::NewObject (this=this

entry=0x0, defConstructor=defConstructor entry=TClass::kClassNew, quiet=quiet entry=false) at /tmp/gitlab-runner/spack-stage/spack-stage-root-6.26.10-i4xdfrlayuwec4fboqshcon324ybwn2e/sp ack-src/core/meta/src/TClass.cxx:4982

10 0x00007fa8cd210950 in TClass::New (this=this

entry=0x0, defConstructor=defConstructor entry=TClass::kClassNew, quiet=quiet entry=false) at /tmp/gitlab-runner/spack-stage/spack-stage-root-6.26.10-i4xdfrlayuwec4fboqshcon324ybwn2e/sp ack-src/core/meta/src/TClass.cxx:4966

11 0x00007fa8b8c43fa3 in podio::ROOTLegacyReader::createCollectionBranches (this=this

entry=0x6f9dd40, collInfo=...) at /tmp/gitlab-runner/spack-stage/spack-stage-podio-0.16.2-cbwgcvkkkqcxq7err ukecsdvy3bi2tr5/spack-src/src/ROOTLegacyReader.cc:194

12 0x00007fa8b8c45172 in podio::ROOTLegacyReader::openFiles (this=0x6f9dd40, filenames=...) at /tmp/gitlab

-runner/spack-stage/spack-stage-podio-0.16.2-cbwgcvkkkqcxq7errukecsdvy3bi2tr5/spack-src/src/ROOTLegacyReade r.cc:167


2. Different crashes depending on the requested collection. For ReconstructedParticles:

$ edm4hep2json /eos/experiment/fcc/ee/generation/DelphesEvents/spring2021/IDEA/p8_ee_ZH_ecm240/events_10102 7117.root -n 4 -l ReconstructedParticles Warning in : no dictionary for class edm4hep::RecoParticleRefData is available PODIO: Reconstructing CollectionTypeInfo branch from other sources in file: '/eos/experiment/fcc/ee/generat ion/DelphesEvents/spring2021/IDEA/p8_ee_ZH_ecm240/events_101027117.root' PODIO: Cannot create the collection type 'edm4hep::RecoParticleRefCollection' stored in branch 'Electron'. Contents of this branch cannot be read. PODIO: Cannot create the collection type 'edm4hep::RecoParticleRefCollection' stored in branch 'Muon'. Cont ents of this branch cannot be read. PODIO: Cannot create the collection type 'edm4hep::RecoParticleRefCollection' stored in branch 'AllMuon'. C ontents of this branch cannot be read. PODIO: Cannot create the collection type 'edm4hep::RecoParticleRefCollection' stored in branch 'Photon'. Co ntents of this branch cannot be read.

Break segmentation violation

. . .

6 0x00007fe4f0db3275 in podio::root_utils::setCollectionAddresses (branche

s=..., collBuffers=...) at /tmp/gitlab-runner/spack-stage/spack-stage-podio-0.16.2-cbwgcvkkkqcxq7errukecsdv y3bi2tr5/spack-src/src/rootUtils.h:91

7 podio::ROOTReader::getCollection (this=0x7ffe25bf8ac0, collInfo=...) at /tmp/gitlab-runner/spack-stage/

spack-stage-podio-0.16.2-cbwgcvkkkqcxq7errukecsdvy3bi2tr5/spack-src/src/ROOTReader.cc:112

8 0x00007fe4f0db3685 in podio::ROOTReader::readCollection (this=0x7ffe25bf8ac0, name=...) at /tmp/gitlab-

runner/spack-stage/spack-stage-podio-0.16.2-cbwgcvkkkqcxq7errukecsdvy3bi2tr5/spack-src/src/ROOTReader.cc:63

9 0x00007fe4efc54d29 in podio::EventStore::doGet (this=this

entry=0x7ffe25bf8b60, name=..., collection= 0x7ffe25bf7ff0: 0x0, setReferences=setReferences entry=true) at /tmp/gitlab-runner/spack-stage/spack-stage-podio-0.16.2-cbwgcvkkkqcxq7errukecsdvy3bi2tr5/spa ck-src/src/EventStore.cc:68

10 0x00007fe4efc54fdd in podio::EventStore::get (this=0x7ffe25bf8b60, id=, collection=

0x7ffe25bf7ff0: 0x0) at /tmp/gitlab-runner/spack-stage/spack-stage-podio-0.16.2-cbwgcvkkkqcxq7errukecsdvy3b i2tr5/spack-src/src/EventStore.cc:30

11 0x00007fe4f0d1e811 in edm4hep::ReconstructedParticleCollectionData::setReferences(podio::ICollectionPro

vider const*, bool) () from /cvmfs/sw.hsf.org/spackages7/edm4hep/0.7.2/x86_64-centos7-gcc11.2.0-opt/nfdxp/l ib64/libedm4hep.so

12 0x00007fe4efc54d48 in podio::EventStore::doGet (this=0x7ffe25bf8b60, name=..., collection=

0x7ffe25bf8ef0: 0x0, setReferences=) at /tmp/gitlab-runner/spack-stage/spack-stage-podio-0.1 6.2-cbwgcvkkkqcxq7errukecsdvy3bi2tr5/spack-src/src/EventStore.cc:71

13 0x0000000000411d8f in void read_events(std::__cxx11::basic_string<char, std::char_tr

aits, std::allocator > const&, std::cxx11::basic_string<char, std::char_traits, std::al locator > const&, std::cxx11::basic_string<char, std::char_traits, std::allocator > con st&, int, bool) ()

14 0x00000000004098c2 in main ()


3. Crash at SetBranchAddress<podio::CollectionIDTable>

6 0x00007f16588c8906 in TTree::SetBranchAddress (ptr=0x0, add=0x7ffed9fb3050, b

name=0x7f16588d8138 "CollectionIDs", this=0x0) at /cvmfs/sw.hsf.org/spackages7/root/6.26.10/x86_64-centos7- gcc11.2.0-opt/i4xdf/include/root/TTree.h:588

7 podio::ROOTReader::openFiles (this=this

entry=0x7ffed9fb3b10, filenames=...) at /tmp/gitlab-runner/spack-stage/spack-stage-podio-0.16.2-cbwgcvkkkqc xq7errukecsdvy3bi2tr5/spack-src/src/ROOTReader.cc:155

8 0x00007f16588c8d8e in podio::ROOTReader::openFile (this=0x7ffed9fb3b10, filename=...) at /tmp/gitlab-ru

nner/spack-stage/spack-stage-podio-0.16.2-cbwgcvkkkqcxq7errukecsdvy3bi2tr5/spack-src/src/ROOTReader.cc:140

9 0x0000000000411b1f in void read_events(std::__cxx11::basic_string<char, std::char_tr

aits, std::allocator > const&, std::cxx11::basic_string<char, std::char_traits, std::al locator > const&, std::cxx11::basic_string<char, std::char_traits, std::allocator > con st&, int, bool) ()

10 0x00000000004098c2 in main ()


4. Cannont find tree podio_metadata

Warning in : no dictionary for class edm4hep::RecoParticleRefData is available Error in : cannot find tree with name podio_metadata in file /eos/experiment/fcc/ee/genera tion/DelphesEvents/spring2021/IDEA/p8_ee_ZH_ecm240/events_101027117.root terminate called after throwing an instance of 'std::runtime_error' what(): File /eos/experiment/fcc/ee/generation/DelphesEvents/spring2021/IDEA/p8_ee_ZH_ecm240/events_1010 27117.root couldn't be found Aborted

In https://github.com/key4hep/EDM4hep/pull/227 the ROOTLegacyReader is used if legacy file is detected and
following crash occures:

$ edm4hep2json /eos/experiment/fcc/ee/generation/DelphesEvents/spr ing2021/IDEA/p8_ee_ZH_ecm240/events_101027117.root -n 4 Warning in : no dictionary for class edm4hep::RecoParticleRefData is available WARNING: Reading legacy file, some collections might not be recognized! PODIO: Reconstructing CollectionTypeInfo branch from other sources in file: 'root://eospublic.cern.ch//eos/ experiment/fcc/ee/generation/DelphesEvents/spring2021/IDEA/p8_ee_ZH_ecm240/events_101027117.root'

Break segmentation violation

. . .

7 0x00007f1dbb0906b6 in TClass::NewObject (this=this

entry=0x0, defConstructor=TClass::kClassNew, quiet=false) at /tmp/root/spack-stage/spack-stage-root-6.28.04 -474lsx3d37cexm4vw5dk7twizjzkdeo4/spack-src/core/meta/src/TClass.cxx:4996

8 0x00007f1dbb093340 in TClass::New (this=0x0, defConstructor=, quiet=) at

/tmp/root/spack-stage/spack-stage-root-6.28.04-474lsx3d37cexm4vw5dk7twizjzkdeo4/spack-src/core/meta/src/TCl ass.cxx:4980

9 0x00007f1dbcf86142 in podio::ROOTLegacyReader::createCollectionBranches(std::vector<std::tuple<unsigned

int, std::cxx11::basic_string<char, std::char_traits, std::allocator >, bool, unsigned int>, std::allocator<std::tuple<unsigned int, std::cxx11::basic_string<char, std::char_traits, std::allo cator >, bool, unsigned int> > > const&) () from /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2023-09 -18/x86_64-centos7-gcc12.2.0-opt/podio/d402f747877239aaedefe993856e26749c91bea3=develop-ivgpec/lib64/libpod ioRootIO.so

10 0x00007f1dbcf874a0 in podio::ROOTLegacyReader::openFiles(std::vector<std::__cxx11::basic_string<char, s

td::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_tr aits, std::allocator > > > const&) () from /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2023-09 -18/x86_64-centos7-gcc12.2.0-opt/podio/d402f747877239aaedefe993856e26749c91bea3=develop-ivgpec/lib64/libpod ioRootIO.so

11 0x00007f1dbcf878ea in podio::ROOTLegacyReader::openFile(std::__cxx11::basic_string<char, std::char_trai

ts, std::allocator > const&) () from /cvmfs/sw-nightlies.hsf.org/key4hep/releases/2023-09-18/x8 6_64-centos7-gcc12.2.0-opt/podio/d402f747877239aaedefe993856e26749c91bea3=develop-ivgpec/lib64/libpodioRoot IO.so

tmadlener commented 1 year ago

Thanks for the thorough report, let me give a few quick answers.

For 1) and 2) The RecoParticleRef datatype has been removed in https://github.com/key4hep/EDM4hep/pull/116. Since the root file layout didn't change as far as FCCAnalyses was concerned, we decided to go ahead with that without waiting for a schema evolution mechanism. Additionally, podio-dump was not around at that point. Long story short, for these files podio-dump is currently broken, and I am not sure how easy it is to fix it. One possibility would be to build the EDM4hep version at the time with a current podio version and then thing might work again. Another possibility could be to patch the meta data of these files such that podio-dump can handle things again.

4 might be related to this, but the posted error simply looks like a potentially broken or non-present file.

The failure in edm4hep2json is most likely also related to this, as ROOT simply doesn't know about the RecoParticleRef class, so it cannot create an instance of this either.

3 looks like a TTree is missing a branch or a TTree is entirely missing. This would need some further debugging to understand what is really going on.

The main question for me to understand is which problem needs to be solved, i.e. what do you want to achieve. If I understand the AuxTools correctly, there the main point is to get to the collectionID table, right? Is that enough or would you need the additional functionality that podio-dump offers as well?

For edm4hep2json I suppose you want to be able to use the spring2021 files also for the event display?

kjvbrt commented 1 year ago

The motivation was to have an understanding what is broken where. I agree, that fixing all the issues is not feasible without nasty hacks, but why not fix the ones which can be done easily :)

Another possibility could be to patch the meta data of these files

How can this be done? Will this effectively disable those reference collections?

3 looks like a TTree is missing a branch or a TTree is entirely missing

I believe the problem here is the addition of dNdX to the edm4hep::Track.

If I understand the AuxTools correctly, there the main point is to get to the collectionID table, right?

We keep it around to be able to get this information in cases where podio-dump is broken. I don't want to add any further functionality to this script.

For edm4hep2json I suppose you want to be able to use the spring2021 files also for the event display?

That would be nice to have, but is not a priority.