groot: consider adding support for DAOD_PHYSLITE

[sbinet]

currently (v0.28.5) trying to run root-ls on a DAOD_PHYSLITE file leads to:

$> root-ls -t https://github.com/CoffeaTeam/coffea/raw/6d548538653e7003281a572f8eec5d68ca57b19f/tests/samples/DAOD_PHYSLITE_21.2.108.0.art.pool.root
=== [https://github.com/CoffeaTeam/coffea/raw/6d548538653e7003281a572f8eec5d68ca57b19f/tests/samples/DAOD_PHYSLITE_21.2.108.0.art.pool.root] ===
version: 60806
panic: rdict: could not build element "m_itemList" type for "EventStreamInfo_p3": rdict: could not create type for std::vector<T> T="pair<unsigned int": rdict: could not find streamer info for "pair<unsigned int" (version=-1): riofs: no streamer for "pair<unsigned int"

goroutine 1 [running]:
go-hep.org/x/hep/groot/rtree.(*tbranchElement).setStreamer(0xc000131b00, {0xd77068, 0xc0000ef260}, {0xd5cfa0, 0xc00025e3c0})
    /home/binet/tmp/go/src/go-hep.org/x/hep/groot/rtree/branch.go:1012 +0x1d6
go-hep.org/x/hep/groot/rtree.(*ttree).attachStreamer(0xc00025e3c0, {0xd7c880, 0xc000131b00}, {0xd77068, 0xc0000ef260}, {0xd5cfa0, 0xc00025e3c0})
    /home/binet/tmp/go/src/go-hep.org/x/hep/groot/rtree/tree.go:514 +0x2ec
go-hep.org/x/hep/groot/rtree.(*ttree).UnmarshalROOT(0xc000096600, 0xc00025e3c0)
    /home/binet/tmp/go/src/go-hep.org/x/hep/groot/rtree/tree.go:472 +0xf39
go-hep.org/x/hep/groot/riofs.(*Key).Object(0xc0002e3a20)
    /home/binet/tmp/go/src/go-hep.org/x/hep/groot/riofs/key.go:370 +0x245
go-hep.org/x/hep/groot/riofs.(*Key).Value(0xc0002e3a20)
    /home/binet/tmp/go/src/go-hep.org/x/hep/groot/riofs/key.go:336 +0x25
go-hep.org/x/hep/groot/rcmd.lsCmd.walk({{0xd5ca20, 0xc000265100}, 0x50, 0xfb}, {0xd5dd60, 0xc000114210}, {0xc00015c000, 0x4, 0x366a, 0x22ff6, ...})
    /home/binet/tmp/go/src/go-hep.org/x/hep/groot/rcmd/ls.go:97 +0x135
go-hep.org/x/hep/groot/rcmd.lsCmd.ls({{0xd5ca20, 0xc000265100}, 0x0, 0xd5}, {0x7fff4ed427b9, 0x86})
    /home/binet/tmp/go/src/go-hep.org/x/hep/groot/rcmd/ls.go:88 +0x40e
go-hep.org/x/hep/groot/rcmd.List({0xd5ca20, 0xc000265100}, {0x7fff4ed427b9, 0x86}, {0xc0001afec8, 0x2, 0x0})
    /home/binet/tmp/go/src/go-hep.org/x/hep/groot/rcmd/ls.go:59 +0xcd
main.run({0xd5dbc0, 0xc000010018}, {0xd5dbc0, 0xc000010020}, {0xc000020190, 0x2, 0x2})
    /home/binet/tmp/go/src/go-hep.org/x/hep/groot/cmd/root-ls/main.go:125 +0x66e
main.main()
    /home/binet/tmp/go/src/go-hep.org/x/hep/groot/cmd/root-ls/main.go:77 +0x6d

[sbinet]

support for DAOD_PHYSLITE is blocked by ATLAS not putting all the required streamers in the file (and relying on ROOT dictionaries being loaded on-the-fly):

• https://its.cern.ch/jira/browse/ATEAM-803

starting from 551dabd7, root-ls fails with:

$> root-ls -t https://github.com/CoffeaTeam/coffea/raw/6d548538653e7003281a572f8eec5d68ca57b19f/tests/samples/DAOD_PHYSLITE_21.2.108.0.art.pool.root
=== [https://github.com/CoffeaTeam/coffea/raw/6d548538653e7003281a572f8eec5d68ca57b19f/tests/samples/DAOD_PHYSLITE_21.2.108.0.art.pool.root] ===
version: 60806
  TTree                                 MetaData                            MetaData (entries=1)
    FileMetaDataAux.                    "FileMetaDataAux."                  TBranchElement
      FileMetaDataAux.xAOD::AuxInfoBase "FileMetaDataAux.xAOD::AuxInfoBase" TBranchElement

[...]
  TTree       ##Links       ##Links (entries=565)
    db_string "db_string/C" TBranch
  TTree       ##Links       ##Links (entries=565)
    db_string "db_string/C" TBranch
panic: rdict: could not build element "m_subEvents" type for "PileUpEventInfo_p5": rdict: could not create type for "vector<PileUpEventInfo_p5::SubEvent>": rdict: could not find streamer info for "PileUpEventInfo_p5::SubEvent" (version=-1): riofs: no streamer for "PileUpEventInfo_p5::SubEvent"

not much we can do about it. (FYI, uproot fails in a similar fashion:

>>> import uproot
>>> f = uproot.open("/data/xaod/r04.AOD.pool.root")
>>> t =f["CollectionTree"]
>>> t.arrays("EventInfo_p4")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/dist-packages/uproot/behaviors/TBranch.py", line 1125, in arrays
    _ranges_or_baskets_to_arrays(
  File "/usr/local/lib/python3.8/dist-packages/uproot/behaviors/TBranch.py", line 3502, in _ranges_or_baskets_to_arrays
    uproot.source.futures.delayed_raise(*obj)
  File "/usr/local/lib/python3.8/dist-packages/uproot/source/futures.py", line 36, in delayed_raise
    raise exception_value.with_traceback(traceback)
  File "/usr/local/lib/python3.8/dist-packages/uproot/behaviors/TBranch.py", line 3446, in basket_to_array
    basket_arrays[basket.basket_num] = interpretation.basket_array(
  File "/usr/local/lib/python3.8/dist-packages/uproot/interpretation/objects.py", line 140, in basket_array
    form = self.awkward_form(branch.file, index_format="i64")
  File "/usr/local/lib/python3.8/dist-packages/uproot/interpretation/objects.py", line 116, in awkward_form
    return self._model.awkward_form(
  File "/usr/local/lib/python3.8/dist-packages/uproot/model.py", line 1136, in awkward_form
    return versioned_cls.awkward_form(
  File "<dynamic>", line 49, in awkward_form
  File "<dynamic>", line 63, in awkward_form
  File "/usr/local/lib/python3.8/dist-packages/uproot/containers.py", line 780, in awkward_form
    uproot._util.awkward_form(
  File "/usr/local/lib/python3.8/dist-packages/uproot/_util.py", line 544, in awkward_form
    return model.awkward_form(
  File "/usr/local/lib/python3.8/dist-packages/uproot/model.py", line 660, in awkward_form
    raise uproot.interpretation.objects.CannotBeAwkward(
uproot.interpretation.objects.CannotBeAwkward: TriggerInfo_p2::StreamTag_p2

[bburghgr]

I haven't had time to test it myself, but supposedly uproot is working with the open-data DAOD_PHYSLITE files:

• https://github.com/atlas-outreach-data-tools/notebooks-collection-opendata/blob/master/for-research/physlite_tutorial.ipynb

I don't know if that's because the streamers were added, or if uproot implemented a workaround of some kind (e.g. just skip anything with a missing streamer, as if it wasn't in the file). Regardless, the jira issue you linked has recently shown some signs of life, so this may be worth another look at some point.

[sbinet]

hello stranger,

(thanks for your interest in groot)

the JIRA issue is pointing at work done by Scott to speed-up my old diff-root PyROOT script. if uproot now works, I guess that's thanks to another strand of work (either on the uproot side or on the ATLAS side).

[bburghgr]

Yes, I see. Sorry for the noise, I was just curious if this situation had improved at all due to e.g. some improvement on the ATLAS side.

Digging a bit more, it looks like uproot is creating a class based on uproot.model.UnknownClass for types that are missing a streamer. So it probably works, but only sort-of... things don't crash, and I can still access primitive types from the Aux container (if I really must), but the UnknownClass types do not seem to be useful for much. That's unfortunate, I had hoped that maybe uproot was working because the missing streamers had been added to the PHYSLITE derivations (in which case, groot support for the PHYSLITE may be easier), but sadly it seems this is not the case.

[bburghgr]

Just making a note that there's (finally) some activity on the ATLAS side to address the issues with missing streamers.