Failure to write std::vector<std::array<T, N> > into a tree/branch

[krasznaa]

• [x] Checked for duplicates

Describe the bug

While updating ATLAS's EDM, we tried to introduce some std::vector<std::array<float, 3> > variables into our output files. But was met with the following types of errors:

RAWtoALL 12:36:50 Error in <TBranchElement::InitializeOffsets>: Could not find the real data member '_M_elems[3]' when constructing the branch 'ITkStripOverlapSpacePointsAuxDyn.topStripDirection' [Likely missing ShowMember].
RAWtoALL 12:36:50 Error in <TBranchElement::InitializeOffsets>: Could not find the real data member '_M_elems[3]' when constructing the branch 'ITkStripOverlapSpacePointsAuxDyn.bottomStripDirection' [Likely missing ShowMember].
RAWtoALL 12:36:50 Error in <TBranchElement::InitializeOffsets>: Could not find the real data member '_M_elems[3]' when constructing the branch 'ITkStripOverlapSpacePointsAuxDyn.stripCenterDistance' [Likely missing ShowMember].
RAWtoALL 12:36:50 Error in <TBranchElement::InitializeOffsets>: Could not find the real data member '_M_elems[3]' when constructing the branch 'ITkStripOverlapSpacePointsAuxDyn.topStripCenter' [Likely missing ShowMember].
RAWtoALL 12:36:50 Error in <TBranchElement::InitializeOffsets>: Could not find the real data member '_M_elems[3]' when constructing the branch 'ITkStripSpacePointsAuxDyn.topStripDirection' [Likely missing ShowMember].
RAWtoALL 12:36:50 Error in <TBranchElement::InitializeOffsets>: Could not find the real data member '_M_elems[3]' when constructing the branch 'ITkStripSpacePointsAuxDyn.bottomStripDirection' [Likely missing ShowMember].
RAWtoALL 12:36:50 Error in <TBranchElement::InitializeOffsets>: Could not find the real data member '_M_elems[3]' when constructing the branch 'ITkStripSpacePointsAuxDyn.stripCenterDistance' [Likely missing ShowMember].
RAWtoALL 12:36:50 Error in <TBranchElement::InitializeOffsets>: Could not find the real data member '_M_elems[3]' when constructing the branch 'ITkStripSpacePointsAuxDyn.topStripCenter' [Likely missing ShowMember].

You can find the details in https://gitlab.cern.ch/atlas/athena/-/merge_requests/59648, but the problem is luckily fairly easy to reproduce in a completely "standalone" way as well.

Expected behavior

We would like to be able to write and read std::vector<std::array<T, N> > variables (and ideally even more complicated combinations of std::vector and std::array types) using TTree branches.

To Reproduce

I wrote the following small example:

• Makefile:

# Set up all variables for the implicit GNU Make rules to work.
CXX:=$(shell root-config --cxx)
CXXFLAGS:=$(shell root-config --cflags)
LD:=$(shell root-config --ld)
LDLIBS:=$(shell root-config --libs)
LINK.o = $(LINK.cc)

# Set up all explicit targets.
all: arrayWrite

arrayWrite: arrayWrite.o Dictionary.o

Dictionary.cpp: LinkDef.h
    rootcint -f $@ LinkDef.h

clean:
    rm -f Dictionary*
    rm -f *.o
    rm -f arrayWrite

distclean: clean
    rm -f *~
    rm -f arrays.root

• LinkDef.h:

#ifndef ARRAYWRITE_LINKDEF_H
#define ARRAYWRITE_LINKDEF_H

#include <array>
#include <vector>

struct INSTANTIATE_TYPES {
  std::array<float, 3> arr1;
  std::vector<std::array<float, 3> > vec1;
};

#pragma link C++ class std::vector<std::array<float, 3> >+;

#endif // ARRAYWRITE_LINKDEF_H

• arrayWrite.cpp:

// ROOT include(s).
#include <TFile.h>
#include <TError.h>
#include <TTree.h>

// System include(s).
#include <array>
#include <vector>
#include <memory>

/// Helper macro for creating a branch with error checking
#define MAKE_BRANCH(NAME, OBJ)                  \
  do {                              \
    TBranch* br = otree->Branch(NAME, &OBJ);            \
    if (br == nullptr ) {                   \
      Error(APPNAME, "Could not create branch \"%s\"", NAME);   \
      return 1;                         \
    }                               \
  } while(false)

int main() {

  // The name of the application.
  static const char* APPNAME = "arrayWrite";

  // Open the output file.
  static const char* FILENAME = "arrays.root";
  std::unique_ptr<TFile> ofile(TFile::Open(FILENAME, "RECREATE"));
  if ((!ofile) || ofile->IsZombie()) {
    Error(APPNAME, "Could not open output file \"%s\"", FILENAME);
    return 1;
  }
  Info(APPNAME, "Opened output file \"%s\"", FILENAME);

  {
    // Create a TTree in the output file.
    auto otree = std::make_unique<TTree>("arrays", "Tree with arrays");
    otree->SetDirectory(ofile.get());

    // Add vectors of arrays to it.
    std::array<float, 3> arr1;
    MAKE_BRANCH("arr1", arr1);
    std::vector<std::array<float, 3> > vec1;
    MAKE_BRANCH("vec1", vec1);

    // Fill a few entries into the tree.
    for (Int_t i = 0; i < 10; ++i) {
      vec1.clear();
      arr1 = {static_cast<float>(i),
              static_cast<float>(i + 1),
              static_cast<float>(i + 2)};
      for (Int_t j = 0; j < i; ++j) {
        vec1.push_back({static_cast<float>(j),
                        static_cast<float>(j + 1),
                        static_cast<float>(j + 2)});
      }
      if (otree->Fill() <= 0) {
        Error(APPNAME, "Failed to record entry %i", i);
        return 1;
      }
    }

    // Make sure that the tree is saved.
    otree->AutoSave();
  }
  Info(APPNAME, "Finished writing \"%s\"", FILENAME);

  // Return gracefully.
  return 0;
}

Building and running this example gets me:

[bash][celeborn]:arrayWrite > make
c++ -pthread -std=c++17 -m64 -I/home/krasznaa/software/root/6.26.06/x86_64-ubuntu2204-gcc11-opt/include   -c -o arrayWrite.o arrayWrite.cpp
rootcint -f Dictionary.cpp LinkDef.h
Warning in cling::IncrementalParser::CheckABICompatibility():
  Possible C++ standard library mismatch, compiled with __GLIBCXX__ '20220324'
  Extraction of runtime standard library version was: '20220421'
c++ -pthread -std=c++17 -m64 -I/home/krasznaa/software/root/6.26.06/x86_64-ubuntu2204-gcc11-opt/include   -c -o Dictionary.o Dictionary.cpp
c++ -pthread -std=c++17 -m64 -I/home/krasznaa/software/root/6.26.06/x86_64-ubuntu2204-gcc11-opt/include    arrayWrite.o Dictionary.o  -L/home/krasznaa/software/root/6.26.06/x86_64-ubuntu2204-gcc11-opt/lib -lCore -lImt -lRIO -lNet -lHist -lGraf -lGraf3d -lGpad -lROOTVecOps -lTree -lTreePlayer -lRint -lPostscript -lMatrix -lPhysics -lMathCore -lThread -lMultiProc -lROOTDataFrame -Wl,-rpath,/home/krasznaa/software/root/6.26.06/x86_64-ubuntu2204-gcc11-opt/lib -pthread -lm -ldl -rdynamic -o arrayWrite
[bash][celeborn]:arrayWrite > ./arrayWrite 
Info in <arrayWrite>: Opened output file "arrays.root"
Error in <TBranchElement::InitializeOffsets>: Could not find the real data member '_M_elems[3]' when constructing the branch 'vec1' [Likely missing ShowMember].
Info in <arrayWrite>: Finished writing "arrays.root"
[bash][celeborn]:arrayWrite >

Setup

Tried it with ROOT 6.26/06 and 6.24/06. With both the "LCG version" on x86_64-centos7-gcc11-opt and with hand-built versions on Ubuntu 22.04.

Additional context

N/A

[pcanal]

Hi Attila,

This is a know limitation. The work-around is to wrap the std::array by another class (for which you need to declare a dictionary).

How urgently would you need the proper implementation?

Cheers, Philippe.

[krasznaa]

Hi Philippe,

I don't think we have a super strict timeline on this.

We would prefer to use STL types in such branches as much as possible. In this case std::array would be preferred because std::vector<std::array<T, N> > is nicely contiguous in memory. Which we would really like to have for using accelerators efficiently with these new data types. (Since such an object would map super easily into an std::mdspan.)

We'll think internally a bit whether we'd go for std::vector<std::vector<T> > or std::vector<ATLASType> for now...

Cheers, Attila

[fwyzard]

You can mark this as "very nice to have" also for CMS :-)

[fwyzard]

CMS is finally getting its own edm::StdArray (https://github.com/cms-sw/cmssw/pull/43076) to work around this issue :-)