search

JuliaHEP / UnROOT.jl

Native Julia I/O package to work with CERN ROOT files objects (TTree and RNTuple)
https://juliahep.github.io/UnROOT.jl/
MIT License
96 stars 17 forks source link

TDirectory not supported #11

Closed Moelf closed 3 years ago

Moelf commented 4 years ago

I've read more source code now and I understand that currently what "struct" is supported depends on manual type-up in the types.jl.

So, no rush to fix this particular "issue", but just a reminder we should cover this eventually.

tamasgal commented 4 years ago

Yep, thanks for the issue and the reminder! That’s the best way to keep track of all the tasks.

mmikhasenko commented 3 years ago

Not sure if it is the same issue, but I am getting

ERROR: UndefVarError: TDirectory not defined error when trying to access a folder in the root file

tamasgal commented 3 years ago

Ah yes indeed, the TDirectory is still experimental (not implemented 🙈 ) I really hope to find some time the next weeks to work on UnROOT. I now also have some ROOT files from another experiment which has TDirectorys and NTuples, so chances are that I have to spend time 😆

mmikhasenko commented 3 years ago

Oh,

on the README page, you have an example

julia> array(f, "KM3NET_EVENT/KM3NET_EVENT/triggeredHits"; raw=true)

aren't KM3NET_EVENT a directory?

mmikhasenko commented 3 years ago

thanks for the quick reply

tamasgal commented 3 years ago 
```julia
julia> array(f, "KM3NET_EVENT/KM3NET_EVENT/triggeredHits"; raw=true)

aren't KM3NET_EVENT a directory?

No, the first level is a TTree and the second level is a TBranchElement (see below). I'll try to figure out the TDirectory, it should not be that hard...

julia> using UnROOT

julia> f = ROOTFile("test/samples/km3net_online.root")
ROOTFile("test/samples/km3net_online.root") with 10 entries and 54 streamers.

julia> f["KM3NET_EVENT"]
UnROOT.TTree
  fName: String "KM3NET_EVENT"
  fTitle: String "km3net_event"
  fLineColor: Int16 1
  fLineStyle: Int16 1
  fLineWidth: Int16 1
  fFillColor: Int16 19
  fFillStyle: Int16 1001
  fMarkerColor: Int16 1
  fMarkerStyle: Int16 1
  fMarkerSize: Float32 1.0f0
  fEntries: Int64 3
  fTotBytes: Int64 5666
  fZipBytes: Int64 5666
  fSavedBytes: Int64 0
  fFlushedBytes: Int64 0
  fWeight: Float64 1.0
  fTimerInterval: Int32 0
  fScanField: Int32 25
  fUpdate: Int32 0
  fDefaultEntryOffsetLen: Int32 1000
  fNClusterRange: UInt32 0x00000000
  fMaxEntries: Int64 1000000000000
  fMaxEntryLoop: Int64 1000000000000
  fMaxVirtualSize: Int64 0
  fAutoSave: Int64 -300000000
  fAutoFlush: Int64 -30000000
  fEstimate: Int64 1000000
  fClusterRangeEnd: Array{Int64}((0,)) Int64[]
  fClusterSize: Array{Int64}((0,)) Int64[]
  fIOFeatures: UnROOT.ROOT_3a3a_TIOFeatures
  fBranches: UnROOT.TObjArray
  fLeaves: UnROOT.TObjArray
  fAliases: Missing missing
  fIndexValues: Array{Any}((0,))
  fIndex: Array{Any}((0,))
  fTreeIndex: Missing missing
  fFriends: Missing missing

julia> f["KM3NET_EVENT/KM3NET_EVENT"]
UnROOT.TBranchElement_10
  cursor: UnROOT.Cursor
  fName: String "KM3NET_EVENT"
  fTitle: String "KM3NET_EVENT"
  fFillColor: Int16 0
  fFillStyle: Int16 1001
  fCompress: Int32 0
  fBasketSize: Int32 65536
  fEntryOffsetLen: Int32 1000
  fWriteBasket: Int32 0
  fEntryNumber: Int64 0
  fIOFeatures: UnROOT.ROOT_3a3a_TIOFeatures
  fOffset: Int32 0
  fMaxBaskets: UInt32 0x0000000a
  fSplitLevel: Int32 1
  fEntries: Int64 3
  fFirstEntry: Int64 0
  fTotBytes: Int64 0
  fZipBytes: Int64 0
  fBranches: UnROOT.TObjArray
  fLeaves: UnROOT.TObjArray
  fBaskets: UnROOT.TObjArray
  fBasketBytes: Array{Int64}((10,)) [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
  fBasketEntry: Array{Int64}((10,)) [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
  fBasketSeek: Array{Int64}((10,)) [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
  fFileName: String ""
  fClassName: String "KM3NETDAQ::JDAQEvent"
  fParentName: String ""
  fClonesName: String ""
  fCheckSum: UInt32 0xe59f689d
  fClassVersion: Int16 4
  fID: Int32 -2
  fType: Int32 0
  fStreamerType: Int32 -1
  fMaximum: Int32 0
  fBranchCount: Missing missing
  fBranchCount2: Missing missing
mmikhasenko commented 3 years ago

alright, thanks

mmikhasenko commented 3 years ago

Just some encouragement (sorry, not PR): most of our files (LHCb tuples) have directories inside. For long time already, I want to move to UnROOT, this seems to be the last obstacle ;)

tamasgal commented 3 years ago

Alright, that's a good motivation! 😆

tamasgal commented 3 years ago

@mmikhasenko do you have a small sample file for us?

mmikhasenko commented 3 years ago

Can check now. Should TDirectory work already?

mmikhasenko commented 3 years ago

Here is a simple tree with Np x 4 number of branches, stored in a directory. By changing Nev, you can adjust the size of the file.

#include "TString.h"
#include "TFile.h"
#include "TTree.h"

#include <string>
#include "stdlib.h"

int create_a_tree() {

  TFile *f = new TFile("test_tree.root", "recreate");
  f->mkdir("Data");
  f->cd("Data");  
  TTree *t = new TTree("mytree", "mytree");

  auto app = "EXYZ";
  const uint Np = 10;

  //
  double v[Np][4];
  for (uint n=0; n<Np; n++) {
    for (uint i=0; i<4; i++) {
      t->Branch(TString::Format("Particle%d_%c", n, app[i]), &v[n][i]);
    }
  }

  const int Nev = 30000;

  for (uint i=0; i <= Nev; i++) {
    for (uint n=0; n<Np; n++)
      for (uint i=0; i<4; i++)
    v[n][i] = (rand() % 100) / 100.0;
    //
    t->Fill();
  }

  t->Write();
  f->Close();

  return 0;
}
mmikhasenko commented 3 years ago

Just checked on the latest release and master

@v1.6 > using UnROOT
@v1.6 > f = ROOTFile(joinpath("Downloads\\test_tree.root")) 
@v1.6 > f["Data"]

throws

UndefVarError: TDirectory not defined
Moelf commented 3 years ago

Tamas is just asking for a sample test file so he can work on it maybe. nothing has changed recently in this direction

mmikhasenko commented 3 years ago

Ok. is what I posted sufficient?

tamasgal commented 3 years ago

Thanks, that should be enough!

@all-contributors please add @mmikhasenko for data

allcontributors[bot] commented 3 years ago

@tamasgal

I've put up a pull request to add @mmikhasenko! :tada:

tamasgal commented 3 years ago

Here is an excellent document describing the TDirectory: https://github.com/root-project/root/blob/master/io/doc/TFile/tdirectory.md

aminnj commented 3 years ago

That output reminds me of this useful method in ROOT:

root [0] TDirectory::Class()->GetStreamerInfo()->ls()

StreamerInfo for class: TDirectory, version=5, checksum=0x1e9b6f70
  TNamed         BASE            offset=  0 type=67 The basis for a named object (name, title)
  TObject*       fMother         offset= 64 type=64 pointer to mother of the directory
  TList*         fList           offset= 72 type=64 List of objects in memory
  TUUID          fUUID           offset= 80 type=62 Unique identifier
   i= 0, TNamed          type= 67, offset=  0, len=1, method=0
   i= 1, fMother         type= 64, offset= 64, len=1, method=0
   i= 2, fList           type= 64, offset= 72, len=1, method=0
   i= 3, fUUID           type= 62, offset= 80, len=1, method=0
root [1] TH1::Class()->GetStreamerInfo()->ls()

StreamerInfo for class: TH1, version=8, checksum=0x1c3740c4
  TNamed         BASE            offset=  0 type=67 The basis for a named object (name, title)
  TAttLine       BASE            offset= 64 type= 0 Line attributes
  TAttFill       BASE            offset= 80 type= 0 Fill area attributes
  TAttMarker     BASE            offset= 96 type= 0 Marker attributes
  int            fNcells         offset=112 type= 3 number of bins(1D), cells (2D) +U/Overflows
  TAxis          fXaxis          offset=120 type=61 X axis descriptor
  TAxis          fYaxis          offset=336 type=61 Y axis descriptor
  TAxis          fZaxis          offset=552 type=61 Z axis descriptor
  short          fBarOffset      offset=768 type= 2 (1000*offset) for bar charts or legos
  short          fBarWidth       offset=770 type= 2 (1000*width) for bar charts or legos
  double         fEntries        offset=776 type= 8 Number of entries
  double         fTsumw          offset=784 type= 8 Total Sum of weights
  double         fTsumw2         offset=792 type= 8 Total Sum of squares of weights
  double         fTsumwx         offset=800 type= 8 Total Sum of weight*X
  double         fTsumwx2        offset=808 type= 8 Total Sum of weight*X*X
  double         fMaximum        offset=816 type= 8 Maximum value for plotting
  double         fMinimum        offset=824 type= 8 Minimum value for plotting
  double         fNormFactor     offset=832 type= 8 Normalization factor
  TArrayD        fContour        offset=840 type=62 Array to display contour levels
  TArrayD        fSumw2          offset=864 type=62 Array of sum of squares of weights
  TString        fOption         offset=888 type=65 histogram options
  TList*         fFunctions      offset=912 type=63 ->Pointer to list of functions (fits and user)
  int            fBufferSize     offset=920 type= 6 fBuffer size
  double*        fBuffer         offset=928 type=48 [fBufferSize] entry buffer
  TH1::EBinErrorOpt fBinStatErrOpt  offset=968 type= 3 option for bin statistical errors
  TH1::EStatOverflows fStatOverflows  offset=972 type= 3 per object flag to use under/overflows in statistics
   i= 0, TNamed          type= 67, offset=  0, len=1, method=0
   i= 1, TAttLine        type=  0, offset= 64, len=1, method=0
   i= 2, TAttFill        type=  0, offset= 80, len=1, method=0
   i= 3, TAttMarker      type=  0, offset= 96, len=1, method=0
   i= 4, fNcells         type=  3, offset=112, len=1, method=0
   i= 5, fXaxis          type= 61, offset=120, len=1, method=0
   i= 6, fYaxis          type= 61, offset=336, len=1, method=0
   i= 7, fZaxis          type= 61, offset=552, len=1, method=0
   i= 8, fBarOffset      type= 22, offset=768, len=2, method=0 [optimized]
   i= 9, fEntries        type= 28, offset=776, len=8, method=0 [optimized]
   i=10, fContour        type= 62, offset=840, len=1, method=0
   i=11, fSumw2          type= 62, offset=864, len=1, method=0
   i=12, fOption         type= 65, offset=888, len=1, method=0
   i=13, fFunctions      type= 63, offset=912, len=1, method=0
   i=14, fBufferSize     type=  6, offset=920, len=1, method=140375097856088
   i=15, fBuffer         type= 48, offset=928, len=1, method=920
   i=16, fBinStatErrOpt  type= 23, offset=968, len=2, method=0 [optimized]

for anything inheriting from TObject. Maybe it's already common knowledge 🤷 but I figured I'd write this somewhere for posterity.

tamasgal commented 3 years ago

Added in https://github.com/tamasgal/UnROOT.jl/pull/107 thanks to @aminnj

@mmikhasenko please have a look whenever you have time :)