afernez
commented
2 months ago I just committed a few scripts that I used to do the re-saving (to fix file compression issue) and to check the contents of the ntuples. First, regarding the re-saving: this wrapper and this macro can be used to re-copy all the new PIDCalib ntuples -- from the ones stored in /home/public/pidcalib_ntuples/remote on glacier to a new remote-ubdt-only folder. Doing this for one file for each particle type/year, I see that the copied ntuples can be opened using ROOT5:
(.virtualenv) alex@physwkpsc3114ub:~/MuonBDTPid$ root /home/public/pidcalib_ntuples/remote-ubdt-only/KPiMu-2017-MagDown/00227913_00000065_1.pidcalib.root
*** DISPLAY not set, setting it to 10.206.36.79:0.0
*******************************************
* *
* W E L C O M E to R O O T *
* *
* Version 5.34/38 12 March 2018 *
* *
* You are welcome to visit our Web site *
* http://root.cern.ch *
* *
*******************************************
ROOT 5.34/38 (heads/v5-34-00-patches@v5-34-36-235-gba5b2a7, Jan 01 1980, 00:00:00 on linuxx8664gcc)
CINT/ROOT C/C++ Interpreter version 5.18.00, July 2, 2010
Type ? for help. Commands must be C++ statements.
Enclose multiple statements between { }.
root [0]
Attaching file /home/public/pidcalib_ntuples/remote-ubdt-only/KPiMu-2017-MagDown/00227913_00000065_1.pidcalib.root as _file0...
Warning in <TClass::TClass>: no dictionary for class ROOT::TIOFeatures is available
root [1] _file0->ls()
TFile** /home/public/pidcalib_ntuples/remote-ubdt-only/KPiMu-2017-MagDown/00227913_00000065_1.pidcalib.root
TFile* /home/public/pidcalib_ntuples/remote-ubdt-only/KPiMu-2017-MagDown/00227913_00000065_1.pidcalib.root
KEY: TDirectoryFile KSLL_PiPTuple;1 KSLL_PiPTuple
KEY: TDirectoryFile KSLL_PiMTuple;1 KSLL_PiMTuple
KEY: TDirectoryFile DSt_PiPTuple;1 DSt_PiPTuple
KEY: TDirectoryFile DSt_KMTuple;1 DSt_KMTuple
KEY: TDirectoryFile DsPhi_KPTuple;1 DsPhi_KPTuple
KEY: TDirectoryFile DSt_PiMTuple;1 DSt_PiMTuple
KEY: TDirectoryFile DSt_KPTuple;1 DSt_KPTuple
KEY: TDirectoryFile Jpsi_MuPTuple;1 Jpsi_MuPTuple
KEY: TDirectoryFile Jpsi_MuMTuple;1 Jpsi_MuMTuple
KEY: TDirectoryFile DsPhi_KMTuple;1 DsPhi_KMTuple
KEY: TDirectoryFile B_Jpsi_MuMTuple;1 B_Jpsi_MuMTuple
KEY: TDirectoryFile B_Jpsi_DTF_MuMTuple;1 B_Jpsi_DTF_MuMTuple
KEY: TDirectoryFile B_Jpsi_MuPTuple;1 B_Jpsi_MuPTuple
KEY: TDirectoryFile B_Jpsi_DTF_MuPTuple;1 B_Jpsi_DTF_MuPTuple
KEY: TDirectoryFile DsPhi_MuPTuple;1 DsPhi_MuPTuple
KEY: TDirectoryFile DsPhi_MuMTuple;1 DsPhi_MuMTuple
root [2] TTree* t = (TTree*)_file0->Get("Jpsi_MuPTuple/DecayTree")
root [3] t->Print()
******************************************************************************
*Tree :DecayTree : DecayTree *
*Entries : 101237 : Total = 573541942 bytes File Size = 254599839 *
* : : Tree compression factor = 2.25 *
******************************************************************************
*Br 0 :Jpsi_0.50_cc_mult : Jpsi_0.50_cc_mult/I *
*Entries : 101237 : Total Size= 406925 bytes File Size = 83153 *
*Baskets : 15 : Basket Size= 28672 bytes Compression= 4.89 *
...But there's still an error when running python scripts/apply_ubdt.py --ymlName spec/pidcalib17.yml (tmp yml file created that points to remote-ubdt-only instead of remote, to 2017 instead of 2016, and to a tmp friends-alex folder -- because I can't write in Emily's friends folder):
/home/public/pidcalib_ntuples/remote-ubdt-only/Mu_nopt-2017-MagDown/ -> /home/public/pidcalib_ntuples/friends-alex/Mu_nopt-2017-MagDown/
trees: Jpsinopt_MuMTuple/DecayTree;23,Jpsinopt_MuMTuple/DecayTree;22,Jpsinopt_MuPTuple/DecayTree;23,Jpsinopt_MuPTuple/DecayTree;22
AddUBDTBranchPidCalib -i /home/public/pidcalib_ntuples/remote-ubdt-only/Mu_nopt-2017-MagDown/00227899_00000021_1.pidcalib.root -o /home/public/pidcalib_ntuples/friends-alex/Mu_nopt-2017-MagDown/00227899_00000021_1.pidcalib.root -p probe -b UBDT -t Jpsinopt_MuMTuple/DecayTree;23,Jpsinopt_MuMTuple/DecayTree;22,Jpsinopt_MuPTuple/DecayTree;23,Jpsinopt_MuPTuple/DecayTree;22
Warning in <TClass::TClass>: no dictionary for class ROOT::TIOFeatures is available
Working on Jpsinopt_MuMTuple/DecayTree
Done loading input data
Jpsinopt_MuMTuple/DecayTree has 364131 entries
Processing ............................................................ [100%]
*** Break *** segmentation violation
===========================================================
There was a crash.
This is the entire stack trace of all threads:
===========================================================
#0 0x00007ff8936e7e56 in wait4 () from /nix/store/adxc893j47gxx3xjw403zdf0liiddvw2-glibc-2.32-48/lib/libc.so.6
#1 0x00007ff893667447 in do_system () from /nix/store/adxc893j47gxx3xjw403zdf0liiddvw2-glibc-2.32-48/lib/libc.so.6
#2 0x00007ff8965b62cb in TUnixSystem::StackTrace() () from /nix/store/i89sm6q0hh7y9iaaqf69afxrynrqmn7p-root-5.34.38/lib/libCore.so
#3 0x00007ff8965b84d4 in TUnixSystem::DispatchSignals(ESignals) () from /nix/store/i89sm6q0hh7y9iaaqf69afxrynrqmn7p-root-5.34.38/lib/libCore.so
#4 <signal handler called>
#5 0x00007ff8957a2043 in TDirectoryFile::Save() () from /nix/store/i89sm6q0hh7y9iaaqf69afxrynrqmn7p-root-5.34.38/lib/libRIO.so
#6 0x00007ff8957a20d4 in TDirectoryFile::Save() () from /nix/store/i89sm6q0hh7y9iaaqf69afxrynrqmn7p-root-5.34.38/lib/libRIO.so
#7 0x00007ff8957a0229 in TDirectoryFile::Close(char const*) () from /nix/store/i89sm6q0hh7y9iaaqf69afxrynrqmn7p-root-5.34.38/lib/libRIO.so
#8 0x00007ff8957b7b3a in TFile::Close(char const*) () from /nix/store/i89sm6q0hh7y9iaaqf69afxrynrqmn7p-root-5.34.38/lib/libRIO.so
#9 0x000000000041c53b in main ()
===========================================================
The lines below might hint at the cause of the crash.
If they do not help you then please submit a bug report at
http://root.cern.ch/bugs. Please post the ENTIRE stack trace
from above as an attachment in addition to anything else
that might help us fixing this issue.
===========================================================
#5 0x00007ff8957a2043 in TDirectoryFile::Save() () from /nix/store/i89sm6q0hh7y9iaaqf69afxrynrqmn7p-root-5.34.38/lib/libRIO.so
#6 0x00007ff8957a20d4 in TDirectoryFile::Save() () from /nix/store/i89sm6q0hh7y9iaaqf69afxrynrqmn7p-root-5.34.38/lib/libRIO.so
#7 0x00007ff8957a0229 in TDirectoryFile::Close(char const*) () from /nix/store/i89sm6q0hh7y9iaaqf69afxrynrqmn7p-root-5.34.38/lib/libRIO.so
#8 0x00007ff8957b7b3a in TFile::Close(char const*) () from /nix/store/i89sm6q0hh7y9iaaqf69afxrynrqmn7p-root-5.34.38/lib/libRIO.so
#9 0x000000000041c53b in main ()
===========================================================
Segmentation fault (core dumped)
sh: 1: 23,Jpsinopt_MuMTuple/DecayTree: not found
sh: 1: 22,Jpsinopt_MuPTuple/DecayTree: not found
sh: 1: 23,Jpsinopt_MuPTuple/DecayTree: not found
sh: 1: 22: not found
WARNING: AddUBDTBranchPidCalib -i /home/public/pidcalib_ntuples/remote-ubdt-only/Mu_nopt-2017-MagDown/00227899_00000021_1.pidcalib.root -o /home/public/pidcalib_ntuples/friends-alex/Mu_nopt-2017-MagDown/00227899_00000021_1.pidcalib.root -p probe -b UBDT -t Jpsinopt_MuMTuple/DecayTree;23,Jpsinopt_MuMTuple/DecayTree;22,Jpsinopt_MuPTuple/DecayTree;23,Jpsinopt_MuPTuple/DecayTree;22 did not execute properly!
...
manuelfs
CoffeeIntoScience
lmeyergarcia
The ntuples with the added
ANNPIDTrainingbranches needed to calculate Greg's $\mu$ BDT have been produced, but Emily and I have been having trouble generating the actual $\mu$ BDT branch using the code in this repo as was done for 2016. Emily copied the files from lxplus (with a handful of copying errors that she fixed), and I've done some checks on what's been copied over and for now taken over trying to get the $\mu$ BDT to work. Our conclusion seemed to be that there was a file incompatibility due to some different compression, with the ntuples assumedly produced using a new ROOT version, while we are forced to use a modified ROOT 5 with Greg'suBoostimplementation (originaluBoostpaper, just for completeness). However, re-saving the files so that they can be opened using ROOT5, the same workflow as used for 2016 still leads to errors for 2017/2018.