search

SuperNEMO-DBD / SNRawDataProducts

C/C++ Raw Data Structures and I/O for SuperNEMO
GNU General Public License v3.0
0 stars 1 forks source link

readme

SNRawDataProducts

Prototype C/C++ Raw Data Model with Boost/Brio/ROOT I/O for SuperNEMO Commissioning and Production.

Modularized from SNFrontEndElectronics project to study the requirements outlined by Analysis Board and tracker commissioning tasks:

This prototype demonstrates that it should be possible to separate the Data Model and I/O out of SNFrontEndElectronics into a lightweight Data/I/O only interface. Both SNFrontEndElectronics and Falaise could link to this library without needing to know any details about each other. Prototype converters are implemented to demonstrate conversion from raw binary data to a ROOT TTree holding the Raw Data Model C++ classes directly (for fast interactive analysis), and preliminary conversion to the BRIO (Storage of Boost binary data in ROOT) format currently used for input to the "Falaise" data processing system.

Quickstart

We assume you have a standard install of the SuperNEMO software. To get the code and build the library and applications:

$ brew snemo-shell
...
snemo-shell> git clone https://github.com/drbenmorgan/SNRawDataProducts.git
snemo-shell> cd SNRawDataProducts
snemo-shell> mkdir build && cd build
snemo-shell> cmake ..
...
snemo-shell> make
...

If you encounter errors at the cmake or make steps, raise an Issue. On a successful build, you will have the following library and applications in the directory where you ran cmake/make:

As this project is experimental, it should not be installed, but all programs, plus interactive ROOT usage, can be run from the directory holding the above programs.

Using RTD Files for Commissioning Analysis/Production Processing

The top level "Offline" Data Model class is RRawTriggerData. Each instance in any of the RTD files represents all data recorded by the DAQ in a single Trigger. At present, the RRawTriggerData class is composed of:

See the links above for details of these classes (the code is the documentation at this point in time). Given an RTD file in the current low level binary .data.gz format, these may be read and RRawTriggerData objects obtained in the following ways:

  1. Convert an RTD file to BRIO and access in flreconstruct

    snemo-shell> ./rtd2brio -i /sps/nemo/snemo/snemo_data/raw_data/RTD/run_102/snemo_run-102_rtd.data.gz -o run_102.brio
snemo-shell> flreconstruct -i run_102.brio
... fails for now ...

    The failure is due to a lack of Metadata, so need work and discussion to find out where this will come from. Note that BRIO files are just ROOT files holding two TTree "stores". They are not directly readable however, as the branches of these TTrees hold opaque buffers of Boost.Serialization binary data.

  2. Convert an RTD file to a properly structured ROOT TFile/TTree and access in root

    snemo-shell> ./rtd2asroot -i /sps/nemo/snemo/snemo_data/raw_data/RTD/run_102/snemo_run-102_rtd.data.gz -o run_102.root
...
snemo-shell> root setupSNemoRawDataObj
... will see Cling errors here, but can be ignored for now...
root [0] TFile f("run_102.root")
root [1] f.ls()
TFile**      run_102.root
TFile*       run_102.root
KEY: TTree   RawTriggerData;13   SuperNEMO Raw Trigger Data
KEY: TTree   RawTriggerData;12   SuperNEMO Raw Trigger Data

    Note that there are two namecycles here due to ROOT's checkpoiting to prevent data loss. No excess data is stored, and use of the RawTriggerData TTree is transparent:

    root [2] TTree* t = (TTree*)f.Get("RawTriggerData")
(TTree *) 0x377ff80
root [3] t->GetEntries()
(long long) 282801
root [4] t->Print()
... long list of branches, their types, and stats

    You can also use TBrowser to open and explore the file interactively.

    The RawTriggerData TTree has a single master branch that stores RRawTriggerData instances in fully split mode. That means you can direct access the data you are interested in interactively, or through the standard ROOT TTree/TBranch mechanisms and the new RDataFrame system. For example, to read all RTD data in a ROOT script:

    #include "snfee/data/RRawTriggerData.h"

void exampleRTDRead(const char* rtdFilename)
{
 TFile rtdFile{rtdFilename};
 TTree* rtdTree = (TTree*)rtdFile.Get("RawTriggerData");
 auto rtdData = new snfee::data::RRawTriggerData{};
 rtdTree->SetBranchAddress("RTD",&rtdData);

 Long64_t nRTD = rtdTree->GetEntries();
 for(Long64_t i{0}; i < nRTD; ++i) {
   rtdTree->GetEntry(i);

   // Do what you need with rtdData instance
 }
}

    then run it in root:

    snemo-shell> root setupSNemoRawDataObj
... will see Cling errors here, but can be ignored for now...
root [0] .L exampleRTDRead.C
... ignore Cling errors ...
root [1] exampleRTDRead("/path/to/your/rtdfile.root")

    One thing to note here is that in contrast to BRIO files, we can store all the Data Model classes without any need for Boost.Serialization. This could allow "Falaise" to use native ROOT I/O for improved usability and performance.

  3. In custom C++ programs:

    #include <snfee/io/multifile_data_reader.h>
#include <snfee/data/rtdReformater.h>
#include <snfee/data/raw_trigger_data.h>
#include <snfee/data/RRawTriggerData.h>

int main()
{
 using multifile_data_reader = snfee::io::multifile_data_reader;
 using raw_trigger_data = snfee::data::raw_trigger_data;
 using RRawTriggerData = snfee::data::RRawTriggerData;

 multifile_data_reader::config_type cfg{{"somertdfile.data.gz"}};
 multifile_data_reader rtdReader{cfg};

raw_trigger_data onlineRTD{};

 while(rtdReader.has_record_tag() && rtdReader.record_tag_is(raw_trigger_data::SERIAL_TAG)) {
   rtdReader.load(onlineRTD);
   RRawTriggerData offlineRTD = snfee::data::rtdOnlineToOffline(onlineRTD);

   // Do what you need with offlineRTD instance...
 }

 return 0;
}

    Just compile and link to the SNemoRawDataObj library (Full support to be added, but see examples/rtd2asroot/CMakeLists.txt for an example).

Compatibility with existing SNFrontEndElectronics code/files