Proposal for a file header

[lucabaldini]

I think we need a file header in the output file written by the daq that can be read by any consumer program (e.g., the reconstruction).

[For one thing yesterday Carmelo was struggling with putting the runId in the output ntuple parsing the file name and it's clear that that information needs to be in the binary file produced by the DAQ.]

I am proposing something along this lines:

1. (uint16_t) format_version (the file format version)
2. (uint16_t) header_length (the length of the file header, including the version and length fields)
3. (uint16_t) run_id (the run Id)
4. (unit32_t) start_time (the number of seconds from January 1, 1970 at the start run)
5. (uint8_t) readout_mode (e.g., full frame or windowed)
... (possibly more stuff)

The applications reading binary files will read the version number and, at that, point will know what to do with the header. Maintaining a list of version will allow to change the file format if needed, and still be able to read all kinds of binary files. We can add a compatibility flag to tell application not to expect a header, so that we maintain the capability of reading old files, where the header was not there.

I'd like to discuss this proposal, come up with an exhaustive set of quantities that the DAQ needs to write in the header to give enough context to process a run and then proceed.

I'll be happy to go with any sensible alternative scheme that people might come up with.

[csgro]

I suggest we keep the possibility to process (reconstruct) old binary files (without header) in a transparent way: i.e. check if the first 16 bits are the event header 0xffff, if yes continue with the old way, with no additional info. One question would be if we want to keep the "run start" in each events of just in the header (I personally prefer the latter).

The recon needs to be adapted, so let's make a decision before a new recon version is distributed (this looks obvious to me, just a reminder).

[gspandre]

Il buon vecchio programma di ricostruzione prevede già la possibilità di avere un header aggiuntivo in cima al file, (questo per soddisfare le esigenze di Soffitta et al) basterebbe riadattare questo al nuovo header. Unica cosa non era un riconoscimento automatico ma si doveva spuntare "header on" nel control panel della Gui. Gloria

Inviato da iPhone

Il giorno 07 mag 2016, alle ore 16:49, Carmelo notifications@github.com ha scritto:

I suggest we keep the possibility to process (reconstruct) old binary files (without header) in a transparent way: i.e. check if the first 16 bits are the event header 0xffff, if yes continue with the old way, with no additional info. One question would be if we want to keep the "run start" in each events of just in the header (I personally prefer the latter).

The recon needs to be adapted, so let's make a decision before a new recon version is distributed (this looks obvious to me, just a reminder).

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub

[lucabaldini]

Modification:

1. (uint16_t) file_header (e.g., 0xefff)
2. (uint16_t) format_version (the file format version)
3. (uint16_t) header_length (the length of the file header, including the version and length fields)
4. (uint16_t) run_id (the run Id)
5. (unit32_t) start_time (the number of seconds from January 1, 1970 at the start run)
6. (uint8_t) readout_mode (e.g., full frame or windowed)
... (possibly more stuff)

This way:

• if the first 16 bits are 0xefff we know that this is a file with a header, and the rest of the header defines what to do with it.
• if the first 16 bits are 0xffff, then we know this is an old file in windowed mode.
• if the first 16 bits are something smaller than 0x0fff, then this is an old file in full frame mode.

Would this catch all cases?