The DetectorData class is our primary container for timestream data from a set of co-sampled detectors. This is used for signal, flags, detector pointing, etc. When storing this data to disk, it is useful to compress it. This work adds support for gzip and flac compression (and a "none" type for testing) of DetectorData:
Add optional compile time support for linking to libFLAC versions >= 1.4.0, which was the first version to support 32bit integers.
Add compiled utility functions for compressing / decompressing 32bit integer data with FLAC, and pybind11 interfaces to these.
Add new file io/compression.py, which has functions for compressing and decompressing DetectorData. The compressed format is a stream of bytes, a set of byte ranges for each detector, and a dictionary of compression properties.
Conditional routines to compress / decompress DetectorData with FLAC. For floating point data, an optional resolution can be specified for each detector to represent the floating point amount of one integer bit. Default behavior maps the full amplitude range of the input to the signed 31bit range supported in flac.
The DetectorData class is our primary container for timestream data from a set of co-sampled detectors. This is used for signal, flags, detector pointing, etc. When storing this data to disk, it is useful to compress it. This work adds support for gzip and flac compression (and a "none" type for testing) of DetectorData:
Add optional compile time support for linking to libFLAC versions >= 1.4.0, which was the first version to support 32bit integers.
Add compiled utility functions for compressing / decompressing 32bit integer data with FLAC, and pybind11 interfaces to these.
Add new file
io/compression.py, which has functions for compressing and decompressing DetectorData. The compressed format is a stream of bytes, a set of byte ranges for each detector, and a dictionary of compression properties.Conditional routines to compress / decompress DetectorData with FLAC. For floating point data, an optional resolution can be specified for each detector to represent the floating point amount of one integer bit. Default behavior maps the full amplitude range of the input to the signed 31bit range supported in flac.