This PR makes an initial attempt at creating a drpall summary file at the end of the drp run. During individual exposure reductions, after the lvmSFrame file has been produced, it adds a row of data to the dprall-[drpver] summary file. It creates the file if it doesn't exist, otherwise it appends the row to the table. During the run, it writes it out as a pandas HDF5 table using tables format. There is a new function that can be run, after the complete pipeline finshes, to convert the final h5 file to a FITS file.
It currently aggregates information from the raw frames metadata file, the headers of the lvmSFrame file, and some path information from sdss-access. As we decide on more metadata to include in the summary file we can update _collect_header_data with the new lvmSFrame header keys to extract.
This PR makes an initial attempt at creating a drpall summary file at the end of the drp run. During individual exposure reductions, after the
lvmSFramefile has been produced, it adds a row of data to the dprall-[drpver] summary file. It creates the file if it doesn't exist, otherwise it appends the row to the table. During the run, it writes it out as a pandas HDF5 table usingtablesformat. There is a new function that can be run, after the complete pipeline finshes, to convert the final h5 file to a FITS file.It currently aggregates information from the raw frames metadata file, the headers of the lvmSFrame file, and some path information from sdss-access. As we decide on more metadata to include in the summary file we can update
_collect_header_datawith the newlvmSFrameheader keys to extract.