This PR closes #94 with an issue when the drpall summary rows were being written by multiple processes at the same time. This PR uses the lockfile package to set up a lock on the drpall hdf5 file, during write, with a retry-timeout of 5 seconds.
This should now allow for safe updates/appends of the drpall file during cluster runs. At the end of the cluster run, we can manually run the convert_h5_to_fits to also produce a FITS file version of the summary file, which can be easier for astronomers to use.
This PR closes #94 with an issue when the drpall summary rows were being written by multiple processes at the same time. This PR uses the
lockfilepackage to set up a lock on the drpall hdf5 file, during write, with a retry-timeout of 5 seconds.This should now allow for safe updates/appends of the drpall file during cluster runs. At the end of the cluster run, we can manually run the
convert_h5_to_fitsto also produce a FITS file version of the summary file, which can be easier for astronomers to use.