To facilitate incremental updates of a daily prod DB load, I suggest adding an UPDATED column to the tiles-daily.fits/csv files, with a timestamp for when the entry for each tile was last updated. This could be used by the DB loader to filter to just the tiles that had changed since the last load. This could also be useful for periodic cleanup checks, like looking for cases where the timestamp of the redrock files is newer than the UPDATED timestamp for that tile, indicating that we re-processed the file but forgot to return the tsnr afterburner to update its tiles entry.
We could also add an equivalent column to the exposures-daily.* files, though I'm not sure what we'd use that for other than perhaps as a convenience to not have to cross match to the tiles files.
bin/desi_tsnr_afterburner is the script that writes those files.
To facilitate incremental updates of a daily prod DB load, I suggest adding an UPDATED column to the tiles-daily.fits/csv files, with a timestamp for when the entry for each tile was last updated. This could be used by the DB loader to filter to just the tiles that had changed since the last load. This could also be useful for periodic cleanup checks, like looking for cases where the timestamp of the redrock files is newer than the UPDATED timestamp for that tile, indicating that we re-processed the file but forgot to return the tsnr afterburner to update its tiles entry.
We could also add an equivalent column to the exposures-daily.* files, though I'm not sure what we'd use that for other than perhaps as a convenience to not have to cross match to the tiles files.
bin/desi_tsnr_afterburner is the script that writes those files.
Heads up @weaverba137 @akremin