desi_tsnr_afterburner can't find more recent GFA files

[weaverba137]

In desi_tsnr_afterburner, read_gfa_data() will not find recent GFA files, because the 'SV1', 'SV2', 'SV3' labels are hard-coded into the function. We should also verify which directory the function is using to find the files.

[weaverba137]

Due to lack of on-sky data in April-May 2024, this hasn't actually caused an issue yet. GFA files labelled 'main' will start appearing as soon as new on-sky data are available though.

[weaverba137]

Is desi_tsnr_afterburner typically only run via the script ~desi/pdaily/bin/pdaily_tsnr_nightqa, or could it potentially be run in multiple ways?

[weaverba137]

For reference: #1724.

[sbailey]

For daily operations, desi_tsnr_afterburner is only called via the pdaily_tsnr_nightqa scronjob wrapper.

For named productions, it is run "by hand" with commands like

salloc -N 4 -C cpu -t 4:00:00 -q interactive

time srun -N 4 -n 128 -c 8 --cpu-bind=cores desi_tsnr_afterburner --mpi --recompute --nproc 1 \
    --aux /global/cfs/cdirs/desi/survey/observations/SV1/sv1-tiles.fits \
    --gfa-proc-dir /global/cfs/cdirs/desi/survey/GFA/ \
    --compute-skymags \
    -o exposures-$SPECPROD.fits -t tiles-$SPECPROD.fits &> tsnr.log

This took many hours and multiple attempts to find the parallelism / memory options that worked for Iron; I expect this to be a challenge for Jura too (perhaps triggering the long-desired rewrite...).

If you have a quick fix for daily ops to accomodate the SV* vs. main issue, please open a PR with that.

[weaverba137]

Thanx @sbailey, that addresses my concern about the directory where it is looking for GFA files (it's correct), so the fix should be easy.

It would actually be simpler to ignore the SV[123] or other labels and simply return the most recent file, which is the current outcome of the function anyway.