Possible fix to "archetypes mode with wrong input dir = non-obvious crash" issue

[abhi0395]

Hi @sbailey

Regarding redrock/issue#243

I guess the reason this happens is that redrock looks for archetype files in the very last steps when it actually looks for the best archetypes for best-fit redshifts. Until then it doesn't actually look if the file exists.

I think this can be solved by checking the args.archetypes in the very beginning. If the archetypes are provided and it does not exist or are unreadable or empty, then rrdesi should just exit, otherwise continue.

I am creating this PR with the branch name missing_archetypes that solves this issue.

I re-ran the same script on missing_archetypes and it succeeds if the provided file exists otherwise it will just exit.

rrdesi --archetypes /does/not/exist -i $CFS/desi/spectro/redux/fuji/tiles/cumulative/80606/20201219/coadd-0-80606-thru20201219.fits --targetids 39627646572697743,39627652591519496 -o test.fits -d test.hd5

gives

ERROR: can't find archetypes_dir

But if the file exists:

rrdesi --archetypes/global/homes/a/abhijeet/software/desisoft/redrock-archetypes -i $CFS/desi/spectro/redux/fuji/tiles/cumulative/80606/20201219/coadd-0-80606-thru20201219.fits --targetids 39627646572697743,39627652591519496 -o test.fits -d test.hd5

it succeeds!

While the main branch fails if run:

rrdesi --archetypes /does/not/exist -i $CFS/desi/spectro/redux/fuji/tiles/cumulative/80606/20201219/coadd-0-80606-thru20201219.fits --targetids 39627646572697743,39627652591519496 -o test.fits -d test.hd5

gives error:

--- Process 0 raised an exception ---
Proc 0: Traceback (most recent call last):
Proc 0:   File "/global/homes/a/abhijeet/software/desisoft/redrock/py/redrock/external/desi.py", line 805, in rrdesi
    scandata, zfit = zfind(targets, dtemplates, mpprocs,
Proc 0:   File "/global/homes/a/abhijeet/software/desisoft/redrock/py/redrock/zfind.py", line 414, in zfind
    spectype = [ el[0].split(':::')[0] for el in tmp['fulltype'] ] #el is a list with one element (corresponding to each minima)
Proc 0: KeyError: 'fulltype'

Traceback (most recent call last):
  File "/global/homes/a/abhijeet/software/desisoft/redrock/bin/rrdesi", line 10, in <module>
    desi.rrdesi(comm=None)
  File "/global/homes/a/abhijeet/software/desisoft/redrock/py/redrock/external/desi.py", line 886, in rrdesi
    raise err
  File "/global/homes/a/abhijeet/software/desisoft/redrock/py/redrock/external/desi.py", line 805, in rrdesi
    scandata, zfit = zfind(targets, dtemplates, mpprocs,
  File "/global/homes/a/abhijeet/software/desisoft/redrock/py/redrock/zfind.py", line 414, in zfind
    spectype = [ el[0].split(':::')[0] for el in tmp['fulltype'] ] #el is a list with one element (corresponding to each minima)
KeyError: 'fulltype'

[coveralls]

Coverage: 37.89% (-0.4%) from 38.316% when pulling f4feb2786207c1de81fde04ea73ccf8b87d28538 on missing_archetypes into 11c9861669d4cb7adaac0c8221cdaa3b78155e7f on main.

[sbailey]

Looks good, thanks for also adding the empty/unreadable check even if it exists.