Support reading zbest redshift files

[weaverba137]

Some early tiles in the daily specprod have zbest instead of redrock files. To support loading daily into a database, these need to be read by desispec.scripts.zcatalog. In this PR:

• read_redrock detects whether a zbest file is being opened and attempts to reconstruct its output as best as possible to match a redrock file.
• Do not require the tiles file to be a FITS file.
• When writing out zcatalog files, include comments on columns as well as units.

Open questions:

• Do we want to put any more effort toward recovering missing data? For example, MEAN_PSF_TO_FIBER_SPECFLUX in the (coadded) fibermap table or the entire TSNR2 table missing from zbest files?
• In order to reduce code complexity, in this PR, read_redrock always converts the per-exposure fibermap into a astropy.table.Table. Previously, the per-exposure fibermap would only be a Table if recoadd_fibermap=True. We need to determine whether this introduces a significant performance overhead.
• While testing, I noticed the output zcatalog files did not have comments for every column. I thought I reported this elsewhere, but I can't find where. Does anyone know? The missing columns were:
  • FIRSTNIGHT
  • NUMEXP
  • NUMTILE
  • MIN_MJD
  • MAX_MJD
  • MEAN_MJD
  • IN_COADD_(B|R|Z), in the EXP_FIBERMAP HDU.

Some additional operational testing should probably be done before merging.

[coveralls]

coverage: 30.128% (-0.04%) from 30.164% when pulling 46c167a9de6edbfc9d83873329ba2d727aca7ab2 on read-zbest into 97b174838a2e951a32aa719f19da07e184585c39 on main.

[weaverba137]

At the data telecon on 2024-09-24, there appeared to be a consensus that we should not put additional effort toward recovering data missing from zbest files. That leaves two open questions: does a particular Table() conversion impact performance, and new/unknown columns.

The second open question is also mentioned in desihub/desidatamodel#199.

[sbailey]

I verified with runs of this branch and main, that the switch from numpy structured arrays to astropy Table has negligible impact on the overall runtime for desi_zcatalog and that they produce identical catalogs, i.e. nothing funny with bytes vs. strings, masked values, etc.