search

desihub / desispec

DESI spectral pipeline
BSD 3-Clause "New" or "Revised" License
36 stars 24 forks source link

desi_group_spectra >1000 files #2198

Closed segasai closed 6 months ago

segasai commented 6 months ago

Hi,

While running desi_group_spectra on some tertiary program data I see this error: ValueError: The header keyword 'INFIL1000' with its value is too long which is clearly caused by >1000 input files. I'm stacking data in small HPX'es, but in the center of the tile with more than 34 exposures you get >1000 cframe files.

I understand that this is somewhat 'special' case, but it is not that unreasonable situation.

My proposal is to put a warning here: https://github.com/desihub/desispec/blob/c2222d2b019634f1d9c125cf9ba47bd12aa3863e/py/desispec/scripts/group_spectra.py#L157 and stop writing INFIL after 1000 files.

Comments ?

sbailey commented 6 months ago

That sounds reasonable. How about also adding a keyword so that the output file can indicate that there were actually more than 1000 input files (as opposed to exactly 1000, ending with INFIL999)? NUMFILES? NUMINFIL? NINFILES? NUMINPUT? FITS keyword 8-character limit grr...

Please go ahead with a PR to support your use case.