This PR is a followup to #303 which broke the ability to write the rrdetails HDF5 file, due to a str-vs-bytes issue on the FITMETHOD column. Forcing FITMETHOD to .astype('S4') left the FITMETHOD column as bytes while the SPECTYPE and SUBTYPE columns were unicode strings. Writing to the fits file worked fine because astropy.io.fits is forgiving about bytes vs. str, but h5py is not and we tripped over our attempt to standardize it for h5py np.char.encode(zfit[colname], 'ascii').
This fix is two-fold:
use .astype('U4') instead of .astype('S4') so that FITMETHOD is a str column just like SPECTYPE and SUBTYPE
while writing the hdf5 file, first check if a unicode-to-bytes conversion is needed before trying to convert a column.
Either fix alone would have been sufficient; including both helps us be more future proof to other bytes-vs-str bugs.
I'm going to self-merge this so that I can proceed with Jura testing (otherwise zproc jobs are crashing when using redrock/main)
coverage: 39.04% (-0.01%) from 39.051%
when pulling 61b66031419a01cb58136b6018a2f0cf767f027e on bytes_str_grr
into f7f95e51e1539c5a08ba3925d3c1cdd7896a513c on main.
This PR is a followup to #303 which broke the ability to write the rrdetails HDF5 file, due to a str-vs-bytes issue on the FITMETHOD column. Forcing FITMETHOD to
.astype('S4')
left the FITMETHOD column as bytes while the SPECTYPE and SUBTYPE columns were unicode strings. Writing to the fits file worked fine because astropy.io.fits is forgiving about bytes vs. str, but h5py is not and we tripped over our attempt to standardize it for h5pynp.char.encode(zfit[colname], 'ascii')
.This fix is two-fold:
.astype('U4')
instead of.astype('S4')
so that FITMETHOD is a str column just like SPECTYPE and SUBTYPEEither fix alone would have been sufficient; including both helps us be more future proof to other bytes-vs-str bugs.
I'm going to self-merge this so that I can proceed with Jura testing (otherwise zproc jobs are crashing when using redrock/main)