desihub / desispec

DESI spectral pipeline
BSD 3-Clause "New" or "Revised" License
36 stars 24 forks source link

Coadd arrays not sorted by fiber number #960

Open dkirkby opened 4 years ago

dkirkby commented 4 years ago

I don't see any datamodel doc for the SV0 coadd files but I notice that fibers in the BRZ_* arrays are not sorted in order of increasing FIBER number, so permuted relative to arrays in (C)FRAME files. Is this deliberate or could we preserve the (C)FRAME ordering to make it easier to match the individual spectra contributing to a coadd?

To reproduce this feature / problem:

# FIBER order in CFRAME spec 0
fitsio.read('/global/cfs/cdirs/desi/spectro/redux/daily/tiles/66003/20200315/cframe-b0-00055655.fits',
            ext='FIBERMAP', columns='FIBER', rows=range(10))
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=int32)

# FIBER order in COADD spec 0
fitsio.read('/global/cfs/cdirs/desi/spectro/redux/daily/tiles/66003/20200315/coadd-0-66003-20200315.fits',
            ext='FIBERMAP', columns='FIBER', rows=range(10))
array([365, 471, 459, 356, 474, 398, 191, 397, 469, 363], dtype=int32)
sbailey commented 4 years ago

When coadding cframes for a single tile, I agree that one might expect/wish them to retain the FIBER order of the input files. However, coadds can also apply to spectra files or cframes from different tiles, where a coadded target is on different fibers and thus doesn't have a uniquely defined input FIBER.

So that's the underlying reason that the coadds don't preserve input order. While acknowledging that that is a pain for this tile-based coadd use-case, I'm inclined to leave it as-is so that people don't get into the habit of assuming that it will always be row-matched with the inputs because that won't be true in the future with more general DESI observations and coadds.

FWIW, the coadds are sorted by TARGETID.

Opinions?