search

desihub / desispec

DESI spectral pipeline
BSD 3-Clause "New" or "Revised" License
36 stars 24 forks source link

Inconsistent night and directory #1281

Open AlexGKim opened 3 years ago

AlexGKim commented 3 years ago

This may be intended but it may be confusing for people downstream. The night column in fibermap can be different within a file and inconsistent with the directory name in which it can be found.

filename = "/global/project/projectdirs/desi/spectro/redux/daily/tiles/cumulative/262/20210503/zbest-6-262-thru20210503.fits" dat = Table.read(filename, format='fits',hdu=2) print(numpy.unique(dat['NIGHT']))

NIGHT

20210502 20210503

akremin commented 3 years ago

This was also raised to me yesterday by Raga Pucha on Slack regarding Denali. In some sense that is an ever more problematic case because only the most recent night was processed for cumulative in Denali. So the previous nights aren't available.

My guess is that the fibers in question were disabled on the last night that the tile was observed (or otherwise didn't get data). While understandably annoying, if my hypothesis is true, then the per-target code is correct that the last night of observations for that target was indeed on an earlier night than the last night of the tile.

That being said, the coadds for the later night will contain all the previous data and the redshifts should be run. So the information is still available in that later night.

I think the workaround will either be: (1) changing the meaning of LAST_NIGHT for an object to mean the last night the tile was observered; (2) adding a LAST_TILE_NIGHT to the per-object table; or (3) improving the documentation to have users match the TILEID to the LAST_NIGHT from the tiles.csv file (tiles-denali.csv in the case of Denali).

AlexGKim commented 3 years ago

@akremin . I personally don't need any changes if your interpretation is correct.