fix and update Tractor catalog (FITS) headers

moustakas commented 5 years ago

The FITS headers written out with the Tractor catalogs need some work. (I've included an example header from dr8b in its entirety below.)

Problems and questions:

Several header cards are cut off, e.g., SURVEYV. We should add more characters or drop the /global/project/projectdirs/ (or /global/projecta/projectdirs/) prefix.
UNWISTD and DEPVER15 cards are cut off in an odd way.
Why does PROCTYPE map to "tile"? Why not "brick"? What are the other options?
HOSTNAME and HOSTFQDN contain the same output; if that's intentional, just keep one.

Strongly recommended keyword changes (more survey agnostic):

DECALSDR --> LSDR or DLISDR
DECALSDT --> RUNDATE, PROCDATE, or something similar

Proposed keyword changes (for clarity):

SURVEYV --> LSDIR (for LEGACY_SURVEY_DIR)
DRVERSIO --> RELEASE
HOSTNAME --> NODENAME
NERSC --> HOSTNAME
BRICKNAM --> BRICK
BRICKBND --> BANDS

Proposed new keywords:

CAMERA or CCDSFILE --> list of cameras used; or maybe the input CCDs filename
Something to indicate which WISE sky background models were used.

Additional proposed changes and checks:

Make sure the code dependency versions written to the haeder are actually correct.
Standardize use of "Legacy Surveys", "LegacySurvey", and "LegacySurveys" (propose the latter).
Consider adding units ("degrees") in the comments to RAMIN, RAMAX, DECMIN, DECMAX, BRICKRA, BRICKDEC, CORN*
Change RA, DEC to decimal degrees and CENTRA, CENTDEC to hms.
Define the BRIGHTBLOB and WISEMASK bits.
Since we define the WISE Vega-->AB conversion factors, should we also include something about the MW transmission factors?
Make sure nothing breaks what the NOAO Data Lab expects.

SIMPLE  =                    T / file does conform to FITS standard             
BITPIX  =                   16 / number of bits per data pixel                  
NAXIS   =                    0 / number of data axes                            
EXTEND  =                    T / FITS dataset may contain extensions            
COMMENT   FITS (Flexible Image Transport System) format is defined in 'Astronomy
COMMENT   and Astrophysics', volume 376, page 359; bibcode: 2001A&A...376..359H 
COMMENT Data product of the DECam Legacy Survey (DECaLS)                        
COMMENT Full documentation at http://legacysurvey.org                           
LEGPIPEV= 'dr8-test10-159-ga885bb45' / legacypipe git version                   
SURVEYV = '/global/project/projectdirs/cosmo/work/legacysurvey/dr8b/runbrick-de'
DECALSDR= 'DR8     '           / DECaLS release name                            
DECALSDT= '2019-03-13T07:50:24.191971' / runbrick.py run time                   
SURVEY  = 'DECaLS+MzLS+BASS'   / Legacy Surveys                                 
SURVEYID= 'DECaLS BASS MzLS'   / Survey name                                    
DRVERSIO=                 8000 / Survey data release number                     
OBSTYPE = 'object  '           / Observation type                               
PROCTYPE= 'tile    '           / Processing type                                
HOSTNAME= 'nid01884'           / Machine where runbrick.py was run              
HOSTFQDN= 'nid01884'           / Machine where runbrick.py was run              
NERSC   = 'edison  '           / NERSC machine where runbrick.py was run        
JOB_ID  = '12647417'           / SLURM job id                                   
ARRAY_ID= 'none    '           / SLURM job array id                             
LONGSTRN= 'OGIP 1.0'           / CONTINUE cards are used                        
UNWISD1 = '/global/projecta/projectdirs/cosmo/work/wise/outputs/merge/neo4/full'
UNWISD2 = '/global/project/projectdirs/cosmo/data/unwise/allwise/unwise-coadds/'
UNWISTD = '/global/projecta/projectdirs/cosmo/work/wise/outputs/merge/neo4' / un
DEPNAM00= 'desiconda'          / Dependency name                                
DEPVER00= '20190311-1.2.7-img' / Dependency version                             
DEPNAM01= 'astropy '           / Dependency name                                
DEPVER01= '2.0.9   '           / Dependency version                             
DEPNAM02= 'matplotlib'         / Dependency name                                
DEPVER02= '2.1.2   '           / Dependency version                             
DEPNAM03= 'mkl     '           / Dependency name                                
DEPVER03= '2019.1  '           / Dependency version                             
DEPNAM04= 'numpy   '           / Dependency name                                
DEPVER04= '1.16.2  '           / Dependency version                             
DEPNAM05= 'python  '           / Dependency name                                
DEPVER05= '3.6.6   '           / Dependency version                             
DEPNAM06= 'scipy   '           / Dependency name                                
DEPVER06= '1.2.1   '           / Dependency version                             
DEPNAM07= 'astrometry'         / Dependency name                                
DEPVER07= '/global/project/projectdirs/cosmo/work/legacysurvey/dr8b/code/build&'
CONTINUE  '/edison/lib/python3.6/site-packages/astrometry'                      
DEPNAM08= 'tractor '           / Dependency name                                
DEPVER08= '/global/project/projectdirs/cosmo/work/legacysurvey/dr8b/code/build&'
CONTINUE  '/edison/lib/python3.6/site-packages/tractor'                         
DEPNAM09= 'fitsio  '           / Dependency name                                
DEPVER09= '/global/common/software/desi/cori/desiconda/20190311-1.2.7-img/cond&'
CONTINUE  'a/lib/python3.6/site-packages/fitsio'                                
DEPNAM10= 'TYCHO2_KD'          / Dependency name                                
DEPVER10= '/global/project/projectdirs/cosmo/staging/tycho2' / Dependency versio
DEPNAM11= 'GAIA_CAT'           / Dependency name                                
DEPVER11= '/global/project/projectdirs/cosmo/work/gaia/chunks-gaia-dr2-astrom/' 
DEPNAM12= 'LARGEGALAXIES'      / Dependency name                                
DEPVER12= '/global/project/projectdirs/cosmo/staging/largegalaxies/v2.0' / Depen
DEPNAM13= 'WISE_PSF'           / Dependency name                                
DEPVER13= '/global/project/projectdirs/cosmo/staging/unwise_psf/2018-11-25/etc' 
DEPNAM14= 'unwise  '           / Dependency name                                
DEPVER14= '/global/projecta/projectdirs/cosmo/work/wise/outputs/merge/neo4/ful&'
CONTINUE  'ldepth:/global/project/projectdirs/cosmo/data/unwise/allwise/unwise&'
CONTINUE  '-coadds/fulldepth'                                                   
DEPNAM15= 'unwise_tr'          / Dependency name                                
DEPVER15= '/global/projecta/projectdirs/cosmo/work/wise/outputs/merge/neo4' / De
BRICKNAM= '0001m002'           / LegacySurvey brick RRRr[pm]DDd                 
BRICKID =               328928 / LegacySurvey brick id                          
RAMIN   =                   0. / Brick RA min                                   
RAMAX   =                 0.25 / Brick RA max                                   
DECMIN  =               -0.375 / Brick Dec min                                  
DECMAX  =               -0.125 / Brick Dec max                                  
BRICKRA =                0.125 / Brick center                                   
BRICKDEC=                -0.25 / Brick center                                   
RA      = '00:00:30.000'       / [h] RA Brick center                            
DEC     = '-00:15:00.000'      / [deg] Dec Brick center                         
CENTRA  =                0.125 / [deg] Brick center RA                          
CENTDEC =                -0.25 / [deg] Brick center Dec                         
CORN1RA =    0.255965935903178 / [deg] Brick corner RA                          
CORN1DEC=   -0.380962387829748 / [deg] Brick corner Dec                         
CORN2RA =     359.994034064097 / [deg] Brick corner RA                          
CORN2DEC=   -0.380962387829748 / [deg] Brick corner Dec                         
CORN3RA =     359.994036676442 / [deg] Brick corner RA                          
CORN3DEC=   -0.119036306008142 / [deg] Brick corner Dec                         
CORN4RA =    0.255963323557502 / [deg] Brick corner RA                          
CORN4DEC=   -0.119036306008142 / [deg] Brick corner Dec                         
BRICK_G =                    T / Does band g touch this brick?                  
CAMS_G  = 'decam   '           / Cameras contributing band g                    
BRICK_R =                    T / Does band r touch this brick?                  
CAMS_R  = 'decam   '           / Cameras contributing band r                    
BRICK_Z =                    T / Does band z touch this brick?                  
CAMS_Z  = 'decam   '           / Cameras contributing band z                    
BRICKBND= 'grz     '           / Bands touching this brick                      
NBANDS  =                    3 / Number of bands in this catalog                
BAND0   = 'g       '           / Band name in this catalog                      
BAND1   = 'r       '           / Band name in this catalog                      
BAND2   = 'z       '           / Band name in this catalog                      
PRODTYPE= 'catalog '           / NOAO data product type                         
APRAD0  =                  0.5 / Aperture radius, in arcsec                     
APRAD1  =                 0.75 / Aperture radius, in arcsec                     
APRAD2  =                   1. / Aperture radius, in arcsec                     
APRAD3  =                  1.5 / Aperture radius, in arcsec                     
APRAD4  =                   2. / Aperture radius, in arcsec                     
APRAD5  =                  3.5 / Aperture radius, in arcsec                     
APRAD6  =                   5. / Aperture radius, in arcsec                     
APRAD7  =                   7. / Aperture radius, in arcsec                     
MASKB0  = 'badpix  '           / Mask bit 2**0=1 meaning                        
MASKB1  = 'satur   '           / Mask bit 2**1=2 meaning                        
MASKB2  = 'interp  '           / Mask bit 2**2=4 meaning                        
MASKB4  = 'cr      '           / Mask bit 2**4=16 meaning                       
MASKB6  = 'bleed   '           / Mask bit 2**6=64 meaning                       
MASKB7  = 'trans   '           / Mask bit 2**7=128 meaning                      
MASKB8  = 'edge    '           / Mask bit 2**8=256 meaning                      
MASKB9  = 'edge2   '           / Mask bit 2**9=512 meaning                      
MASKB10 = 'longthin'           / Mask bit 2**10=1024 meaning                    
MASKB11 = 'outlier '           / Mask bit 2**11=2048 meaning                    
WISEAB1 =                2.699 / WISE Vega to AB conv for band 1                
WISEAB2 =                3.339 / WISE Vega to AB conv for band 2                
WISEAB3 =                5.174 / WISE Vega to AB conv for band 3                
WISEAB4 =                 6.62 / WISE Vega to AB conv for band 4

bradlyke commented 5 years ago

PROCTYPE changed to 'brick'
added LSDIRPFX for LEGACY_SURVEYS_DIR_PREFIX with value '/global/project/projectdirs'
stripped the prefix from SURVEYV card
Renamed SURVEYV card to LSDIR
Removed HOSTFQDN Card (it duplicates HOSTNAME)
Renamed HOSTNAME card to NODENAME
Renamed NERSC card to HOSTNAME
Renamed DRVERSIO card to RELEASE
Renamed DECALSDT card to RUNDATE
Renamed DECALSDR card to LSDR
Renamed BRICKNAM card to BRICK
Renamed BRICKBND card to BANDS
Added '[deg]' to RAMIN, DECMIN, RAMAX, DECMAX comments
Removed BRICKRA and BRICKDEC cards as they duplicated CENTRA, CENTDEC
CENTRA, CENTDEC remained in degrees (as requested by NOAO).
RA/DEC remained in hms/dms (as requested by NOAO). Changed units in comments to [hms] and [dms]
Changed "LegacySurvey" and "Legacy Survey" to "LegacySurveys" where they appear in card comments.

Skipped:

Left dependency versions as they are. Not a quick solution.
Brightblob and WISEMASK bits not changed
MW Transition factors not changed
Did not add CCDSFILE or CAMERA card.

bradlyke commented 5 years ago

Coadds/Image files

The following cards are missing the comment field:
- TELESCOP, OBSERVAT, OBS-LAT, OBS-LONG, OBS-ELEV, INSTRUME, FILTER, FILTERX, EQUINOX
Does ZDITHER0 need units?
Do Reference RA and Reference DEC need units?

moustakas commented 5 years ago

Largely addressed by #298. Additional fixes are being tracked in #314 and #320.

legacysurvey / legacypipe

fix and update Tractor catalog (FITS) headers #275