search

LSSTDESC / DC2-production

Configuration, production, validation specifications and tools for the DC2 Data Set.
BSD 3-Clause "New" or "Revised" License
11 stars 7 forks source link

WCSAXES keyword errors in the outut ImSim FITS files #351

Closed heather999 closed 5 years ago

heather999 commented 5 years ago

While trying to check the FITS files before transferring to IN2P3, we run fitsverify. Every output file, and every HDU returns this error: *** Error: WCSAXES keyword #100 appears after other WCS keyword CD2_2 #49

While I fully expect this is not a huge problem, it does make checking the FITS files more tedious in that these errors need to be explicitly ignored. Is there any way to look at how the WCS keywords are being written to the header and hopefully avoid this for future production?

villarrealas commented 5 years ago

Pinging on @jchiang87 since he would have more information regarding how imSim outputs are being written.

jchiang87 commented 5 years ago

@heather999 Can you please post the code that produces this error? I don't get any errors when I run the .verify method for a raw file read in with astropy.io.fits.

heather999 commented 5 years ago

Here is one example, using fverify available in the heasoft module at NERSC. Looking at astropy's doc, my understanding is that they are lenient with input files.

Astropy is to be tolerant in input and strict in output. When Astropy reads a FITS file which is not conforming to FITS standard, it will not raise an error and exit. It will try to make the best educated interpretation and only gives up when the offending data is accessed and no unambiguous interpretation can be reached

so something like a mis-ordering of the keywords may not be counted as an error 

heatherk@cori09:~> module load heasoft
heatherk@cori09:~> fverify /global/projecta/projectdirs/lsst/production/DC2_ImSim/Run2.1i/sim/y1-y2-wfd/00000000to00071840/00012444/lsst_a_12444_R43_S22_y.fits 

                      FVERIFY V4.18 (CFITSIO V3.420)                       
                      ------------------------------                       

HEASARC conventions are being checked.

File: /global/projecta/projectdirs/lsst/production/DC2_ImSim/Run2.1i/sim/y1-y2-wfd/00000000to00071840/00012444/lsst_a_12444_R43_S22_y.fits

17 Header-Data Units in this file.

=================== HDU 1: Primary Array ===================

*** Error:   WCSAXES keyword #76 appears after other WCS keyword CD2_2 #25

 105 header keywords

 Null data array; NAXIS = 0 

=================== HDU 2: BINARY Table ====================

*** Warning: Keyword EXTNAME is duplicated in card #26 and card #132.
*** Error:   WCSAXES keyword #100 appears after other WCS keyword CD2_2 #49

 133 header keywords

 Segment10  (1 columns x 2048 rows)

 Col# Name (Units)       Format
   1 COMPRESSED_DATA      1PB(647)  

=================== HDU 3: BINARY Table ====================

*** Warning: Keyword EXTNAME is duplicated in card #26 and card #132.
*** Error:   WCSAXES keyword #100 appears after other WCS keyword CD2_2 #49

 133 header keywords

 Segment11  (1 columns x 2048 rows)

 Col# Name (Units)       Format
   1 COMPRESSED_DATA      1PB(656)  

=================== HDU 4: BINARY Table ====================

*** Warning: Keyword EXTNAME is duplicated in card #26 and card #132.
*** Error:   WCSAXES keyword #100 appears after other WCS keyword CD2_2 #49

 133 header keywords

 Segment12  (1 columns x 2048 rows)

 Col# Name (Units)       Format
   1 COMPRESSED_DATA      1PB(652)  

=================== HDU 5: BINARY Table ====================

*** Warning: Keyword EXTNAME is duplicated in card #26 and card #132.
*** Error:   WCSAXES keyword #100 appears after other WCS keyword CD2_2 #49

 133 header keywords

 Segment13  (1 columns x 2048 rows)

 Col# Name (Units)       Format
   1 COMPRESSED_DATA      1PB(668)  

=================== HDU 6: BINARY Table ====================

*** Warning: Keyword EXTNAME is duplicated in card #26 and card #132.
*** Error:   WCSAXES keyword #100 appears after other WCS keyword CD2_2 #49

 133 header keywords

 Segment14  (1 columns x 2048 rows)

 Col# Name (Units)       Format
   1 COMPRESSED_DATA      1PB(646)  

=================== HDU 7: BINARY Table ====================

*** Warning: Keyword EXTNAME is duplicated in card #26 and card #132.
*** Error:   WCSAXES keyword #100 appears after other WCS keyword CD2_2 #49

 133 header keywords

 Segment15  (1 columns x 2048 rows)

 Col# Name (Units)       Format
   1 COMPRESSED_DATA      1PB(636)  

=================== HDU 8: BINARY Table ====================

*** Warning: Keyword EXTNAME is duplicated in card #26 and card #132.
*** Error:   WCSAXES keyword #100 appears after other WCS keyword CD2_2 #49

 133 header keywords

 Segment16  (1 columns x 2048 rows)

 Col# Name (Units)       Format
   1 COMPRESSED_DATA      1PB(636)  

=================== HDU 9: BINARY Table ====================

*** Warning: Keyword EXTNAME is duplicated in card #26 and card #132.
*** Error:   WCSAXES keyword #100 appears after other WCS keyword CD2_2 #49

 133 header keywords

 Segment17  (1 columns x 2048 rows)

 Col# Name (Units)       Format
   1 COMPRESSED_DATA      1PB(674)  

=================== HDU 10: BINARY Table ===================

*** Warning: Keyword EXTNAME is duplicated in card #26 and card #132.
*** Error:   WCSAXES keyword #100 appears after other WCS keyword CD2_2 #49

 133 header keywords

 Segment07  (1 columns x 2048 rows)

 Col# Name (Units)       Format
   1 COMPRESSED_DATA      1PB(634)  

=================== HDU 11: BINARY Table ===================

*** Warning: Keyword EXTNAME is duplicated in card #26 and card #132.
*** Error:   WCSAXES keyword #100 appears after other WCS keyword CD2_2 #49

 133 header keywords

 Segment06  (1 columns x 2048 rows)

 Col# Name (Units)       Format
   1 COMPRESSED_DATA      1PB(640)  

=================== HDU 12: BINARY Table ===================

*** Warning: Keyword EXTNAME is duplicated in card #26 and card #132.
*** Error:   WCSAXES keyword #100 appears after other WCS keyword CD2_2 #49

 133 header keywords

 Segment05  (1 columns x 2048 rows)

 Col# Name (Units)       Format
   1 COMPRESSED_DATA      1PB(670)  

=================== HDU 13: BINARY Table ===================

*** Warning: Keyword EXTNAME is duplicated in card #26 and card #132.
*** Error:   WCSAXES keyword #100 appears after other WCS keyword CD2_2 #49

 133 header keywords

 Segment04  (1 columns x 2048 rows)

 Col# Name (Units)       Format
   1 COMPRESSED_DATA      1PB(664)  

=================== HDU 14: BINARY Table ===================

*** Warning: Keyword EXTNAME is duplicated in card #26 and card #132.
*** Error:   WCSAXES keyword #100 appears after other WCS keyword CD2_2 #49

 133 header keywords

 Segment03  (1 columns x 2048 rows)

 Col# Name (Units)       Format
   1 COMPRESSED_DATA      1PB(648)  

=================== HDU 15: BINARY Table ===================

*** Warning: Keyword EXTNAME is duplicated in card #26 and card #132.
*** Error:   WCSAXES keyword #100 appears after other WCS keyword CD2_2 #49

 133 header keywords

 Segment02  (1 columns x 2048 rows)

 Col# Name (Units)       Format
   1 COMPRESSED_DATA      1PB(660)  

=================== HDU 16: BINARY Table ===================

*** Warning: Keyword EXTNAME is duplicated in card #26 and card #132.
*** Error:   WCSAXES keyword #100 appears after other WCS keyword CD2_2 #49

 133 header keywords

 Segment01  (1 columns x 2048 rows)

 Col# Name (Units)       Format
   1 COMPRESSED_DATA      1PB(654)  

=================== HDU 17: BINARY Table ===================

*** Warning: Keyword EXTNAME is duplicated in card #26 and card #132.
*** Error:   WCSAXES keyword #100 appears after other WCS keyword CD2_2 #49

 133 header keywords

 Segment00  (1 columns x 2048 rows)

 Col# Name (Units)       Format
   1 COMPRESSED_DATA      1PB(652)  

++++++++++++++++++++++ Error Summary  ++++++++++++++++++++++

 HDU#  Name (version)       Type             Warnings  Errors
 1                          Primary Array    0         1     
 2     Segment10            Binary Table     1         1     
 3     Segment11            Binary Table     1         1     
 4     Segment12            Binary Table     1         1     
 5     Segment13            Binary Table     1         1     
 6     Segment14            Binary Table     1         1     
 7     Segment15            Binary Table     1         1     
 8     Segment16            Binary Table     1         1     
 9     Segment17            Binary Table     1         1     
 10    Segment07            Binary Table     1         1     
 11    Segment06            Binary Table     1         1     
 12    Segment05            Binary Table     1         1     
 13    Segment04            Binary Table     1         1     
 14    Segment03            Binary Table     1         1     
 15    Segment02            Binary Table     1         1     
 16    Segment01            Binary Table     1         1     
 17    Segment00            Binary Table     1         1     

**** Verification found 16 warning(s) and 17 error(s). ****
jchiang87 commented 5 years ago

@villarrealas This is fixed in https://github.com/LSSTDESC/imSim/releases/tag/v0.5.1

villarrealas commented 5 years ago

Okay. So this won't get caught in the current years already running, but we'll switch over to v0.5.1 once we can.

katrinheitmann commented 5 years ago

Since this is fixed, I will close this issue now. @heather999 feel free to reopen it if this shows up again.