Improve the astrometric higher-order coeffs for DECam early CP-processed data

[djschlegel]

We inherit all of the astrometric coefficients in the headers of the CP-processed frame, and only apply a scalar offset in RA and Dec for each CCD (as recorded in the ccds file). There's a description of this on the wiki here: https://desi.lbl.gov/trac/wiki/DecamLegacy/Reductions/Astrometry

The CP astrometric terms (stored as PV terms) were updated starting with CP v3.11 and later (Jan 11, 2017) using GAIA stars. As a result, any frames processed after that date pick up more accurate astrometric terms, and the astrometric residuals are indeed noticeably better. This is sort of shown in the plot below, showing the astrometry residuals vs. the date of observation (even though this is really a function of CP processing date).

A simple improvement that we could make is to ignore the PV coefficients in the CP image headers, and always use a more recent set of terms. This is 24 numbers per CCD (although 2 of those numbers, PV1_3 and PV2_3 are always set to zero).

Longer-term, it would be better to use these coefficients as determined by DES along with some of their other terms like for the tree rings, written up in Bernstein et al 2017 https://arxiv.org/abs/1703.01679 .

[djschlegel]

Attached is a FITS binary table with the PV coefficients used as the starting point for CP for all of the reductions in the past few years. As discussed, we should try using these with linear terms for DECam. A random subsample of images should show us whether this improves astrometry, especially for the earlier reductions that used coefficients that weren't as good. Solving for linear terms using the Tractor centroids should enforce more self-consistency on the astrometry.

[arjundey]

According to Frank Valdes, this is not going to work as well as we expect. It would be better to just reprocess the relevant < Sep 2016 frames and I will discuss this with him and report back. I am nervous about applying David's suggestion without running some thorough tests.

[fvaldes]

A simple check shows that all PV terms are tweaked by CP astrometric solution which is why one can't use a fix set of coefficients from some calibration library such as from DES. Remember that the astrometic solution is more than just the metrology of the camera, it must include refraction, any cold plate changes, and any errors in the telescope that might rotate the camera slightly.

[djschlegel]

We should definitely test on some random sampling of images, but my expectation is that this should improve even for the later data. These low-order terms are sufficient for the refraction components that the CP puts in, and by doing so using the Tractor centroids rather than the PSFex centroids, there’s more consitency on the astrometry.

Refraction (or color-dependent refraction) does appear to then be the next leading-order problem in astrometry, see attached.

[moustakas]

Pushing this issue to DR9, before which we'll hopefully have reprocessed all these images with the updated astrometry.

[moustakas]

@djschlegel can you regenerate your diagnostic plots and close this issue if we're happy with all the reprocessed astrometry?

[arjundey]

Please see the attached plot, which shows the astrometric error as a function of MJD for the N4 CCD on DECam for z-band filter in the DR9 annotated CCD file. If you compare this plot to the one on the github issue page here: https://github.com/legacysurvey/legacypipe/issues/307 , DR9 looks much better. The “jump” at (MJD-57000)~600 is gone, and the fraction of points with |astrometric error| > 0.1 has reduced from ~50% (DR8) to ~4 to 5% (for DR9), and only 0.01% if the zeropoints suggest that it was “photometric”). There are occasional nights where the rms seems higher, and there is some room for further improvement, but I think we should declare victory on this issue now.

[moustakas]

Fantastic, @arjundey, thank you for looking into this issue!