stsci-hack
commented
4 years ago Cross-matching problems does not, by itself, introduce the linear residuals in Y vs DY for each chip. Using a short exposure as the reference, these systematics were introduced to all exposures. However, using an exposure with longer exposure time relegated the systematics (with inverted slopes) to only the short exposure. This confirms that these residuals are due to CTE impacting the centering of the sources used for the alignment.
The RMS values from the fit can still be used to indicate whether this CTE-related systematics are affecting the fit, though, as the RMS in Y will be greater than the RMS in X (usually at least double).
Also, this supports the contention that short exposures should NOT be used as a reference unless all exposures are short.
Evaluation by M. Cara of the alignment residuals for j6d508020 (as an example) produced by the new tweakwcs-based pipeline alignment processing indicates that the residuals in Y from one chip to the next are most likely caused by an incorrect cross-matching of sources. Correcting the cross-matches used for the fit was enough to eliminate these systematic residuals across each chip based on his test cases.
This type of residuals can be determined by comparing the residuals in X and Y vs delta-X and delta-Y (like the residuals plots generated by tweakreg) and looking at the RMS of in each axis. If the RMS in X and Y are similar, then the cross-matching did the best job possible. If the RMS in Y is >2x RMS in X, then, there is most likely an issue with the cross-matching that would need to be corrected.
Work should be done to see how to integrate such a check into the alignment code for pipeline use, with a plan for dealing with those instances where the cross-matching indicates a problem. One possible solution would be iterate on the matching and fitting, starting with only the brightest 10% (or less depending on how many sources were identified) and using extremely liberal settings for the matching (searchrad > 250, tolerance > 5, for example), then performing another match and fit on that solution with more tailored matching parameters (searchrad < 10, tolerance < 2, for example).