To recap, smoothing calibration solutions introduces unusual (somewhat wavy) structure in the power spectrum residuals at low-k, and this structure looks to appear at delays around roughly twice the smoothing scale used. To diagnose this issue, some proposed tests are as follows:
Use gains with simple delay structure (e.g. a sine wave) to uncalibrate the visibilities and run the test again.
Using the same gains as in this test, re-do the smoothing step with different smoothing scales (suggested 50 ns and 1000 ns as a first attempt) and see how the residuals change.
Important to note is that the structure remains after performing an incoherent average over all times and another average over all baseline pairs. This may suggest that the same structure is appearing for every baseline at each time, but it is hard to say given how bright the foregrounds are at the k-modes of interest. So, it may be worthwhile to see if the structure in the residuals is baseline- or time-dependent. Alternatively, the suggested tests may be performed on a single file (~10 integrations), with all other parts of the analysis left unchanged.
Details regarding
smooth_cal
follow-up test:To recap, smoothing calibration solutions introduces unusual (somewhat wavy) structure in the power spectrum residuals at low-k, and this structure looks to appear at delays around roughly twice the smoothing scale used. To diagnose this issue, some proposed tests are as follows:
Important to note is that the structure remains after performing an incoherent average over all times and another average over all baseline pairs. This may suggest that the same structure is appearing for every baseline at each time, but it is hard to say given how bright the foregrounds are at the k-modes of interest. So, it may be worthwhile to see if the structure in the residuals is baseline- or time-dependent. Alternatively, the suggested tests may be performed on a single file (~10 integrations), with all other parts of the analysis left unchanged.
@jsdillon would you like to add anything?
Originally posted by @r-pascua in https://github.com/HERA-Team/hera-validation/pull/32#issuecomment-582284671