Check phase shifts

[jmoldon]

I added a very interesting step to the pipeline: the possibility to perform phase shifts to any field in the MS. Some notes:

• This is not meant to correct a phase center (although it can be done easily).
• The idea is to add new fields to the MS pointing to additional directions.
• This is useful when you know there is a source in the field at a significant distance, but you still want to average the data.
• The output will be a MS with additional fields at different positions.
• This works similar to multi-phase center approach, with all phase centers in the same MS.

All the additional fields (phase centers) will have the same time stamps and scan numbers:

  Date        Timerange (UTC)          Scan  FldId FieldName             nRows     SpwIds   Average Interval(s)    ScanIntent
              11:18:01.5 - 11:24:59.5     4      3 1531+3533                50400  [0,1,2,3,4,5,6,7]  [0.995, 0.995, 0.995, 0.995, 0.995, 0.995, 0.995, 0.995] 
              11:18:01.5 - 11:24:59.5     4      4 1531+3533_pos1           50400  [0,1,2,3,4,5,6,7]  [0.995, 0.995, 0.995, 0.995, 0.995, 0.995, 0.995, 0.995] 
              11:25:02.5 - 11:27:58.5     5      0 1522+3144                21360  [0,1,2,3,4,5,6,7]  [0.989, 0.989, 0.989, 0.989, 0.989, 0.989, 0.989, 0.989] 
              11:28:01.5 - 11:34:58.5     6      3 1531+3533                50280  [0,1,2,3,4,5,6,7]  [0.995, 0.995, 0.995, 0.995, 0.995, 0.995, 0.995, 0.995] 
              11:28:01.5 - 11:34:58.5     6      4 1531+3533_pos1           50280  [0,1,2,3,4,5,6,7]  [0.995, 0.995, 0.995, 0.995, 0.995, 0.995, 0.995, 0.995] 
              11:35:01.5 - 11:37:58.5     7      0 1522+3144                21480  [0,1,2,3,4,5,6,7]  [0.989, 0.989, 0.989, 0.989, 0.989, 0.989, 0.989, 0.989] 
              11:38:01.5 - 11:44:59.5     8      3 1531+3533                50400  [0,1,2,3,4,5,6,7]  [0.995, 0.995, 0.995, 0.995, 0.995, 0.995, 0.995, 0.995] 
              11:38:01.5 - 11:44:59.5     8      4 1531+3533_pos1           50400  [0,1,2,3,4,5,6,7]  [0.995, 0.995, 0.995, 0.995, 0.995, 0.995, 0.995, 0.995] 
              11:45:02.5 - 11:47:58.5     9      0 1522+3144                21360  [0,1,2,3,4,5,6,7]  [0.989, 0.989, 0.989, 0.989, 0.989, 0.989, 0.989, 0.989] 
...
...
  ID   Code Name                RA               Decl           Epoch        nRows
  3         1531+3533           15:31:25.370000 +35.33.40.00000 J2000       502560
  4         1531+3533_pos1      15:31:25.000000 +35.33.45.00000 J2000       502560

This structure seems to work, and I don't see any problem with that. But I want to check that calibration, flagging, plotting, and final result make sense in a real data set. Feel free to experiment as much as you want.

Some additional notes:

• As explained in the documentation shift_field_pos, this does not work with MMS, because they cannot concatenate new data. You can use virtualconcat in that case, but it is not implemented in the pipeline now.

• If you just want to correct the phase center of a field but you don't want duplicated fields in your MS you can just not select the fields with the wrong phase center in the average_1 step, for example, then it will be discarded.

[jmoldon]

It worked perfectly.

This is an image produced by the pipeline of a source located 7.8 arcmin away from the phase center with a dataset that was averaged to 128chan/spw and 5 seconds integration time after one phase shift to this location:

The pipeline also produced an image for the science target source in the original field, because both phase centers were calibrated in parallel.

Note that shifting a field phase center using correlated data is not cheap: the shift of one field took 1.5h (for a 225GB, 24-h L-band data set), and the size of the unaveraged data set increased to 356 GB due to the shifted target data. However, the averaged data set size was only 33.5 GB, including model and corrected_data columns. So the final trade off is really good compared to keeping full resolution just to image one individual sources at the edge of the primary beam.

[jradcliffe5]

Looks good Javier. The only issue here is that sometimes the use of fixvis/UVFIX is that the astrometry can be off. We found that in the eMERGE data this could be as much as 60mas which could be fairly significant for eMERLIN resolutions especially @ C-band. We should probably put this as a caveat.

[jmoldon]

Ok, good point. Things that we may want to check for a source a few arcmin away of the phasecenter:

• Difference in astrometry between full-resolution data and FIXVIS shifted data.
• Difference in phase self-calibration between full-resolution data and FIXVIS shifted data.

With that information we will possibly distinguish three science cases and what they need:

1. Standard widefield imaging, already requires full-resolution.
2. In-beam source for astrometry. Is FIXVIS good enough, or full-res is required?
3. In-beam phase calibrator to improve selfcal. Does faint-source detection improve even when using FIXVIS?

This is more general than the pipeline development, so we should continue it in e-Merlin User Group meeting.