search

lofar-astron / DP3

DP3: streaming processing pipeline for radio interferometric data
GNU General Public License v3.0
15 stars 10 forks source link

Fulljones applycal flags international stations #311

Closed tikk3r closed 4 years ago

tikk3r commented 4 years ago

For widefield long baseline stuff we use killMS solutions at one point, which are fulljones. Before applying them they are converted to h5parm and the international stations are added with amplitudes of 1 and phases of 0, as they don't exist in the killMS solutions.

The resulting h5parm looks ok to me (amp and phase are 1 and 0 for international stations), but applying it results in all baselines with international stations getting flagged. They do not get flagged if just XX and YY are taken from the killMS solutions and processed the same way and applied as normal diagonal phase and amplitude solutions.

Is there a detail about fulljones solutions that I'm perhaps overlooking?

tammojan commented 4 years ago

I don't think there are details. If you could provide a test MS + h5parm, we could dive into it.

tikk3r commented 4 years ago

I have put a small (untouched) chunk of the MS, the relevant h5parm and parset here: https://public.spider.surfsara.nl/project/lofarvwf/fulljones_mwe.tar Applying the h5parm results in DP3 reporting 80% of the data for international stations being flagged, which I also see in rfigui (international stations completely flagged).

tikk3r commented 4 years ago

After insightful discussion with @rvweeren, it turns out this is not an issue with DP3, but a misunderstanding of the cross-hand gains on my part. When adding dummy entries the gain amplitudes for XY and YX should be set to 0, not 1.

tammojan commented 4 years ago

That is correct. Thanks for reporting back. Sorry, I hadn't noticed it yet in the files you sent. Glad it works now!