1425 implements Filippo's u=0 flagging into CARACal.
I've made some tests and code edits for that PR and everything seemed OK, but I'm now getting some differences between the CARACal implementation and a previous, stand-alone script with the same algorithm. The latter is correct, so something is not right in CARACal.
I'm getting these differences on large Mk datasets, which are not the same I used for testing #1425.
The first comparison below shows that the selection of UV cells to flag is nearly identical in all scans. So that seems to be working fine in CARACal. (Not shown below, the FFT image stats are nearly identical in the two cases, so it makes sense that the UV cells selection is the same.)
Separate script
CARACal
The second comparison below shows that, however, the final flagging is quite different, with a lot more data flagged in CARACal in all scans -- even though the flagging stats reported in the plots say otherwise. Notably, a scan with no UV cells selected for flagging (3rd from top) ends up with a lot of flagged visibilities in CARACal.
Separate script
CARACal
Strangely, the full-MS plots look quite similar again, which I don't really understand.
Separate script
CARACal
So for now I'd recommend to not use CARACal's u=0 flagging. Hopefully we manage to fix this soon!
1425 implements Filippo's u=0 flagging into CARACal.
I've made some tests and code edits for that PR and everything seemed OK, but I'm now getting some differences between the CARACal implementation and a previous, stand-alone script with the same algorithm. The latter is correct, so something is not right in CARACal.
I'm getting these differences on large Mk datasets, which are not the same I used for testing #1425.
The first comparison below shows that the selection of UV cells to flag is nearly identical in all scans. So that seems to be working fine in CARACal. (Not shown below, the FFT image stats are nearly identical in the two cases, so it makes sense that the UV cells selection is the same.)
The second comparison below shows that, however, the final flagging is quite different, with a lot more data flagged in CARACal in all scans -- even though the flagging stats reported in the plots say otherwise. Notably, a scan with no UV cells selected for flagging (3rd from top) ends up with a lot of flagged visibilities in CARACal.
Strangely, the full-MS plots look quite similar again, which I don't really understand.
So for now I'd recommend to not use CARACal's u=0 flagging. Hopefully we manage to fix this soon!