Severely flagged gains generated by the `tec_and_offset` solver

rcyndie / QuartiCal

CubiCal, but with greater power.

MIT License

0 stars 0 forks source link

Severely flagged gains generated by the `tec_and_offset` solver #5

Open rcyndie opened 4 months ago

rcyndie commented 4 months ago

Describe the bug The gains produced are severely flagged.

Data description (if applicable) Simulated MeerKAT ms with 1 timestamp, 1 correlation and 2000 channels.

To Reproduce Run the attached parset file and use the gains from the solver to produce phase plots.

Expected behavior No gain flags should be raised.

Screenshots The resulting phase plots and the following phase plots are obtained if I were to disable the gain flags in my plotting function, .

Version 0.2.2

landmanbester commented 4 months ago

Gain flagging is a necessary pain. You can at least avoid propagating the flags to the MS by setting output.flags=false. Curious that this gets triggered though. Is this with or without the initial estimate?

JSKenyon commented 4 months ago

Yeah - I mentioned to Cyndie that this seems like it might be a bug. I wouldn't expect gain flags to come out of the phase-only terms (the trend-based flagging is essentially disabled).

JSKenyon commented 4 months ago

I have the necessary info to reproduce (if possible), so I will take a stab at that tomorrow.

rcyndie commented 4 months ago

Gain flagging is a necessary pain. You can at least avoid propagating the flags to the MS by setting output.flags=false. Curious that this gets triggered though. Is this with or without the initial estimate?

Yes these plots are from the simulations with the initial estimates in this issue (Figure 9a). And the output.flags are set to False.

JSKenyon commented 4 months ago

Just closing the loop here. QuartiCal does overflag the gains in this case for the following reasons:

The mixture of delay and tec parameters is not currently supported i.e. the solver gets very confused by the mixed initial estimates.
QuartiCal flags solutions which are not adequately converged. This needs to be improved as it can easily lead to overflagging on parameterised terms with a low number of iterations.

As discussed, running the solver is probably unnecessary right now. I will work on implementing a delay_and_tec solver which will support including both sets of parameters.