apply_to_orig_MSes.py not calling all passed solution tables

[samdf96]

Hi John (and Patrick),

I have some data that I am re-processing through this automated pipeline. The produced script to apply the solutions back to the original MSes are not calling all the successful iterations of the self-calibration procedure, and was wondering if this was the intended result:

One of my fields have passed the following iterations:

inf_EB, 18.14s, 8.06s, 4.03 It has failed the following: int But then has passed the amplitude solve: inf_ap

However, looking at the output python script to apply them back to the MSes, this is what I see: applycal(vis="uid___A002_Xb60be4_X2c8f_targets.ms",gaintable=['Target_OrionBN_547252_02059_uid___A002_Xb60be4_X2c8f_targets.selfcal.ms_Band_3_inf_EB_0_p.g', 'Target_OrionBN_547252_02059_uid___A002_Xb60be4_X2c8f_targets.selfcal.ms_Band_3_4.03s_3_p.g', 'Target_OrionBN_547252_02059_uid___A002_Xb60be4_X2c8f_targets.selfcal.ms_Band_3_inf_ap_5_ap.g'],interp=['linearPD', 'linearPD', 'linearPD'], calwt=False,spwmap=[[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]], applymode="calflag",field="OrionBN-547252+02059",spw="0,1,2,3")

It is only calling inf_EB, 4.03s, and the inf_ap. I can see in the code base (Line 988 auto_selfcal.py) where this file is written, that it iterates through the keys of the self-calibration dictionary and the subsequently checks if that pass was a success. I can also see that it calls selfcal_library[target][band][vis]['gaintable_final'] for which tables to write out to the script. I've inspected this and I can see that indeed there are only three solution tables here.

Is it intended that only a certain portion of the solutions are applied back to the original MSes, or should all of those solution tables be called? I can see they exist in the directory, but aren't getting passed into this script to use. Perhaps there exists a step that condenses some of the solution tables together that I'm not seeing...

Thanks both, just want to make sure I can go ahead and get my own scripts ready to process some new data with this wonderful code base!

[jjtobin]

Hi Sam,

Thanks for your question. When calculating the gain solutions for the 18.14s, 8.06s, and 4.03s intervals, only the inf_EB solution interval is pre-applied in the call to gaincal. So 18.14s, 8.06s, 4.03s, and int all have the full phase correction and are not incremental with respect to the previous passed solution interval. Then for inf_ap, the inf_EB, and the final phase-only solution interval are pre-applied in their gaincal calls.

So when applying the final tables you only need those three gain tables and not the tables from all the phase solutions. Does that make sense?

John

[samdf96]

I'll have a think on this (I may send an email along separately)!

Good to know that the output scripts are correct though, so thanks for that.