search

caracal-pipeline / caracal

Containerized Automated Radio Astronomy Calibration (CARACal) pipeline
GNU General Public License v2.0
28 stars 6 forks source link

OTF applycal not working with multiple targets #1454

Closed paoloserra closed 1 year ago

paoloserra commented 1 year ago 
    This is the syntax used for the callib files in casa: https://casa.nrao.edu/casadocs/casa-5.1.0/reference-material/cal-library-syntax

I think we could simply include all the caltables in the callib files (also the polcal solutions are not the same for all of the calibrators) adding the "field" keyword which specified over which fields the caltable has to be applied.

I am wondering if it could be better to directly write a txt file...

Originally posted by @francescaLoi in https://github.com/caracal-pipeline/caracal/issues/1280#issuecomment-726142966

Since this we have been adding the field in the callib txt file. That works when you've got one field only. It does not when you have more than one, because we only write the first field.

paoloserra commented 1 year ago

The relevant lines in the transform worker are:

https://github.com/caracal-pipeline/caracal/blob/63019ef007277d8dea245b4070ef3950c6169684/caracal/workers/transform_worker.py#L113

and then

https://github.com/caracal-pipeline/caracal/blob/63019ef007277d8dea245b4070ef3950c6169684/caracal/workers/transform_worker.py#L133

Everything non-pol works if the latter is replaced with

output_fields=[''],

But I suppose that would break polcal. @francescaLoi @o-smirnov could you please comment on this?

paoloserra commented 1 year ago

Alternatively, I suppose there is a callib-friendly way of changing L113 above to include all targets of the MS in output_fields instead of just the first one?

o-smirnov commented 1 year ago

It might be as simple as moving the resolve_calibration_library call into this loop here:

https://github.com/caracal-pipeline/caracal/blob/63019ef007277d8dea245b4070ef3950c6169684/caracal/workers/transform_worker.py#L156

where we already have a single target.

paoloserra commented 1 year ago

Indeed, I considered that, but I have no good understanding of what's needed for polcal. For regular calibration the solution above works fine, too, and it's simpler.

henedler commented 1 year ago

I have been puzzling about exactly this for the last day. I'll give the workaround suggested by Paolo a try.

o-smirnov commented 1 year ago

@Athanaseus you also want to take a look at this.

For regular calibration the solution above works fine, too, and it's simpler.

Simpler doesn't always mean more correct. :) Setting output_fields=[''] means "return cal tables appropriate for default targets". This should be the same as specifying a target by name. However, the point of the resolve_calibration_library() interface is to let that function decide what tables are most appropriate for a particular field. I'm thinking about future support of cases where we have several gain calibrators and several targets in the MS.

paoloserra commented 1 year ago

understood

henedler commented 1 year ago

Hm, this did not work for me I fear. I replaced in line 133 as suggested by Paolo: output_fields=['']. Then I deleted the corrected target field measurement sets and I continued caracal from the transform-target step on. Is this not sufficient and I need to re-run the whole thing?

This is what the transform-step logs for the otf-applycal:

MSTransformManager::generateIterator    OTF calibration activated, using calibration record spec to generate iterator
Calibrater::    Arranging to calibrate MS: /stimela_mount/input/1662797070_sdp_l0.ms
OldCalibrater::setcallib2(callib)       Arranging to APPLY:
        .   B Jones: table=/stimela_mount/output/caltables/mcvs-1662797070_sdp_l0-kgb_primary.B1 (by cal library) calWt=false
        .   0:
+            MS: obs= fld= intent= spw=
+            CT: tinterp=linear finterp=linear
+                obsmap=[]         fldmap=[1, 1, 1, 1, 1, 1, 1, 1]
+                spwmap=[]         antmap=[]
OldCalibrater::setcallib2(callib)       Arranging to APPLY:
        .   K Jones: table=/stimela_mount/output/caltables/mcvs-1662797070_sdp_l0-kgb_primary.K1 (by cal library) calWt=false
        .   0:
+            MS: obs= fld= intent= spw=
+            CT: tinterp=nearest finterp=linear
+                obsmap=[]         fldmap=[1, 1, 1, 1, 1, 1, 1, 1]
+                spwmap=[]         antmap=[]
OldCalibrater::setcallib2(callib)       Arranging to APPLY:
        .   G Jones: table=/stimela_mount/output/caltables/mcvs-1662797070_sdp_l0-kgb_secondary.F0 (by cal library) calWt=false
        .   0:
+            MS: obs= fld= intent= spw=
+            CT: tinterp=linear finterp=linear
+                obsmap=[]         fldmap=[0, 0, 0, 0, 0, 0, 0, 1]
+                spwmap=[]         antmap=[]
OldCalibrater::setcallib2(callib)       The following calibration terms are arranged for apply:
OldCalibrater::setcallib2(callib)       .   B Jones: table=/stimela_mount/output/caltables/mcvs-1662797070_sdp_l0-kgb_primary.B1 (by cal library) calWt=false
OldCalibrater::setcallib2(callib)       .   K Jones: table=/stimela_mount/output/caltables/mcvs-1662797070_sdp_l0-kgb_primary.K1 (by cal library) calWt=false
OldCalibrater::setcallib2(callib)       .   G Jones: table=/stimela_mount/output/caltables/mcvs-1662797070_sdp_l0-kgb_secondary.F0 (by cal library) calWt=false

Should I be able to see if things are going wrong here? "Field 1" is the one I want to correct.

paoloserra commented 1 year ago

Can you post the full log?

henedler commented 1 year ago

Sure, here you go: log-caracal.txt

o-smirnov commented 1 year ago

Also, if you can post the calibration libraries please, i.e. the .txt and .yml files under caltables in your output directory.

henedler commented 1 year ago

Ah damn, I deleted those in the meantime, sorry. I'll post them once I am at the step again.

paoloserra commented 1 year ago

Shame.

I deleted the corrected target field measurement sets and I continued caracal from the transform-target step on. Is this not sufficient and I need to re-run the whole thing?

That is fine. You don't need to re-run crosscal. In fact, you don't even need to delete the corrected MS, CARACal will take care of that.

I'm a little confused by your log. The initial transform and crosscal are enabled but are not running?

Anyway, the OTFcal part of the log looks fine. Have you looked at the first continuum image? Does that still show those scrambled phases?

BTW, in case you had not realised, you could leave the target unset in the yml file, and CARACal will loop through all targets.

henedler commented 1 year ago

I'm a little confused by your log. The initial transform and crosscal are enabled but are not running?

Yes, I used the flag --sw transform__2 to continue running from the troublesome step on after doing the change to the code.

Anyway, the OTFcal part of the log looks fine. Have you looked at the first continuum image? Does that still show those scrambled phases?

Yes, it looks like the phases are crazy, like before.

paoloserra commented 1 year ago

OK.

The problem I was having should show as in the example below:

Here's the OTF applycal log for the first target in the MS, where the calibration is applied alright and the phases look fine.

# 2022-11-08 10:13:02   INFO    mstransform::::+        ##########################################
# 2022-11-08 10:13:02   INFO    mstransform::::+        ##### Begin Task: mstransform        #####
# 2022-11-08 10:13:02   INFO    mstransform:::: mstransform(vis="/stimela_mount/input/1662968476_sdp_l0.ms",outputvis="/stimela_mount/msdir/1662968476_sdp_l0-virgo065-corr.ms",createmms=False,separationaxis="auto",numsubms="auto",
# 2022-11-08 10:13:02   INFO    mstransform::::+                tileshape=[0],field="virgo065",spw="",scan="",antenna="",
# 2022-11-08 10:13:02   INFO    mstransform::::+                correlation="",timerange="",intent="",array="",uvrange="",
# 2022-11-08 10:13:02   INFO    mstransform::::+                observation="",feed="",datacolumn="corrected",realmodelcol=False,keepflags=True,
# 2022-11-08 10:13:02   INFO    mstransform::::+                usewtspectrum=True,combinespws=False,chanaverage=False,chanbin=1,hanning=False,
# 2022-11-08 10:13:02   INFO    mstransform::::+                regridms=False,mode="channel",nchan=-1,start=0,width=1,
# 2022-11-08 10:13:02   INFO    mstransform::::+                nspw=1,interpolation="linear",phasecenter="",restfreq="",outframe="",
# 2022-11-08 10:13:02   INFO    mstransform::::+                veltype="radio",preaverage=False,timeaverage=False,timebin="",timespan="",
# 2022-11-08 10:13:02   INFO    mstransform::::+                maxuvwdistance=0.0,docallib=True,callib="/stimela_mount/output/caltables/callib-oct-1662968476_sdp_l0-1gc-transform.txt",douvcontsub=False,fitspw="",
# 2022-11-08 10:13:02   INFO    mstransform::::+                fitorder=0,want_cont=False,denoising_lib=True,nthreads=1,niter=1,
# 2022-11-08 10:13:02   INFO    mstransform::::+                disableparallel=False,ddistart=-1,taql="",monolithic_processing=False,reindex=True)
# 2022-11-08 10:13:02   INFO    mstransform:::: Parse docallib parameters
# 2022-11-08 10:13:02   INFO    MSTransformManager::parseMsSpecParams   Input file name is /stimela_mount/input/1662968476_sdp_l0.ms
# 2022-11-08 10:13:02   INFO    MSTransformManager::parseMsSpecParams   Data column is CORRECTED
# 2022-11-08 10:13:02   INFO    MSTransformManager::parseMsSpecParams   Output file name is /stimela_mount/msdir/1662968476_sdp_l0-virgo065-corr.ms
# 2022-11-08 10:13:02   INFO    MSTransformManager::parseMsSpecParams   Re-index is enabled
# 2022-11-08 10:13:02   INFO    MSTransformManager::parseMsSpecParams   WEIGHT_SPECTRUM will be written in output MS
# 2022-11-08 10:13:02   INFO    MSTransformManager::parseMsSpecParams   Tile shape is [0]
# 2022-11-08 10:13:02   INFO    MSTransformManager::parseDataSelParams  field selection is virgo065
# 2022-11-08 10:13:02   INFO    MSTransformManager::parseCalParams      Calibration is activated
# 2022-11-08 10:13:02   INFO    MSTransformManager::colCheckInfo        Adding DATA column to output MS from input virtual CORRECTED_DATA column
# 2022-11-08 10:13:02   INFO    MSTransformManager::initDataSelectionParams     Selected Fields Ids are [1]
# 2022-11-08 10:13:02   INFO    MSTransformManager::open        Select data
# 2022-11-08 10:13:02   INFO    MSTransformManager::createOutputMSStructure     Create output MS structure
# 2022-11-08 10:13:07   INFO    MSTransformDataHandler::makeSelection   552240 out of 3279810 rows are going to be considered due to the selection criteria.
# 2022-11-08 10:13:08   INFO    MSTransformManager::checkFillWeightSpectrum     Optional column WEIGHT_SPECTRUM found in input MS will be written to output MS
# 2022-11-08 10:13:08   INFO    MSTransformManager::generateIterator    OTF calibration activated, using calibration record spec to generate iterator
# 2022-11-08 10:13:09   INFO    Calibrater::    Arranging to calibrate MS: /stimela_mount/input/1662968476_sdp_l0.ms
# 2022-11-08 10:13:09   INFO    OldCalibrater::setcallib2(callib)       Arranging to APPLY:
# 2022-11-08 10:13:09   INFO            .   B Jones: table=/stimela_mount/output/caltables/oct-1662968476_sdp_l0-1gc_primary.B1 (by cal library) calWt=false
# 2022-11-08 10:13:09   INFO            .   0:
# 2022-11-08 10:13:09   INFO    +            MS: obs= fld=virgo065 intent= spw=
# 2022-11-08 10:13:09   INFO    +            CT: tinterp=linear finterp=linear
# 2022-11-08 10:13:09   INFO    +                obsmap=[]         fldmap=[1, 1, 1, 1, 1, 1, 1, 1]
# 2022-11-08 10:13:09   INFO    +                spwmap=[]         antmap=[]
...

And here is the case of the second target, where you see that OTF applycal fails because of the mismatch in the field.

# 2022-11-08 10:17:20   INFO    mstransform::::+        ##########################################
# 2022-11-08 10:17:20   INFO    mstransform::::+        ##### Begin Task: mstransform        #####
# 2022-11-08 10:17:20   INFO    mstransform:::: mstransform(vis="/stimela_mount/input/1662968476_sdp_l0.ms",outputvis="/stimela_mount/msdir/1662968476_sdp_l0-virgo044-corr.ms",createmms=False,separationaxis="auto",numsubms="auto",
# 2022-11-08 10:17:20   INFO    mstransform::::+                tileshape=[0],field="virgo044",spw="",scan="",antenna="",
# 2022-11-08 10:17:20   INFO    mstransform::::+                correlation="",timerange="",intent="",array="",uvrange="",
# 2022-11-08 10:17:20   INFO    mstransform::::+                observation="",feed="",datacolumn="corrected",realmodelcol=False,keepflags=True,
# 2022-11-08 10:17:20   INFO    mstransform::::+                usewtspectrum=True,combinespws=False,chanaverage=False,chanbin=1,hanning=False,
# 2022-11-08 10:17:20   INFO    mstransform::::+                regridms=False,mode="channel",nchan=-1,start=0,width=1,
# 2022-11-08 10:17:20   INFO    mstransform::::+                nspw=1,interpolation="linear",phasecenter="",restfreq="",outframe="",
# 2022-11-08 10:17:20   INFO    mstransform::::+                veltype="radio",preaverage=False,timeaverage=False,timebin="",timespan="",
# 2022-11-08 10:17:20   INFO    mstransform::::+                maxuvwdistance=0.0,docallib=True,callib="/stimela_mount/output/caltables/callib-oct-1662968476_sdp_l0-1gc-transform.txt",douvcontsub=False,fitspw="",
# 2022-11-08 10:17:20   INFO    mstransform::::+                fitorder=0,want_cont=False,denoising_lib=True,nthreads=1,niter=1,
# 2022-11-08 10:17:20   INFO    mstransform::::+                disableparallel=False,ddistart=-1,taql="",monolithic_processing=False,reindex=True)
# 2022-11-08 10:17:20   INFO    mstransform:::: Parse docallib parameters
# 2022-11-08 10:17:20   INFO    MSTransformManager::parseMsSpecParams   Input file name is /stimela_mount/input/1662968476_sdp_l0.ms
# 2022-11-08 10:17:20   INFO    MSTransformManager::parseMsSpecParams   Data column is CORRECTED
# 2022-11-08 10:17:20   INFO    MSTransformManager::parseMsSpecParams   Output file name is /stimela_mount/msdir/1662968476_sdp_l0-virgo044-corr.ms
# 2022-11-08 10:17:20   INFO    MSTransformManager::parseMsSpecParams   Re-index is enabled
# 2022-11-08 10:17:20   INFO    MSTransformManager::parseMsSpecParams   WEIGHT_SPECTRUM will be written in output MS
# 2022-11-08 10:17:20   INFO    MSTransformManager::parseMsSpecParams   Tile shape is [0]
# 2022-11-08 10:17:20   INFO    MSTransformManager::parseDataSelParams  field selection is virgo044
# 2022-11-08 10:17:20   INFO    MSTransformManager::parseCalParams      Calibration is activated
# 2022-11-08 10:17:20   INFO    MSTransformManager::colCheckInfo        Adding DATA column to output MS from input virtual CORRECTED_DATA column
# 2022-11-08 10:17:20   INFO    MSTransformManager::initDataSelectionParams     Selected Fields Ids are [2]
# 2022-11-08 10:17:20   INFO    MSTransformManager::open        Select data
# 2022-11-08 10:17:20   INFO    MSTransformManager::createOutputMSStructure     Create output MS structure
# 2022-11-08 10:17:20   INFO    MSTransformDataHandler::makeSelection   554010 out of 3279810 rows are going to be considered due to the selection criteria.
# 2022-11-08 10:17:21   INFO    MSTransformManager::checkFillWeightSpectrum     Optional column WEIGHT_SPECTRUM found in input MS will be written to output MS
# 2022-11-08 10:17:21   INFO    MSTransformManager::generateIterator    OTF calibration activated, using calibration record spec to generate iterator
# 2022-11-08 10:17:22   INFO    Calibrater::    Arranging to calibrate MS: /stimela_mount/input/1662968476_sdp_l0.ms
# 2022-11-08 10:17:22   INFO    OldCalibrater::setcallib2(callib)       Arranging to APPLY:
# 2022-11-08 10:17:22   INFO            .   B Jones: table=/stimela_mount/output/caltables/oct-1662968476_sdp_l0-1gc_primary.B1 (by cal library) calWt=false
# 2022-11-08 10:17:22   WARN            .   The following callib entry matches no data
# 2022-11-08 10:17:22   WARN    +       .   in the selected MS (1662968476_sdp_l0.ms) and will be ignored:
# 2022-11-08 10:17:22   WARN    +       .   0:
# 2022-11-08 10:17:22   WARN    +            MS: obs= fld=virgo065 intent= spw=
# 2022-11-08 10:17:22   WARN    +            CT: tinterp=linear finterp=linear
# 2022-11-08 10:17:22   WARN    +                obsmap=[]         fldmap=[1, 1, 1, 1, 1, 1, 1, 1]
# 2022-11-08 10:17:22   WARN    +                spwmap=[]         antmap=[]
...

If you don't see the warning above then OTF applycal is fine, at least as far as the bug reported in this issue goes.

henedler commented 1 year ago

Ok, indeed this warning I did not encounter before. Here the calibration libraries now: callib-mcvs-1662797070_sdp_l0-kgb.yml.txt callib-mcvs-1662797070_sdp_l0-kgb-transform__2.txt

o-smirnov commented 1 year ago

Looks fine to me -- I don't think OTF is your problem in this case.

paoloserra commented 1 year ago

I don't think so either.

Screenshot 2022-11-10 at 11 20 58

Do you see something like this for virgo048? (It's the dirty image.)

henedler commented 1 year ago

This is what I get as virgo048_0-MFS-image, you can see the artifacts of M87 but not really the source itself (FoV: 1.5x1.5deg). This is, as expected, very similar to imaging the data-column of the virgo048-corrected.MS directly after the transform__2 step. Similar results I also get for the other field I tried. grafik

And this is what I get as image of the secondary calibrator, this still looks fine to me. grafik

It could be that something went wrong in the pipeline by starting/re-starting and changing some of the parameters in-between (e.g. using all targets / just virgo048), or, alternatively, it is related to using the "nyq" method for the SPECTRAL_WEIGHTS (see https://github.com/caracal-pipeline/caracal/issues/1451#issuecomment-1302200235).

Do you have in mind any tests I could to to pin down the problem? Thanks a lot for the help already.

paoloserra commented 1 year ago

Could you please try field 113? It's an easier field.

henedler commented 1 year ago

This is for field 129, most distant from M87, still the same problem.

grafik

I also tried virgo065 from the second pilot, again with the same messed up phases. After a few iterations, sources indeed start to appear, so the data is probably fine but the initial phases are quite off. grafik

paoloserra commented 1 year ago

There's definitely something wrong there.

This is what I get for field 129 in the freq range 1395 - 1437 MHz:

after cross-cal

Screenshot 2022-11-14 at 12 45 26

after self-cal

Screenshot 2022-11-14 at 12 46 46

It's not wonderful (and I don't need it to be), but it looks right.

henedler commented 1 year ago

Indeed ever so slightly better than what I am getting ;). Would you mind sharing your parameter set with me so I can compare them? And you are using the current master (except for the one change you described above)?

paoloserra commented 1 year ago

I'm on this branch: https://github.com/caracal-pipeline/caracal/tree/callib-multitarget (but yes, it's just that one change compared to master)

.yml files attached. I run them in the order:

configs.tar.gz

henedler commented 1 year ago

I ran a test where I just applied the K and B solutions, and not the gaincal. Indeed, there I get a reasonable image (below: dirty/restored). So the point where things go wrong for me should be quite constrained. One thing I was wondering - for the gaincal solutions, I always see a "fldmap" that looks like this: fldmap=[0, 0, 0, 0, 0, 0, 0, 1]. I wonder what that means, shouldn't this all be the same index (the one of the gaincal?). However, I see the same also in an archival MeerKat observation where the calibration works fine.

grafik
paoloserra commented 1 year ago

I'm not sure about fldmap. Have you looked at the gain plots? (G and F)

o-smirnov commented 1 year ago

I think the last entry in fldmap corresponds to what is used for the bpcal, so it uses itself rather than the gaincal.

henedler commented 1 year ago

For the record: I pinned down the issue, it was not related to caracal. I assumed the data set I had at hand was the raw data, when actually the calibrator solutions were already applied by the observatory (only to the target field, not to the cals, so the solutions appeared normal). So basically I was "corrupting" the already corrected data by correcting them a second time...

I re-downloaded the raw data, now running caracal gives the expected results.

paoloserra commented 1 year ago

phew!

o-smirnov commented 1 year ago

Awesome! Now I wish someone would solve https://github.com/caracal-pipeline/caracal/issues/1456 :)

paoloserra commented 1 year ago

Well, we still need to solve #1454 too!!!! :)

Athanaseus commented 1 year ago

@paoloserra can you please try the fix in branch apply_multiple_targets.

Log section of mstransform now looks like:

2022-12-06 07:40:46     INFO    mstransform:::: mstransform(vis="/stimela_mount/input/1543664600_sdp_l0.ms",outputvis="/stimela_mount/msdir/1543664600_sdp_l0-T16R02C04-corr.ms",createmms=False,separationaxis="auto",numsubms="auto",
2022-12-06 07:40:46     INFO    mstransform::::+                tileshape=[0],field="T16R02C04",spw="",scan="",antenna="",
2022-12-06 07:40:46     INFO    mstransform::::+                correlation="",timerange="",intent="",array="",uvrange="",
2022-12-06 07:40:46     INFO    mstransform::::+                observation="",feed="",datacolumn="corrected",realmodelcol=False,keepflags=True,
2022-12-06 07:40:46     INFO    mstransform::::+                usewtspectrum=True,combinespws=False,chanaverage=False,chanbin=1,hanning=False,
2022-12-06 07:40:46     INFO    mstransform::::+                regridms=False,mode="channel",nchan=-1,start=0,width=1,
2022-12-06 07:40:46     INFO    mstransform::::+                nspw=1,interpolation="linear",phasecenter="",restfreq="",outframe="",
2022-12-06 07:40:46     INFO    mstransform::::+                veltype="radio",preaverage=False,timeaverage=False,timebin="",timespan="",
2022-12-06 07:40:46     INFO    mstransform::::+                maxuvwdistance=0.0,docallib=True,callib="/stimela_mount/output/caltables/callib-mgps-hii-regions-1543664600-ar2-1543664600_sdp_l0-1kgb-transform__2.txt",douvcontsub=False,fitspw="",
2022-12-06 07:40:46     INFO    mstransform::::+                fitorder=0,want_cont=False,denoising_lib=True,nthreads=1,niter=1,
2022-12-06 07:40:46     INFO    mstransform::::+                disableparallel=False,ddistart=-1,taql="",monolithic_processing=False,reindex=True)
2022-12-06 07:40:46     INFO    mstransform:::: Parse docallib parameters
2022-12-06 07:40:46     INFO    MSTransformManager::parseMsSpecParams   Input file name is /stimela_mount/input/1543664600_sdp_l0.ms
2022-12-06 07:40:46     INFO    MSTransformManager::parseMsSpecParams   Data column is CORRECTED
2022-12-06 07:40:46     INFO    MSTransformManager::parseMsSpecParams   Output file name is /stimela_mount/msdir/1543664600_sdp_l0-T16R02C04-corr.ms
2022-12-06 07:40:46     INFO    MSTransformManager::parseMsSpecParams   Re-index is enabled
2022-12-06 07:40:46     INFO    MSTransformManager::parseMsSpecParams   WEIGHT_SPECTRUM will be written in output MS
2022-12-06 07:40:46     INFO    MSTransformManager::parseMsSpecParams   Tile shape is [0]
2022-12-06 07:40:46     INFO    MSTransformManager::parseDataSelParams  field selection is T16R02C04
2022-12-06 07:40:46     INFO    MSTransformManager::parseCalParams      Calibration is activated
2022-12-06 07:40:46     INFO    MSTransformManager::colCheckInfo        Adding DATA column to output MS from input virtual CORRECTED_DATA column
2022-12-06 07:40:46     INFO    MSTransformManager::initDataSelectionParams     Selected Fields Ids are [3]
2022-12-06 07:40:46     INFO    MSTransformManager::open        Select data
2022-12-06 07:40:46     INFO    MSTransformManager::createOutputMSStructure     Create output MS structure
2022-12-06 07:40:46     INFO    MSTransformDataHandler::makeSelection   249612 out of 3496459 rows are going to be considered due to the selection criteria.
2022-12-06 07:40:46     INFO    MSTransformManager::checkFillWeightSpectrum     Optional column WEIGHT_SPECTRUM found in input MS will be written to output MS
2022-12-06 07:40:46     INFO    MSTransformManager::generateIterator    OTF calibration activated, using calibration record spec to generate iterator
2022-12-06 07:40:47     INFO    Calibrater::    Arranging to calibrate MS: /stimela_mount/input/1543664600_sdp_l0.ms
2022-12-06 07:40:47     INFO    OldCalibrater::setcallib2(callib)       Arranging to APPLY:
2022-12-06 07:40:47     INFO            .   B Jones: table=/stimela_mount/output/caltables/mgps-hii-regions-ar4-1543664600_sdp_l0-1kgb_primary.B1 (by cal library) calWt=false
2022-12-06 07:40:47     INFO            .   0:
2022-12-06 07:40:47     INFO    +            MS: obs= fld=T16R02C02,T16R04C04,T16R03C01,T16R04C02,T16R04C06,T16R03C03,T16R02C04,T16R02C06,T16R03C05 intent= spw=
2022-12-06 07:40:47     INFO    +            CT: tinterp=linear finterp=linear
2022-12-06 07:40:47     INFO    +                obsmap=[]         fldmap=[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
2022-12-06 07:40:47     INFO    +                spwmap=[]         antmap=[]
2022-12-06 07:40:47     INFO    OldCalibrater::setcallib2(callib)       Arranging to APPLY:
2022-12-06 07:40:47     INFO            .   K Jones: table=/stimela_mount/output/caltables/mgps-hii-regions-ar4-1543664600_sdp_l0-1kgb_primary.K2 (by cal library) calWt=false
2022-12-06 07:40:47     INFO            .   0:
2022-12-06 07:40:47     INFO    +            MS: obs= fld=T16R02C02,T16R04C04,T16R03C01,T16R04C02,T16R04C06,T16R03C03,T16R02C04,T16R02C06,T16R03C05 intent= spw=
2022-12-06 07:40:47     INFO    +            CT: tinterp=nearest finterp=linear
2022-12-06 07:40:47     INFO    +                obsmap=[]         fldmap=[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
2022-12-06 07:40:47     INFO    +                spwmap=[]         antmap=[]
2022-12-06 07:40:47     INFO    OldCalibrater::setcallib2(callib)       Arranging to APPLY:
2022-12-06 07:40:47     INFO            .   G Jones: table=/stimela_mount/output/caltables/mgps-hii-regions-ar4-1543664600_sdp_l0-1kgb_secondary.F0 (by cal library) calWt=false
2022-12-06 07:40:47     INFO            .   0:
2022-12-06 07:40:47     INFO    +            MS: obs= fld=T16R02C02,T16R04C04,T16R03C01,T16R04C02,T16R04C06,T16R03C03,T16R02C04,T16R02C06,T16R03C05 intent= spw=

NB: fld list all targets

And previously, it only listed index 0 target field only.

2022-11-15 09:11:13     INFO    mstransform:::: mstransform(vis="/stimela_mount/input/1543664600_sdp_l0.ms",outputvis="/stimela_mount/msdir/1543664600_sdp_l0-T16R02C04-corr.ms",createmms=False,separationaxis="auto",numsubms="auto",
2022-11-15 09:11:13     INFO    mstransform::::+                tileshape=[0],field="T16R02C04",spw="",scan="",antenna="",
2022-11-15 09:11:13     INFO    mstransform::::+                correlation="",timerange="",intent="",array="",uvrange="",
2022-11-15 09:11:13     INFO    mstransform::::+                observation="",feed="",datacolumn="corrected",realmodelcol=False,keepflags=True,
2022-11-15 09:11:13     INFO    mstransform::::+                usewtspectrum=True,combinespws=False,chanaverage=False,chanbin=1,hanning=False,
2022-11-15 09:11:13     INFO    mstransform::::+                regridms=False,mode="channel",nchan=-1,start=0,width=1,
2022-11-15 09:11:13     INFO    mstransform::::+                nspw=1,interpolation="linear",phasecenter="",restfreq="",outframe="",
2022-11-15 09:11:13     INFO    mstransform::::+                veltype="radio",preaverage=False,timeaverage=False,timebin="",timespan="",
2022-11-15 09:11:13     INFO    mstransform::::+                maxuvwdistance=0.0,docallib=True,callib="/stimela_mount/output/caltables/callib-mgps-hii-regions-1543664600-ar2-1543664600_sdp_l0-1kgb-transform__2.txt",douvcontsub=False,fitspw="",
2022-11-15 09:11:13     INFO    mstransform::::+                fitorder=0,want_cont=False,denoising_lib=True,nthreads=1,niter=1,
2022-11-15 09:11:13     INFO    mstransform::::+                disableparallel=False,ddistart=-1,taql="",monolithic_processing=False,reindex=True)
2022-11-15 09:11:13     INFO    mstransform:::: Parse docallib parameters
2022-11-15 09:11:13     INFO    MSTransformManager::parseMsSpecParams   Input file name is /stimela_mount/input/1543664600_sdp_l0.ms
2022-11-15 09:11:13     INFO    MSTransformManager::parseMsSpecParams   Data column is CORRECTED
2022-11-15 09:11:13     INFO    MSTransformManager::parseMsSpecParams   Output file name is /stimela_mount/msdir/1543664600_sdp_l0-T16R02C04-corr.ms
2022-11-15 09:11:13     INFO    MSTransformManager::parseMsSpecParams   Re-index is enabled
2022-11-15 09:11:13     INFO    MSTransformManager::parseMsSpecParams   WEIGHT_SPECTRUM will be written in output MS
2022-11-15 09:11:13     INFO    MSTransformManager::parseMsSpecParams   Tile shape is [0]
2022-11-15 09:11:13     INFO    MSTransformManager::parseDataSelParams  field selection is T16R02C04
2022-11-15 09:11:13     INFO    MSTransformManager::parseCalParams      Calibration is activated
2022-11-15 09:11:13     INFO    MSTransformManager::colCheckInfo        Adding DATA column to output MS from input virtual CORRECTED_DATA column
2022-11-15 09:11:13     INFO    MSTransformManager::initDataSelectionParams     Selected Fields Ids are [3]
2022-11-15 09:11:13     INFO    MSTransformManager::open        Select data
2022-11-15 09:11:13     INFO    MSTransformManager::createOutputMSStructure     Create output MS structure
2022-11-15 09:11:14     INFO    MSTransformDataHandler::makeSelection   249612 out of 3496459 rows are going to be considered due to the selection criteria.
2022-11-15 09:11:14     INFO    MSTransformManager::checkFillWeightSpectrum     Optional column WEIGHT_SPECTRUM found in input MS will be written to output MS
2022-11-15 09:11:14     INFO    MSTransformManager::generateIterator    OTF calibration activated, using calibration record spec to generate iterator
2022-11-15 09:11:15     INFO    Calibrater::    Arranging to calibrate MS: /stimela_mount/input/1543664600_sdp_l0.ms
2022-11-15 09:11:15     INFO    OldCalibrater::setcallib2(callib)       Arranging to APPLY:
2022-11-15 09:11:15     INFO            .   B Jones: table=/stimela_mount/output/caltables/mgps-hii-regions-ar4-1543664600_sdp_l0-1kgb_primary.B1 (by cal library) calWt=false
2022-11-15 09:11:15     WARN            .   The following callib entry matches no data
2022-11-15 09:11:15     WARN    +       .   in the selected MS (1543664600_sdp_l0.ms) and will be ignored:
2022-11-15 09:11:15     WARN    +       .   0:
2022-11-15 09:11:15     WARN    +            MS: obs= fld=T16R02C02 intent= spw=
2022-11-15 09:11:15     WARN    +            CT: tinterp=linear finterp=linear
2022-11-15 09:11:15     WARN    +                obsmap=[]         fldmap=[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
2022-11-15 09:11:15     WARN    +                spwmap=[]         antmap=[]
2022-11-15 09:11:15     INFO    OldCalibrater::setcallib2(callib)       Arranging to APPLY:
2022-11-15 09:11:15     INFO            .   K Jones: table=/stimela_mount/output/caltables/mgps-hii-regions-ar4-1543664600_sdp_l0-1kgb_primary.K2 (by cal library) calWt=false
2022-11-15 09:11:15     WARN            .   The following callib entry matches no data
2022-11-15 09:11:15     WARN    +       .   in the selected MS (1543664600_sdp_l0.ms) and will be ignored:
2022-11-15 09:11:15     WARN    +       .   0:
2022-11-15 09:11:15     WARN    +            MS: obs= fld=T16R02C02 intent= spw=
2022-11-15 09:11:15     WARN    +            CT: tinterp=nearest finterp=linear
2022-11-15 09:11:15     WARN    +                obsmap=[]         fldmap=[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
2022-11-15 09:11:15     WARN    +                spwmap=[]         antmap=[]
2022-11-15 09:11:15     INFO    OldCalibrater::setcallib2(callib)       Arranging to APPLY:
2022-11-15 09:11:15     INFO            .   G Jones: table=/stimela_mount/output/caltables/mgps-hii-regions-ar4-1543664600_sdp_l0-1kgb_secondary.F0 (by cal library) calWt=false
2022-11-15 09:11:15     WARN            .   The following callib entry matches no data
Athanaseus commented 1 year ago

new calib:

caltable="/stimela_mount/output/caltables/mgps-hii-regions-ar4-1543664600_sdp_l0-1kgb_secondary.F0" calwt=False tinterp='linear' finterp='linear' fldmap='nearest' field='T16R02C02,T16R04C04,T16R03C01,T16R04C02,T16R04C06,T16R03C03,T16R02C04,T16R02C06,T16R03C05' spwmap=0
caltable="/stimela_mount/output/caltables/mgps-hii-regions-ar4-1543664600_sdp_l0-1kgb_primary.B1" calwt=False tinterp='linear' finterp='linear' fldmap='nearest' field='T16R02C02,T16R04C04,T16R03C01,T16R04C02,T16R04C06,T16R03C03,T16R02C04,T16R02C06,T16R03C05' spwmap=0
caltable="/stimela_mount/output/caltables/mgps-hii-regions-ar4-1543664600_sdp_l0-1kgb_primary.K2" calwt=False tinterp='nearest' finterp='linear' fldmap='nearest' field='T16R02C02,T16R04C04,T16R03C01,T16R04C02,T16R04C06,T16R03C03,T16R02C04,T16R02C06,T16R03C05' spwmap=0

old calib:

caltable="/stimela_mount/output/caltables/mgps-hii-regions-ar4-1543664600_sdp_l0-1kgb_secondary.F0" calwt=False tinterp='linear' finterp='linear' fldmap='nearest' field='T16R02C02' spwmap=0
caltable="/stimela_mount/output/caltables/mgps-hii-regions-ar4-1543664600_sdp_l0-1kgb_primary.B1" calwt=False tinterp='linear' finterp='linear' fldmap='nearest' field='T16R02C02' spwmap=0
caltable="/stimela_mount/output/caltables/mgps-hii-regions-ar4-1543664600_sdp_l0-1kgb_primary.K2" calwt=False tinterp='nearest' finterp='linear' fldmap='nearest' field='T16R02C02' spwmap=0