e-merlin / eMERLIN_CASA_pipeline

This is CASA eMERLIN pipeline to calibrate data from the e-MERLIN array. Please fork the repository before making any changes and read the Coding Practices page in the wiki. Please add issues with the pipeline in the issues tab.
GNU General Public License v3.0
14 stars 11 forks source link

Averaging step fails since mstransformer sourcenames are wrong #73

Closed varenius closed 6 years ago

varenius commented 6 years ago

2017-11-04 10:57:36 | INFO | Starting pipeline 2017-11-04 10:57:36 | INFO | Running pipeline from: pipeline/ 2017-11-04 10:57:36 | INFO | Pipeline version: v0.6.1 2017-11-04 10:57:36 | INFO | Using github branch: master 2017-11-04 10:57:36 | INFO | github last commit: ad51f0d 2017-11-04 10:57:36 | INFO | This log uses UTC times 2017-11-04 10:57:36 | INFO | Inputs from file: inputs.txt 2017-11-04 10:57:36 | INFO | Parameters in inputs file: 2017-11-04 10:57:36 | INFO | fits_path : ./fitsidi/ 2017-11-04 10:57:36 | INFO | inbase : TSARP220_C_20171013 2017-11-04 10:57:36 | INFO | targets : 1534+2330 2017-11-04 10:57:36 | INFO | phscals : 1532+2344,1516+1932 2017-11-04 10:57:36 | INFO | fluxcal : 3C286 2017-11-04 10:57:36 | INFO | bpcal : OQ208 2017-11-04 10:57:36 | INFO | ptcal : 3C84 2017-11-04 10:57:36 | INFO | refant : Mk2 2017-11-04 10:57:36 | INFO | Lo_dropout_scans: 2017-11-04 10:57:36 | INFO | manual_flags_a : ./inputfg_a.flags 2017-11-04 10:57:36 | INFO | manual_flags_b : ./inputfg_b_avg.flags 2017-11-04 10:57:36 | INFO | run_importfits : 0 2017-11-04 10:57:36 | INFO | summary_weblog : 0 2017-11-04 10:57:36 | INFO | hanning : 0 2017-11-04 10:57:36 | INFO | ms2mms : 0 2017-11-04 10:57:36 | INFO | flag_0_aoflagger: 0 2017-11-04 10:57:36 | INFO | flag_1_apriori : 0 2017-11-04 10:57:36 | INFO | flag_2a_manual : 0 2017-11-04 10:57:36 | INFO | average_1 : 1 2017-11-04 10:57:36 | INFO | plot_data : 0 2017-11-04 10:57:36 | INFO | flag_2b_manual : 0 2017-11-04 10:57:36 | INFO | init_models : 0 2017-11-04 10:57:36 | INFO | bandpass_0 : 0 2017-11-04 10:57:36 | INFO | flag_3_tfcropBP : 0 2017-11-04 10:57:36 | INFO | delay : 0 2017-11-04 10:57:36 | INFO | gain_0_p_ap : 0 2017-11-04 10:57:36 | INFO | fluxscale : 0 2017-11-04 10:57:36 | INFO | bandpass_1_sp : 0 2017-11-04 10:57:36 | INFO | gain_1_amp_sp : 0 2017-11-04 10:57:36 | INFO | applycal_all : 0 2017-11-04 10:57:36 | INFO | flag_4_rflag : 0 2017-11-04 10:57:36 | INFO | plot_corrected : 0 2017-11-04 10:57:36 | INFO | weblog : 0 2017-11-04 10:57:36 | INFO | Loaded previous flags list from: ./flags.pkl 2017-11-04 10:57:36 | INFO | Targets: 1534+2330 2017-11-04 10:57:36 | INFO | Phasecals: 1532+2344,1516+1932 2017-11-04 10:57:36 | INFO | Fluxcal: 3C286 2017-11-04 10:57:36 | INFO | Bandpass: OQ208 2017-11-04 10:57:36 | INFO | Pointcal: 3C84 2017-11-04 10:57:36 | INFO | Start split 2017-11-04 10:57:37 | INFO | Sources in MS TSARP220_C_20171013.ms: 1516+1932,1407+284,0319+415,1532+2344,1331+305,1534+2330 2017-11-04 10:57:37 | WARNING | Fields 3C84,3C286,OQ208 not present in MS but listed in inputs file. 2017-11-04 10:57:37 | WARNING | All fields will be included in the averaged MS. 2017-11-04 10:57:37 | INFO | Input MS: TSARP220_C_20171013.ms 2017-11-04 10:57:37 | INFO | Output MS: TSARP220_C_20171013_avg.ms 2017-11-04 10:57:37 | INFO | width=4, timebin=2s 2017-11-04 10:57:37 | INFO | Fields: 1,5,1,6,+,1,9,3,2,,,1,4,0,7,+,2,8,4,,,0,3,1,9,+,4,1,5,,,1,5,3,2,+,2,3,4,4,,,1,3,3,1,+,3,0,5,,,1,5,3,4,+,2,3,3,0 2017-11-04 10:57:37 | INFO | Data column: data 2017-11-04 10:57:37 SEVERE mstransformer:: (file ../../tools/mstransformer/mstransformer_cmpt.cc, line 32)Exception Reported: Field Expression: No match found for name "+" 2017-11-04 10:57:37 SEVERE mstransformer:: (file ../../tools/mstransformer/mstransformer_cmpt.cc, line 32)+(near char. 9 in string "1,5,1,6,+,1,9,3,2,,,1,4,0,7,+,2,8,4,,,0,3,1,9,+,4,1,5,,,1,5,3,2,+,2,3,4,4,,,1,3,3,1,+,3,0,5,,,1,5,3,4,+,2,3,3,0") 2017-11-04 10:57:37 SEVERE ParallelDataHelper:::: Field Expression: No match found for name "+" 2017-11-04 10:57:37 SEVERE ParallelDataHelper::::+ (near char. 9 in string "1,5,1,6,+,1,9,3,2,,,1,4,0,7,+,2,8,4,,,0,3,1,9,+,4,1,5,,,1,5,3,2,+,2,3,4,4,,,1,3,3,1,+,3,0,5,,,1,5,3,4,+,2,3,3,0") 2017-11-04 10:57:37 WARN listobs::utils::verify Argument vis failed to verify. 2017-11-04 10:57:37 SEVERE listobs:::: An error occurred running task listobs. 2017-11-04 10:57:37 | INFO | Listobs file in: TSARP220_C_20171013_avg.ms.listobs 2017-11-04 10:57:37 | INFO | End split 2017-11-04 10:57:37 | INFO | Using MS file: TSARP220_C_20171013.ms 2017-11-04 10:57:37 | INFO | Loading msinfo from: TSARP220_C_20171013.ms.msinfo.pkl 2017-11-04 10:57:37 | INFO | Reading ms file information for MS: TSARP220_C_20171013.ms 2017-11-04 10:57:37 | INFO | Targets: 1534+2330 2017-11-04 10:57:37 | INFO | Phasecals: 1532+2344,1516+1932 2017-11-04 10:57:37 | INFO | Fluxcal: 3C286 2017-11-04 10:57:37 | INFO | Bandpass: OQ208 2017-11-04 10:57:37 | INFO | Pointcal: 3C84 2017-11-04 10:57:37 | INFO | Sources in MS TSARP220_C_20171013.ms: 1516+1932,1407+284,0319+415,1532+2344,1331+305,1534+2330 2017-11-04 10:57:38 | INFO | Antennas in MS TSARP220_C_20171013.ms: ['Cm' 'Da' 'De' 'Kn' 'Mk2' 'Pi'] 2017-11-04 10:57:41 | INFO | Elevation and uvcov plots will not be created again as long as the TSARP220_C_20171013.ms.msinfo.pkl file exists. 2017-11-04 10:57:41 | INFO | Pipeline finished 2017-11-04 10:57:41 | INFO | #################

varenius commented 6 years ago

There are actually two errors here: 1) The source names are not translated from e.g. OQ208 to 1407+284. 2) The source names seem to be splitted by letter instead of "comma", since the list logged to process seems to be "Fields: 1,5,1,6,+,1,9,3,2,,,1,4,0,7,+,2,8,4,,,0,3,1,9,+,4,1,5,,,1,5,3,2,+,2,3,4,4,,,1,3,3,1,+,3,0,5,,,1,5,3,4,+,2,3,3,0"

varenius commented 6 years ago

This appears fixed in github revision a3cc1cb, so I'm closing this issue. Sorry, should have checked for a pull also today.