Plotting error - Githubissues

varenius commented 6 years ago

Perhaps this may have to do with the fact that I flagged one antenna, De, completely using a manual flag file with contents "antenna='De'", since Defford receivers were warm here? If the plotting always assumes all antennas to be there, it may fail because of this?

varenius commented 6 years ago

Two notes: 1) The plotting step seem to have run even thought it was not selected in the "input.txt", see the input log parameters above. This should not happen. 2) It seems that the "antennas in MS" are correctly identified, so it should know that De is not there. So perhaps the error is something else.

varenius commented 6 years ago

Actually, it seems this plotting calls are always run, no matter what input parameter are chosen. The code is the following in the main pipeline script, after the "average_1" step:

    logger.info('Using MS file: {0}'.format(msfile))

    ### Load or create dictionary with ms information.
    if os.path.isfile(msfile+'.msinfo.pkl'):
        logger.info('Loading msinfo from: {}'.format(msfile+'.msinfo.pkl'))
        #msinfo = load_obj(msfile+'.msinfo')  I decide to read it again,
        #because it is very fast, and it is needed if user changes some inputs.
        msinfo = em.get_msinfo(msfile, inputs)
        save_obj(msinfo, msfile+'.msinfo')
        logger.info('Elevation and uvcov plots will not be created again as long as the {}.pkl file exists.'.format(msfile+'.msinfo'))
    elif os.path.isdir(msfile):
        msinfo = em.get_msinfo(msfile, inputs)
        ### Produce basic observation plots
        emplt.make_elevation(msfile, msinfo)
        emplt.make_uvcov(msfile, msinfo)
        save_obj(msinfo, msfile+'.msinfo')
        logger.info('Saving information of MS {0} in: {1}'.format(msfile, msfile+'.pkl'))
    else:
        logger.info('No data or msinfo found. Calibration will not work.')
        pass

This code is always run, which I think is the wrong behaviour? If nothing is specified in inputs.txt, the pipeline should do nothing,

jmoldon commented 6 years ago

The original error is weird. That selection is completely transparent:

    ms.open(msfile)
    staql={'field':field, 'spw':'0'}
    ms.msselect(staql)

it looks more like casa bug? And I have not find the same problem, I will try to replicate it. In any case, I don't know the solution.

I had a very long discussion with myself regarding when to produce the elevation and uvcov plots. It is basic information that I want to have available every time any weblog is produced (at the beginning or the end). They are very fast to produce, so I didn't want an additional step only for them, and I don't want to include them in the other plotting steps, which can be very long (>40min). Also I didn't want to include them in the last weblog step because I like that step to be immediate. So my solution was to produce those plots the first time the pipeline is run, and then don't remake them if they have been produced (so if the pkl file is there). I see now the alternative to move the same code but inside the weblog step. So they will be produced the first time the weblog is run, but not again if not needed.

The original reason for all that code you copied is that in previous versions getdata2 was slow, so it took long to produce the msinfo dictionary (sources, antennas, spw, num_channels, etc.), but it was fundamental information for many steps. So I decided to run it only once and keep the information in a msinfo.pkl file that could be read later. If now we move the elev and uvcov plots, I don't think there is any reason to keep it there, as it is better to read all the msinfo every time.

varenius commented 6 years ago

Right, thanks for the info.

I did a "reset" by removing all pipeline products and starting from scratch. This time it ran without problems. The only change I made was the source names in the inputs-file, e.g. OQ208->1407+284. Perhaps some failure due to the wrong names in an earlier step gave rise to this. Anyway, this particular error seems gone so I'm closing this issue.

varenius commented 6 years ago

I re-ran the pipeline another time (to reset flags) and it failed again! 2017-11-06 13:39:30 | INFO | Pipeline version: v0.6.2 2017-11-06 13:39:30 | INFO | Using github branch: master 2017-11-06 13:39:30 | INFO | github last commit: a3cc1cb ... 2017-11-06 15:29:38 | INFO | Plotting uvcov for 0319+415: ./plots/plots_observation/TSARP220_C_20171013_avg_uvcov_0319+415.png Traceback (most recent call last): File "/opt/packages/casainst/casa-release-5.1.0-74.el7/lib/python2.7/init_welcome.py", line 24, in execfile(candidates[0]) File "pipeline/eMERLIN_CASA_pipeline.py", line 387, in run_pipeline(inputs=inputs) File "pipeline/eMERLIN_CASA_pipeline.py", line 193, in run_pipeline emplt.make_uvcov(msfile, msinfo) File "pipeline/functions/eMERLIN_CASA_plots.py", line 197, in make_uvcov u, v = read_uvw(msfile, f) File "pipeline/functions/eMERLIN_CASA_plots.py", line 177, in read_uvw ms.msselect(staql) File "/opt/packages/casainst/casa-release-5.1.0-74.el7/lib/python2.7/casac__/ms.py", line 1710, in msselect return _ms.ms_msselect(self, *args, **kwargs) RuntimeError: MSSelectionNullSelection : The selected table has zero rows.

I ran this on "embe1". I saw that you, javi, also ran things there today. Is there perhaps some CASA-issue with running multiple CASA instances? I have no idea, they should be fully independent. But since this part of the code is causing the problem, it would be nice to be able to turn it off.

jmoldon commented 5 years ago

Resolved by #100

e-merlin / eMERLIN_CASA_pipeline

Plotting error #75