providing a mask for a facet

[twshimwell]

If you provide a casa region file with a mask for the facet calibrator I noticed that it still runs pybdsm and attempts to create another mask. Is this the desired behaviour? Perhaps to combine the masks?

For a very complex and extended source this running of pybdsm seems to be causing a failure because pybdsm cannot find the source (we could tune the pybdsm parameters so that it could find the source).

[rvweeren]

Yes, it still runs PyBDSM when there is a casa mask, that is done on purpose (because the could be other sources in the "calibrator box").

[wndywllms]

In the scripts the user-specified mask was always added to the pybdsm-generated mask, usually because the automatic mask did not pick up some extended emission. In the facet images this is definitely the desired behaviour, and also for larger calibration images that include other sources - I don't want to have to manually draw the mask for all the sources in the calibrator/facet images.

But, I have seen a case where the automatic mask includes some artefacts that it shouldn't (in a low dec field where the artefacts extend north-south). This could be solved by improved masking, but until then an option to subtract from the mask could be useful...

[twshimwell]

OK, makes sense. The error in this scenario is:

2016-04-05 11:55:30 DEBUG facetselfcal_facet_patch_228.executable_args: Remote command stdout: No islands found. Clean mask cannot be made.

WARNING: No islands found. Skipping wavelet decomposition. stty: standard input: Inappropriate ioctl for device --> Grouping Gaussians into sources Number of sources formed from Gaussians : 0

which causes the factor/factor/scripts/make_clean_mask.py to give an error. The make_clean_mask.py requires you to have some pybdsm components at the statement:

    if img.nisl == 0:
        print('No islands found. Clean mask cannot be made.')
        sys.exit(1)

so perhaps this doesn't need to be a requirement if you have a user input clean mask.

[AHorneffer]

I think we actually only use pyBDSM to get the threshold for the next clean step when a clean-mask is given. :-) The full pyBDSM call is in there because I was too lazy - I mean, didn't have the time - to do it properly. (Which would require quite a bit more work on the code.)

There is still the plan to fully rework that script: in addition to cleaning up the code we could include to merge the clean masks (from pyBDSM and the given mask) and to convert a given casa region file to a fits image if WSClean is used as the imager.

[Joshuaalbert]

I would like to let you know that I'm working on a better clean mask generation tool. I'm using a neural network, specifically a convolutional network, that learns to pick out a region based on an image region. It's using Google's algorithm that they use to recognize house numbers. Still in infant steps. On 5 Apr 2016 18:04, AHorneffer notifications@github.com wrote:I think we actually only use pyBDSM to get the threshold for the next clean step when a clean-mask is given. :-) The full pyBDSM call is in there because I was too lazy - I mean, didn't have the time - to do it properly. (Which would require quite a bit more work on the code.)

There is still the plan to fully rework that script: in addition to cleaning up the code we could include to merge the clean masks (from pyBDSM and the given mask) and to convert a given casa region file to a fits image if WSClean is used as the imager.

—You are receiving this because you are subscribed to this thread.Reply to this email directly or view it on GitHub

[darafferty]

I've changed the masking (on the reimage branch only for now) to return the user-supplied mask in this case instead of exiting with an error. Hopefully this change will fix the problem that Tim had.

[astrofle]

Testing. However, in the reimage branch, factor/lib/operations.py is missing lines 140-146 with respect to the master branch, which causes an undefined key error in the next steps. I.e., the lines:

Update the dictionaries with the attributes of the operation's

    # direction object. Any attributes set in the direction object that are
    # also in the parms_dict will be set to those of the direction object
    # (e.g., 'max_cpus_per_node', which is set in the direction object by
    # factor.cluster.divide_nodes() will override the value set above)
    self.cfg_dict.update(self.direction.**dict**)
    self.parms_dict.update(self.direction.**dict**)

are missing in the reimage branch.

[darafferty]

These lines were moved to the update_dicts() method, which is called during setup by the scheduler. Can you post the full traceback for the error?

[astrofle]

Hi David, sorry I removed the log file (I will revert the changes to generate the error again). But after adding back the above lines the scripts work fine and the make_clean_mask.py step is performed without a problem. Thanks!

The condition on line 301 of make_clean_masks.py should be:

 if region_file is None or region_file == '[]':

instead of:

 if region_file is None or region_file != '[]':

[darafferty]

OK, thanks Pedro. The bug above in make_clean_mask.py should be fixed now.

[darafferty]

I suspect the error above (related to the cfg_dict) is the same one as #57. If so, it should be fixed now.

[darafferty]

I assume this issue has been fixed. If not please reopen.