Using a hand-made mask with factor

[agwilber]

Hey everyone,

I am working on extended emission in Abell 1132, specifically a halo and long tail. I want to use the brightest sources in the cluster center as calibrators and this region is shared with the halo. I need to make sure the halo and diffuse regions of the tail which are in the calibration region are masked during calibration. I also want to make sure that these sources are masked during imaging. Factor and PYBDSM do not pick up and mask these diffuse sources, so I have made a mask by hand and put this mask into the directions.txt file under both the cal region and the field. The mask I made works in CASA when I image outside of factor, but inside factor something strange is happening (please see attached images). I checked that the facet boundaries are the same as before, not cutting through the target, so I think this strange box mask is an issue with PYBDSM merge. I am not sure how else I can get these sources masked during calibration and imaging within factor. Any help would be much appreciated. Thanks!

[agwilber]

Handmask I made as a casa region file. Works in casa during imaging. Forgot to mention that I am also using casaimager in Factor.

[AHorneffer]

I guess it would help if you could attach the region file with the mask here.

[agwilber]

Here is the hand-made mask I put into the directions.txt file, thanks!

mask.reg.zip

[rvweeren]

Hmm, it looks lone none of the info in the mask is used (based on the images above...).

I did manage to load the mask into casa (although with a lot of warnings about degenerate spectral/polarization axes, not sure if that is because I loaded in on top of an NVSS image).

My guess is that (1) the mask is not correctly passed through (maybe the full path is not given in the setup file, or it is placed in the wrong entry, or something related, so double check that), or (2) factor has trouble parsing the mask. David might have to look into this if this is the case (because I remember he did work on this a couple of months ago).

[agwilber]

Hey Reinout, the mask was using some of the regions correctly, but a large polygon was placed over the central region with the halo and tail. Outside of this central area everything else was correct.

I tried running factor again after slightly modifying the mask (I redrew regions for the sources where the box corner begins and modified a few others), again the resulting mask is quite strange. Looks like more of the regions were accepted but I am getting these polygons over large areas. I checked with David and we made sure that the path is set correctly and the file is indeed a casa region file, and again if I use the same file with casa clean outside of factor I have no problems...

image518mask.zip

[rvweeren]

Well, if the mask works ok when you run casa yourself this suggest it is a problem in factor (maybe how it parses the masks....). Are you using wsclean for imaging in the selfcal loop?

I am wondering if your mask goes outside the imaged area in the selfcal loop (maybe the parsing has trouble with that)

[agwilber]

I am not sure what you mean by parsing, perhaps how PYBDSM tries to merge with the handmask? I am using casa imager to image the selfcal iterations. I ran it once more, slightly modifying the mask again and now I am seeing my mask initially. But factor changes the mask and picks up artifacts (see image below). Would be useful if I could turn off factor's function which attempts to adjust the mask between iterations.

[rvweeren]

You will have to ask David about that functionality.

[wndywllms]

As it stands you can only add to the mask that factor automatically creates. If you aren't already using it, it may help to set selfcal_adaptive_threshold = True to improve the masking that factor does automatically to pick up fewer artefacts.

[twshimwell]

I may be wrong but I think maybe you can set skip_source_detection to True but for now have to edit line 221 of factor/scripts/make_clean_mask.py rather than the parset.

[agwilber]

Okay I will try these things, thank you!

It's a bit of a pain to mask every source in the field, and I'm finding that casa changes the size of my region ellipses when I reload the region file into the viewer. I would like to just mask the halo and tail and then have factor find all the other sources. When I tried this in the past, my mask showed up, but then factor did not properly mask all the other sources (see below). Why would the merge between my mask and source detection by PYBDSM end up like this?

[twshimwell]

Ah nope my suggestion didn't work but instead causes factor to crash attempting to read in a casa region file thinking it is a fits file.

Instead an alteration you do seem to be able to make (until there is an option in the parset) is to alter the merging bit around line 560 of the make_clean_mask.py file with something like

    if region_file is not None and region_file != '[]':
        # Merge the CASA regions with the mask
        casa_polys = read_casa_polys(region_file.strip('[]"'), new_mask)
        # Set mask to zero everywhere ## NEW LINE
        data[0, 0,:,:] = 0 ## NEW LINE
        for poly in casa_polys:
            # Find unmasked regions
            unmasked_ind = np.where(data[0, 0] == 0)
            # Find distance to nearest poly edge and mask those that
            # are inside the casa region (dist > 0)
            dist = poly.is_inside(unmasked_ind[0], unmasked_ind[1])
            inside_ind = np.where(dist > 0.0)
            if len(inside_ind[0]) > 0:
                data[0, 0, unmasked_ind[0][inside_ind], unmasked_ind[1][inside_ind]] = 1

which then sets the pybdsm computed mask to zero everywhere when merging it with the ds9 mask.

[agwilber]

I used the selfcal_adaptive_threshold and modified my mask, again. This time my mask showed up! No idea why it "worked" this time. But it didn't really go well because the calibration looks poor and the artifacts are stronger than I've seen before. Something really strange seems to be happening, because there is little to no improvement through the iterations.

[rvweeren]

For now the issue seems to have fixed itself. If this reappears we can reopen this Issue.