Deep negatives bowls in selfcal/*low_MFS-image*

[botteon]

My fist facet (patch_653) present deep negatives bowls in the low_MFS-image. Note that the attached image was made with 80SBs but the problem is present even with less (40SBs) or more (>100SBs). The cleaning mask is picked up correctly. As you can see switching on/off the multires doesn't help (wrong label in the screenshot, it is multires, not multiscale).

I don't have errors during the selfcal, indeed the full-MFS-image looks good (with multires=yes).

What about the negatives in facet_patch_579 (my target)? Sources are subtracted in the Init-Subtract step, but I don't understand why they are so different switching the multires on/off.

If I continue the run, the selfcal of next facets proceeds without errors and full-MFS-image looks good, while the low res images present bowls. After 6 facets calibrated I image only the target facet and this is what I obtain:

Has anybody already seen this behaviour? What is going on?

[darafferty]

How deep are the holes in the two cases (multires and non-multires)? It's not uncommon to have some shallow ones, I think.

What about the negatives in facet_patch_579 (my target)? Sources are subtracted in the Init-Subtract step, but I don't understand why they are so different switching the multires on/off.

This is strange -- selfcal (multires or not) should not affect the neighboring facets. How did you do the tests? Did you reset any of the directions (with the "-r" flag to runfactor) during them? Maybe the reset is not working properly? It looks like sources were subtracted more than once somehow...

[botteon]

In attachment the min values in for 4 different negative bowls (note the displacement of the negative bowls in the red circle, top left panel, with respect to the others). I reported the relevant settings used in the parset for the 4 different runs.

I delete the facetselfcal, facetsub, log and state files before each run (so no runfactor -r). Note that the next facets present similar behaviour.

Could this issue related to #159 , by any chance?

[darafferty]

Ah -- if you delete the facetselfcal, facetsub, log and state files, Factor does not know that it already subtracted the new model (with the new solutions), and so it does not add them back properly when it starts selfcal (instead it adds the old model with the old solutions). If you reset with the "-r" flag, Factor will first revert the changes made during the previous selfcal before proceeding to do selfcal again.

[botteon]

Ok, I am starting from scratch now (it is the first facet in the field, so it won't take so long). I will let you know. Thanks.

A question related to -r flag. If I selfcal N facets and then runfactor -r facet_patch_(N-2), will factor selfcal again only facet(N-2), or it will start from facet(N-2) and continue with the next ones?

[darafferty]

It will only selfcal the one that was reset

[botteon]

I started from scratch but the negative bowls are still present (with the same min values)

[darafferty]

I wonder if the sky-model interpolation is going wrong somehow? Can you compare the sky models for each band used for the subtraction (the *.make_high_facet_skymodel files) with the corresponding model images (the *.wsclean_image_full_low-XXXX-model.fits ones)? For example, the sky model that is at the same frequency as the first channel model image (*.wsclean_image_full_low-0000-model.fits) should have source fluxes that match the fluxes in this image.

[botteon]

I can't find a match between the values in the *.make_high_facet_skymodel and the *.wsclean_image_full_low-XXXX-model.fits. I checked the values with ds9. It seems that I have more components in the .fits than that listed in *.make_high_facet_skymodel. I also notice the following:

cc62, POINT, 13:32:25.461, +50.05.53.015, 0.00202805595472, 0.0, 0.0, 0.0, 121189880.371
cc63, POINT, 13:32:25.305, +50.05.53.015, 0.00516824610531, 0.0, 0.0, 0.0, 121189880.371

My pixelscale is 1.5 arcsec. How can I have two CC (with different fluxes) in the pixel so?

[darafferty]

Sorry, I should have written *.wsclean_image_full-XXXX-model.fits (without the _low: the low ones are used for the next subtraction step). Can you compare the sky models to these images instead?

Also, the two clean components above are 1.5" apart, so are consistent with being in neighboring pixels.

[botteon]

Yeah, here the fluxes are consistent.

[botteon]

Just for comparison:

Jit provided me the *full_low-MFS-image.fits of the first facet (patch_800) in his field. He didn't have problems in the imaging.

[darafferty]

I think I know now why the negative holes are occurring (and why it is observation/direction dependent). It's due to Factor ignoring the negative clean components during the subtraction (which, when subtracted, would add flux, filling in the holes). Often this is not a problem, but in some cases they can be important, so I will alter Factor to include them.

[twshimwell]

these holes seem to be up to 0.4Jy though. It seems strange to have that much flux in the negative clean components?

[twshimwell]

(I guess this could be checked from the model images)

[darafferty]

Perhaps the negative holes are exacerbated by the low resolution (so many clean components are summed together), and they wouldn't be so deep in the high-resolution images?

At any rate, I've updated the sky model script to include negative clean components. Andrea, can you update and try it again?

[botteon]

Could you update the file in CEP3? I am working there at the moment.

[darafferty]

Could you update the file in CEP3? I am working there at the moment.

Done.

[rvweeren]

Perhaps the negative holes are exacerbated by the low resolution (so many clean components are summed together), and they wouldn't be so deep in the high-resolution images?

The problem is that on the low-resolution images you should not clean with "point sources". The emission picked up should come -only- from diffuse extended sources, representing them with delta functions is simply not ideal (and at high-resolution you get severe over subtraction). Not sure how stable this will be, but maybe we could clean with multiscale, removing the delta function from the scale list, so all clean components will be Gaussians with a width > 0 (afterall the point sources should be removed in the high-res facet image). Obviously this needs testing.

[botteon]

Yes, now the bowls are not so deep:

[botteon]

Ok, I can confirm that with the new fix also the image looks ok:

[rvweeren]

That's much better indeed. So it seems that this issue has been addressed.

[botteon]

I am not sure that the problem has been entirely solved. In the run with 240SBs I still have a deep hole (-0.4Jy) in the facetselfcal/full_low-MFS-image.fits of the taget facet. See below the comparison with the full-MFS-image.fits

This results in the following facetimage/full1-MFS-image.mask (left) and facetimage/full2-MFS-model.fits (right)

Note that the other facets don't show such negative bowls but I didn't image them (image_target_only=True). As David mentioned above, perhaps here we are looking to a peculiar case where there are two bright sources embedded in a bright diffuse emission (radio halo) and the low resolution enhance brutally the holes?

[AHorneffer]

In my run with the current version of Factor (commit from 31.1.), I still see negative "bowls" in some (two) of the wsclean_post images. In the one facet where I looked at it more closely most of the negative clean-components fall outside the mask and thus aren't picked up by Factor.

Up to now it doesn't look like it is going to be a problem for me, but it isn't nice. I also don't know how to solve this without going to the image1 - mask - image2 again. (Maybe make that optional?)

P.S. Issue re-opened because I believe we are not fully done yet.

[darafferty]

Are the negative features usually associated with the calibrator sources? (I ask since the calibrator region already uses the image-mask-image sequence.)

Maybe we could expand (dilate) the mask a little to pick up the negative CCs? The danger with expanding the source masks is that we might start picking up artifacts (though only those from sources in other facets would be problematic, and I suspect this would be rare, especially if we only expand them by a pixel or two).

[AHorneffer]

Are the negative features usually associated with the calibrator sources?

Nope.

[botteon]

In the case of my facet_patch_579 they are. Note that in the facetimage/*full1-MFS-image.mask that I reported above there is a wide mask in the facet center that I use in the selfcal and imaging (my calibrator is the radio halo and the two sources inside that mask)

[agwilber]

I don't know if this issue is related to one I had a while back, but I found that I needed to reduce the number of multiscale levels because I was getting big holes in my .model. This is when I was using casa clean... and I would get an error saying that casa couldn't use the 5th or 6th scale for multiscale, but there was still this hole-effect to my .model. I reduced the multiscale to 4 levels and the holes were gone. The factor default is 0, 3, 7, 25, 60, 150 ... but perhaps the higher ones are not necessary? I am unsure as to why this happened though - maybe someone can understand and explain better.

[rvweeren]

and I would get an error saying that casa couldn't use the 5th or 6th scale for multiscale

That is not an error but a warning (because the largest scale is larger than the size of your biggest masked region).

BTW negative components are (always) possible in a model image (to compensate for inaccurate subtraction positive components, or from calibration problems). How does you actual restored image looks like? Ideally, the scales should be matched to the spatial scales of your emission, but that is hard to predict beforehand. I am not even sure if the negative holes you are seeing are really problematic in this case...In fact despite a few holes I would say your first model image is better because it picks up more extended emission

[agwilber]

I thought that there shouldn't be any negative components in the model - does it mean I overcleaned? I thought that the new multiscale [0,3,7,25] looked better in the end... Should we always use more multiscales? Because of the warning, I thought it would be best to not use the last two. Here are the final images (btw this is the old data heavily flagged data for a1132):

[agwilber]

Also, from Annalisa about my images above:

"...but in the first model image the largest scale component (the blue one) picks up emission in scale which is clearly too big compared to the cluster. I believe it is not a real model component, and I believe that - if you had sources subtracted - it would cause large patches, positive or negative, likely similar to those that Andrea is seeing."

Perhaps if Andrea lowered the scales it would help, but the only way to check would be to try cleaning with casa probably because wsclean doesn't allow for changes in multiscale. Is this something we could implement with factor? Can the user still choose casa cleaner instead of wsclean?

[rvweeren]

I thought that there shouldn't be any negative components in the model - does it mean I over cleaned?

Not necessarily.

(however, ideally the largest scale should be matched to the largest spatial scale of your emission, as Annalisa mentioned, of course this requires you to know how the source looks like....)

btw Your images look very similar so in your case it does not really matter so much.

[botteon]

I tried to image facet_579 without multiscale (mscale_field_do False in the direction file) inside factor. You can see the improvements with respect to the original run (with mscale_field_do empty in the direction file) on the right (check the model I reported some posts above with the large negative and positive pattern).

I tried also to image the dataset outside factor, with the same parameters used in factor without multiscale (left) and with multiscale on (right)

[agwilber]

Looks like more diffuse emission is showing up in your non-multiscale image outside of factor. Scary what happened with mscale_field_do... I haven't noticed this happening to my images, but maybe it is happening on a smaller scale - I did have some rings and bowls in my last image made with mscale_field_do = True. @darafferty Maybe we should not use this feature right now?

[rvweeren]

Be careful, you expect more diffuse emission in the non-multiscale images. Because the diffuse emission is not fully cleaned in these images it appears more bright (because of the LOFAR PSF shape). However, if the diffuse emission is not cleaned you get wrong flux estimates.

[darafferty]

Hopefully, this issue has been solved by recent changes. Reopen if not.