wsclean model is very different from the mask

[spectram]

Wsclean fails to create a model that represents the mask in containers: CARACal.Stimela.make_cube-HI-field0-iter1-with_user_mask and CARACal.Stimela.make_cube-HI-field0-iter2-with_SoFiA_mask (At some point I had tried the automasking parameter as well and from the looks of the old log it had recovered more flux from certain channels. But unfortunately, I deleted the data).

In iter1, I put the mask in the masking folder and the mask seems to have re-gridded and input into wsclean correctly:

wsclean -name /stimela_mount/output/cubes/cube_1/All_Haro11_HI_1 -mem 100 -no-update-model-required -weight briggs 1.000 -taper-gaussian 0.0 -size 2048 2048 -scale 2.0asec -channels-out 459 -pol I -channel-range 0 459 -niter 1000000 -auto-threshold 0.3 -gain 0.1 -mgain 0.99 -multiscale -multiscale-scale-bias 0.6 -multiscale-scales 0,8,16,24,32 -fits-mask /stimela_mount/output/masking/All_Haro11_HI.image_sofmask_All_regrid.fits -padding 1.2 /stimela_mount/msdir/1613822473_sdp_l0-Haro11-galx_chan.ms /stimela_mount/msdir/1628971877_sdp_l0-Haro11-galx_chan.ms

iter2 running command:

wsclean -name /stimela_mount/output/cubes/cube_2/All_Haro11_HI_2 -mem 100 -no-update-model-required -weight briggs 1.000 -taper-gaussian 0.0 -size 2048 2048 -scale 2.0asec -channels-out 459 -pol I -channel-range 0 459 -niter 1000000 -auto-threshold 0.3 -gain 0.1 -mgain 0.99 -multiscale -multiscale-scale-bias 0.6 -multiscale-scales 0,8,16,24,32 -fits-mask /stimela_mount/output/cubes/cube_2/All_Haro11_HI_2.image_clean_mask.fits -padding 1.2 /stimela_mount/msdir/1613822473_sdp_l0-Haro11-galx_chan.ms /stimela_mount/msdir/1628971877_sdp_l0-Haro11-galx_chan.ms

In iter1, the *model.fits is empty whereas in iter2, the *model.fits has some consistent weird models in regions not included in the mask. (see the image attached)

I do see that in the directory */output/cubes/cube_2, the files have been named as All_Haro11_HI.*.fits instead of All_Haro11_HI_2.*.fits (from line 88516 in the log). I am a bit confused about the use of files from cube_1 as well. Does iter1 correspond to cube_1 and iter2 to cube_2? And the final cubes are files in cube_2 that get renamed after the stokesremoval and freqtovel operations. Do I understand this correctly?

See YAML input parameters below. The full log is also attached: cw-car-6020015-stderr.log

I am using STIMELA_IMAGES_1.7.5 and Caracal version 1.0.6 (from setup.py) on ilifu.

line_galx: 
  enable: true
  label_in: galx_chan
  line_name: HI
  restfreq: '1.420405752GHz'
  make_cube:
    enable: true 
    use_mstransform: false
    npix: [2048]
    cell: 2.0
    robust: 1.0
    nchans: 0
    firstchan: 0
    binchans: 1
    wscl_auto_mask: 3
    wscl_auto_thr: 0.5
    wscl_mgain: 0.99
    wscl_onlypsf: false
    wscl_multiscale: true
    wscl_multiscale_scales: 0, 8, 16, 24, 32
    wscl_nrdeconvsubimg: 1
    wscl_sofia_niter: 2
    wscl_sofia_converge: 0
    wscl_removeintermediate: false
    wscl_user_clean_mask: 'All_Haro11_HI.image_sofmask.fits'

[spectram]

I ran this again without giving a mask. cube_1 based on the simple threshold mask cleaned fine but the issue with the 'incorrect model' repeats in cube_2. I am not sure if I am using an incorrect parameter. Copying the yaml for the cleaning below.

  make_cube:
    enable: true  ## will create the cube in another script
    use_mstransform: false
    npix: [2048]
    cell: 2.0
    robust: 0.5
    nchans: 0
    firstchan: 0
    binchans: 1
    wscl_auto_mask: 4
    wscl_auto_thr: 0.5
    wscl_mgain: 0.95
    wscl_onlypsf: false
    wscl_multiscale: true
    wscl_nrdeconvsubimg: 1
    wscl_sofia_niter: 2
    wscl_sofia_converge: 0
    wscl_removeintermediate: false

[paoloserra]

[Edited following https://github.com/caracal-pipeline/caracal/issues/1505#issuecomment-1594256054 from @KshitijT -- thanks for spotting the mistakes!]

Sorry for the slow reply @spectram

Your problem is likely caused by a bug in WSClean ~3.2~ 3.1 (which is still there in WSClean 3.2). We reported this in https://github.com/caracal-pipeline/caracal/issues/1462#issue-1553145435 .

If you update to ~the most recent release of Stimela~ Stimela 1.7.7 or higher you will have WSClean 3.3 there, which fixes this issue.

Alternatively, you can row back to WSClean ~3.1~ 3.0 as suggested at https://github.com/caracal-pipeline/caracal/issues/1462#issuecomment-1420469485 .

Please let me know whether using WSClean ~3.1~ 3.0 or 3.3 solves the issue. Thanks!

[spectram]

@paoloserra Thanks a lot for your response. I confirm it seems to be the same bug. Switching to an older version of WSClean using the 'cab' variable solves the issue. I will raise a request with ilifu to pull the latest Stimela. Closing the issue as it is resolved.

[paoloserra]

great!

[KshitijT]

Alternatively, you can row back to WSClean 3.1 as suggested at https://github.com/caracal-pipeline/caracal/issues/1462#issuecomment-1420469485 .

Please let me know whether using WSClean 3.1 or 3.3 solves the issue. Thanks!

Note for future reference that 1.7.3 gives wsclean version 3.0, which works fine, whereas the error seems to be associated with wsclean version 3.1 (based on a chat with @spectram) (and version 3.2?). Upgrading to wsclean 3.3 also sorts the issue, as noted before.