lwimager: first channel of cube is empty

[o-smirnov]

As reported by @SpheMakh: "When imaging with lwimager using a multi-frequency MS the first channel is empty. A possible workaround would be to add an extra channel at start of the MS and image from the "second" channel. We can do something similar with Ed's cube." See https://github.com/ska-sa/ceiling-kat/issues/2.

I have seen this before, and I have a suspicion this is something to do with frequency/velocity frames (or rather a bug in how they're treated internally). Need to squash this bug at some point, but at present I don't have the emotional energy to delve into CASA code.

(Copying in @twillis449 as he may have seen the problem too.)

[ludwigschwardt]

Isn't this related to the fact that channel 0 out of any channelized correlator is basically junk as it differs substantially from the rest? It's real instead of complex, for one, and is corrupted by any DC offsets on the individual antenna input signals. Maybe lwimager correctly refuses to image this...

[IanHeywood]

An alternative to using lwimager is just to invoke the CASA clean task directly from Pyxis. This is what I've been doing since (as far as I am aware) lwimager also refuses to image Measurement Sets that have multiple spectral windows. A bunch of parameters in the snippet below are hard wired but they could easily be moved up to the function call depending on what flexibility you require.

This also converts the image and the PSF to FITS and deletes the CASA format image product. I feel I should point out the fact that there's an 'rm -rf' in there as a system call.

def imageMS(msName,imageName,dirty,fixedbeam):
    cc = 'clean('
    cc += 'vis="'+msName+'",'
    cc += 'imagename="'+imageName+'",'
    cc += 'mode="mfs",'
    cc += 'gridmode="widefield",'
    cc += 'wprojplanes=64,'
    if dirty:
        cc+='niter=0,'
    else:
        cc += 'niter=2500,'
    cc += 'imsize=[2240,2240],'
    cc += 'cell=["12.0arcsec"],'
    cc += 'multiscale=[],'
    cc += 'weighting="briggs",'
    if fixedbeam:
        cc += 'restoringbeam = ["2.097E-02deg","1.559E-02deg","-1.779E+02deg"],' 
    cc += 'robust=0.0)\n'
    cc += 'exportfits(imagename="'+imageName+'.image",fitsimage="'+imageName+'.fits",overwrite=True)\n'
    cc += 'exportfits(imagename="'+imageName+'.psf",fitsimage="'+imageName+'.psf.fits",overwrite=True)'
    std.runcasapy(cc)
#   deleteExtensions = ['.image','.psf','.flux','.residual','.model']
    deleteExtensions = ['.image','.flux','.residual','.model','.psf']
    for ext in deleteExtensions:
        syscall = 'rm -rf '+imageName+ext
        os.system(syscall)

[o-smirnov]

@ludwigschwardt: it's not the imager's business to nanny us (though perhaps this particular piece of code comes from an Aus contributor?) The MS can easily be resampled/averaged/whatnot, with junk channels trimmed, etc. For example, the JVLA data I get from Rick Perley comes nicely preprocessed. In this case channel 0 is as good as any other, and if the user wants it imaged, it's not up to some old overengineered C++ code to deny it.

[o-smirnov]

Actually lwimager does image multiple spws (specified from the command line as e.g. spwid=0,1,2), I just haven't wrapped this in Pyxis properly.

It's about time for me to wrap the CASA imager too, so thanks for that code...

[IanHeywood]

Ah cheers, my mistake.

[ludwigschwardt]

Good point, Oleg!

On 30 September 2014 16:54, Oleg Smirnov notifications@github.com wrote:

Actually lwimager does image multiple spws (specified from the command line as e.g. spwid=0,1,2), I just haven't wrapped this in Pyxis properly.

It's about time for me to wrap the CASA imager too, so thanks for that code...

— Reply to this email directly or view it on GitHub https://github.com/ska-sa/lwimager/issues/2#issuecomment-57284828.

[twillis449]

I have also seen the error reported by Oleg. In real life its not that critical. Bandpass filters and attenuation, various FFT effects etc often make the beginning and ending channels of data sets not all that useful.

[foxmouldy]

I usually clip off the edge channels before I image a line dataset, so my first channel has useful data in it. I would even clip these channels before I start calibrating, since they are junk.

[gigjozsa]

If the bandpass (if present) range doesn't exactly match the spectral range of the data, even trimming (by channel) would not help and could lead to the behavior described, if lwimager checks these things. Just to add another thing to check. In reality it can e.g. happen if a calibrator is observed at a frequency slightly different from the target (e.g. due to Doppler tracking). Should not turn up in simulations, though, so it would be a bug somewhere.