53MHz cube of IC5264 from pipeline

[SpheMakh]

The image is here:

/scratch/sphe/reductions/meerkathi-32k-16bands/output-2048/slicker-pipeline_1_HI-cube.fits

[paoloserra]

Is this the cube where we reach thermal noise? (Sorry, the telecon's sound quality was not great, I hope I didn't misunderstand). I measure 4.5 mJy/beam.

[gigjozsa]

I am wondering the same. I put two example cubes here: com4:/scratch/josh/example/IC5264_170406_nohan_fullres.fits and com4:/scratch/josh/example/IC5264_170406.fits The first is naturally weighted full velocity resolution, which has 2x rms (so 1.7mJy/beam). The second is averaged by 2 chans and Hanning smoothed.

[SpheMakh]

I selected bandpass and gain calibrators automatically for this run. I'll re-run with bandpass=PKS1934 and gaincal=ATCA2259-375

[SpheMakh]

I'm getting an rms of 2.0mJy/beam in a channel. The full cube is at com4:/scratch/sphe/reductions/meerkathi-32k-16bands/output-2048/slicker-pipeline-contsub_HI-cube.fits

[gigjozsa]

So that's the 2 x thermal rms. We probably need a tool to estimate the thermal rms from the continuum-subtracted data.

[KshitijT]

How do the fluxscale plots look for this run?

[SpheMakh]

@KshitijT Very good

[SpheMakh]

Also, I run the old pipeline (master branch) and got the same thing.

[SpheMakh]

That is, I can't reproduce the flux-scaling error.

[SpheMakh]

@gigjozsa Could you please point me to the data set that was calibrated and reached thermal noise?

[KshitijT]

So, the fluxscale error is not what is producing the 2X thermal noise rms?

[SpheMakh]

Doesn't seem to be. But I'm still puzzled because the noise in the continuum images with the correct fluxscale was ~2x larger than the images which didn't.

[SpheMakh]

I changed the self-cal strategy a bit, and now I get 1.7mJy rms in a channel. I think @gigjozsa got a thermal noise close to this using Sean's script (#65).

The improvement is mainly due to including more sources in the calibration at the start, and increasing the solution intervals (10mins, 6.5MHz). However, the final calibration step (once a good model has been built up) is done with smaller (32s, 2.3MHz) solution intervals.

The cube is at com4:/scratch/sphe/reductions/meerkathi-32k-16bands/output-two-2/meerkathi-pipeline_HI-cube.fits

[gigjozsa]

That's correct. Did you also change the flagging strategy or only the calibration? If not, we need to work on the flagging. We're definitely getting there. One thing I remark is that there are problems with the continuum subtraction. Looking at the source at 22:57:26 -35:48:00 it appears that chromatic aberration is not taken into account or something strange. The source appears to move radially, which it should not do if the uv-coordinates are calculated correctly. Is there a non-continuum-subtracted cube to verify that?

[SpheMakh]

The flagging is the same, and the I ran uvcontsub with order=1.

[gigjozsa]

Should try order=2, that's what I did. But... what about the radial shift with frequency? That's not expected if the uv-coordinates are calculated correctly. It must be some bug.

[SpheMakh]

I'll re-run uvcontdub with order=2. But its worth noting that 22:57:26 -35:48:00 is about 43' from the phase centre, and the effect may be from smearing.

[SpheMakh]

I averaged 8s instead of 32s, and used order=2 in uvcontsub, and the issue seemed to have been fixed. The cube is at /scratch/sphe/reductions/meerkathi-32k-16bands/output-two-4/meerkathi-pipeline_HI-cube.fits

[gigjozsa]

Could you do both separately? I still do not understand it.

Am 21.08.2017 09:44 schrieb "Sphesihle Makhathini" <notifications@github.com

:

I averaged 8s instead of 32s, and used order=2 in uvcontsub, and the issue seemed to have been fixed. The cube is at /scratch/sphe/reductions/ meerkathi-32k-16bands/output-two-4/meerkathi-pipeline_HI-cube.fits

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ska-sa/meerkathi/issues/64#issuecomment-323670637, or mute the thread https://github.com/notifications/unsubscribe-auth/AITaVCQ3MxWfaLgq10JColIeTLHOeXDoks5saTVngaJpZM4OzJtJ .

[gigjozsa]

Point is this is a radial feature. That should not be solvable by decreasing integration time. If it is the fit order solving the problem we should set it aside but it is still to be addressed.

Am 21.08.2017 09:44 schrieb "Sphesihle Makhathini" <notifications@github.com

:

I averaged 8s instead of 32s, and used order=2 in uvcontsub, and the issue seemed to have been fixed. The cube is at /scratch/sphe/reductions/ meerkathi-32k-16bands/output-two-4/meerkathi-pipeline_HI-cube.fits

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ska-sa/meerkathi/issues/64#issuecomment-323670637, or mute the thread https://github.com/notifications/unsubscribe-auth/AITaVCQ3MxWfaLgq10JColIeTLHOeXDoks5saTVngaJpZM4OzJtJ .

[SpheMakh]

Decreasing the integration time, and using order=1 gives the same result. However, I forgot that problematic source was not in the calibration model for selfcal (self-cal images were not as big the HI cube), so the problem is fixed by subtracting the source during selfcal.

[gigjozsa]

The data quality is slowly reaching the commissioning results when using order=2. Time averaging is not related to the smearing problem. We have to be aware of the following: during selfcal we are imaging at a large bandwidth. The consequence is that sources far from the phase centre are radially stretched in the continuum images. Any source model based on clean will hence appear elongated. Using the corresponding model for a continuum subtraction would then result in an undersubtraction at the centre and an oversubtraction further out. This does not appear to happen. Here, the problem might be connected to the NULL of the primary beam moving through the source(s): while the source is modelled based on an averaged primary beam, it will vanish and reappear when the NULL moves through. If a constant model (maybe some spectral index) is subtracted, it will oversubtract whenever the NULL moves through, and slightly undersubtract everywhere else. However, the fact that the oversubtraction moves radially might imply that the subtracted source is still elongated.

Nevertheless we should create a map in which only the 1 Jy source is subtracted. I would like to see if there is no chromatic aberration visible (it should not).

Is it possible to do that AND repeat the whole thing with t_int=32s and first order subtraction, but using - say - 16 channels for the continuum mapping instead of 2?

[SpheMakh]

All of this possible, but maybe better if you do some of it. Its basically setting things on and off in the configuration file.

[gigjozsa]

This is a commissioning issue and will be followed up (or not) there.