Fake polarised sources?

[sposullivan]

A potential issue with the DR2 dd calibration is that in some cases I think it is creating fake polarised sources throughout a field where it happens that one of the main calibrator sources is polarised.

For example, the field P219+52 is a good example where a ~9 Jy Stokes I source is ~0.5% polarised (14:43:02, 52:01:28), and its fractional polarisation and RM seems to be mirrored throughout the whole field, and not just in that facet.

I'm pretty sure this not just "lucky" ionosphere conditions, as the RM and %p values are all too similar.

As I understand, the current algorithm assumes the calibrator has zero Q and U, which suppresses the leakage very well. It would be useful to discuss how we might account for the possibility of a calibrator having appreciable polarisation.

[sposullivan]

This is the Faraday spectrum of the calibrator that I mentioned above:

[mhardcastle]

Ah yes, 3C303... getting a polarization signal from the hotspot is quite interesting but I agree we don't want it all over the sky.

I wonder if it's the DI calibration rather than the DD that's causing this?

[sposullivan]

Ah yes maybe so, the DI calibration step done by the ddf-pipeline, not the prefactor one you mean?

[mhardcastle]

Yeah.

[mhardcastle]

(would account for why it's all over the field and we haven't seen it in earlier processing)

[rvweeren]

Agree, very likely at the DI step.

[cyriltasse]

Yeah it's definitly the DI, the DD solve are always scalar-only. So what I was telling @twshimwell during lunch is that we should probably just not use these pointings. Only a few should be affected, when the hypothesis that the overall flux is unpolarised breaks down like in that case.

[sposullivan]

Annoyingly this effects all four surrounding pointings (P219+52, P223+52, P219+50, P223+50). This problem is also noticeable for the P17 field due to an 84 mJy polarised source (the above source, 3C303, is 98 mJy in the prefactor data/Van Eck catalog). There isn't an immediately obvious effect for the other HETDEX fields with polarised sources < 50 mJy. The concern would be that this could be harder to spot in the non-HETDEX fields (no Van Eck reference values), and if there were some more subtle effects occurring...

[twshimwell]

So in these fields do you see e.g. every single sources is 0.5% polarised?

[sposullivan]

No, there is a small variation within each field and a larger variation between fields for %p. The RM spread is unrealistically narrow.

[twshimwell]

Is it possible in these fields for you to distinguish the real polarised sources from the artificial ones based on e.g. the %p and the RM value?

Also how do you know in fields where you are not impacted that you are genuinely not impacted.

[sposullivan]

Potentially yes if any additional real peaks are sufficiently distinct in RM space eg. the smallest peak in these FDFs are close to the Van Eck values (the ones at +15 rad/m2 are fake and leakage at -2 rad/m2). If the DR2 results for a particular field match the Van Eck results (or my own 20" imaging of the prefactor data) and the extra sources that appear in DR2 are not obviously ones that should've been found in the prefactor data, then I've assumed things are okay. But my concern now would be if they are impacted in a more subtle way.

[sposullivan]

A potential way to identify the fields with fake sources (and just bad fields) is by looking at the median absolute deviation of the RM and the degree of polarisation within each field. eg.

Here the fields are colour-coded by the number of polarised sources in a field, and the fields with fake sources have un-physically low MAD RM and MAD %p (bottom left corner of plot). The fields with very high %p variance seem to be fields badly affected by leakage. It is still difficult to assess if there are some more subtle effects on the pol and RM values due to current strategy.

Thinking longer term, how difficult might it be to include a model of known polarised sources (say taken from the prefactor data)?

[twshimwell]

For the model wouldnt we need the polarisation fraction of each pixel and also the RM. That seems quite tricky/expensive to build up.

An alternative would be to not use the DATA_DI_CORRECTED for the polarisation maps but it would require additional calibration so would also be pretty slow

[gheald]

Hi there, just noticed this thread relatively late and thought I'd chime in. If I understand the essence of what's going on here, I think that the possible addition of a polarization model would only need to go as far as the bright-tier polarized sources, which at the moment seems to be <=1 per field. I'm not familiar with how the model has to be presented to the pipeline, though, so maybe this is the tricky part that @twshimwell referred to? But otherwise the prefactor output seems to be sufficient to be able to recover the required info (q0,u0, RM) without much drama.

Still, in addition to this there is the possibility of a more general underlying problem as alluded to by @twshimwell a couple of weeks ago - is the same problem also present at a lower level in other fields currently thought to be fine? I guess that I'd say that if there was a mechanism for injecting one or more dominant polarized sources into the sky model early on, we'd be able to do some directed tests to figure that out.

[twshimwell]

Some simulations may indeed be a nice place to start because we already have all the data processed... I dont think it would be too important to correct for ionospheric effects in the simulation so i think just injecting a fake polarised source into the prefactor output would be sufficient and then we can run those data through ddf-pipeline. Would be nice to look at a range of polarised flux. 3c303 that was causing lots of trouble just say 9Jy*0.005 = 50mJy of polarised flux. Maybe nice to explore between a few mJy and 50mJy. I think a field with a nice ionosphere and very few other polarised sources would be a good one to inject a fake polarised source into.

[gheald]

Agreed that this would be a very good & useful simulation concept. I'd go further and say that if the symptoms reported by @sposullivan can be reproduced in this way, it would then be useful to see whether they can be mitigated by injecting polarization information (even if coarse) in the sky model.

[sposullivan]

@twshimwell In terms of a field with a decent ionosphere but very few polarised sources, what about P178+67? No polarised source above 1 mJy/beam in this field.

[twshimwell]

sounds good. Now we just need someone to make the simulations :)

[sposullivan]

Below are the details of how I insert a fake polarised source into the prefactor visibilities and then image with the ddf-pipeline. It crashes pretty early in the pipeline for me, with a blank dirty image. If anyone can spot any potential issues with what I'm doing please let me know, or even try to test it yourself. I can make a dirty image from the same data using wsclean, so I should be writing to/reading from the correct data columns.

Text file (fakepol_P11Hetdex12.txt) of a bright polarised source: fake1, POINT, 11:30:13.5, +47.17.20, 10.0, 20.0, 0.01, 0.0

makesourcedb in=fakepol_P11Hetdex12.txt out=fakesrc_P11Hetdex12 outtype=casa format=Name,Type,Ra,Dec,I,RotationMeasure,PolarizedFraction,PolarizationAngle

NDPPP parset inputs to add fake source to visibilities (for each of the frequency chunks separately):

msin = L231639_SB000_uv.dppp.pre-cal_1249F5274t_121MHz.pre-cal.ms msin.datacolumn = CORRECTED_DATA msout = L231639_SB000_uv.dppp.pre-cal_1249F5274t_121MHz.pre-cal.fakesrc_P11Hetdex12.ms steps=[fake] fake.type = predict fake.sourcedb = fakesrc_P11Hetdex12 fake.usebeammodel = True fake.operation = add fake.beammode = array_factor fake.onebeamperpatch = False fake.usechannelfreq = False

Then run ddf-pipeline with: make_mslists.py and then: pipeline.py tier1-jul2018.cfg