uvf extent problems

[nicholebarry]

This is a problem that has been looked at over the past month, but making an official issue now.

There appears to be some window contamination that is dependent on how far out we grid before going to Healpix. Below is 2D dirty PS (from a in-situ sim) from gridding out to 60, 100, 150, and 300 wavelengths. There is a rise in yellow at the lower part of the EoR window the farther out we grid.

This can also be seen from the 1D power spectra as well.

Looking at real data seems to complicate the issue. Below is the 6-panel 2D PS from a regular gridding extent (300 wavelengths) and from a small gridding extent (60 wavelengths) for 64 obs from the golden set. Looking at the 1D power spectrum for the model, there appears to be more than one issue going on that was not apparent immediately from the simulations.

There is an rise in the lower part of the EoR window and an overall EoR floor.

[jpober]

The 1D power spectrum may be a confusing metric unless you're doing a k-perp cut on the 3D k cube before binning. That is, I think spherical shells of constant |k| can grab more/less wedge depending on how far out in k_perp your extent is. Maybe you're already doing this, but I think you want to take the k cubes and keep only voxels corresponding to the 60 wavelength cut. This should also allow you to make 2D difference plots which might be more informative.

[nicholebarry]

Thanks Jonnie. The 1D plots are only including kperp of wavelengths 10 to 50. The 2D difference plots would definitely be more informative than by-eye comparisons, so I'll take a look at that.

[nicholebarry]

I've made difference plots out to 60 wavelengths for each test. The below 2D differences are between: 60 - 100, 60 - 150, and 60 - 300, where "60" is the gridded 60 wavelength extent, "100" is the gridded 100 wavelength extent, etc.

[nicholebarry]

Here are the same tests, but with uvf inputs into eppsilon. This essentially skips FFTing (FHD), Healpix interpolation (FHD), and the subsequent DFTing (eppsilon). I'll be the first to say they don't look great; the resolution on those cubes are ridiculously high and do not get any nice smearing from going to Healpix, resulting in streakiness. BUT, it appears as though there is no floor problem seen above.

Gridded out to 60,100,150, and 300 wavelengths.

[bhazelton]

Ideas we just generated on this issue include: -- Do baselines get clipped separately at different frequencies or if they don't make it in the lowest frequency are they excluded in the higher frequencies? -- Does the uv plane get clipped before the FT to image space if the full plane isn't gridded (i.e can we be getting aliasing from that edge) -- Are we gridding to a square rather than a circle? Could that be causing some aliasing?

[nicholebarry]

We wanted to run a test where we made it worse (I know, surprising!).

We ran a test where we constrained the max baseline. This would give us zeros along the outside of the uv plane depending on the ps_kspan specified. Currently, when ps_kspan is specified, it forces that uv plane to be that extent (so, there is no zero-padding....so no weird clipping/aliasing as mentioned in the above comment). But, we forced there to be zero-padding to get a feel for the resulting error.

Below is a comparison between when 1) the max baseline is constrained to 150 wavelengths and zero-padded out to 300 wavelengths and 2) ps_kspan constrained to 150 wavelengths. There are two issues being looked at here: what is the effect of zero padding and is there a problem with Healpix interpolation. We have not changed the Healpix pixels, so (1) is has a matching pixel area while (2) does not.

(1) minus (2)

power kslice for (1) and (2)

weights kslice for (1) and (2)

I think the next logical step is to compare to something with the same Healpix interpolation. That is running. Any one have revelations from this, though?

[bhazelton]

So that I understand, you're saying that the previous plots you made where you were "gridding out to" some wavelength extent was actually changing the ps_kspan variable rather than setting the max_baseline variable?

If that's the case, I agree that the 2nd idea I posted 2 comments ago is a moot point.

[nicholebarry]

That's right, Bryna.

And now for the other promised test.

(1) is the same test (the max baseline is constrained to 150 wavelengths and zero-padded out to 300 wavelengths). (2) is now FHD gridding out to 300 wavelengths and eppsilon cutting that down to 150 wavelengths. This removes any difference in the Healpix interpolation.

(1) minus (2)

power slice for (1) (repeated from above) and (2)

weights slice for (1) (repeated from above) and (2)

My interpretation: clipping in the uv plane is probably not the issue. The clipped uv plane still performs better than the 300-wavelength-gridded uv plane.

Comments?

[nicholebarry]

Here is confirmation that there is no signal loss when there is less gridding extent by FHD.

I put through a hash signal and had FHD only grid out to 60 wavelengths (blue). There is no signal loss compared to gridding out to 300 wavelengths (black). So we really have a bug on our hands here!

[nicholebarry]

Left: orthoslant image, gridding out to 100 wavelengths in UV space. Right: orthoslant image, gridding out to 300 wavelengths in UV space. Units are all over the place, but that's not the point.

Bryna pointed out horizontal/vertical aliasing features in the left image. It's present in the right as well, but on different scales. This is most likely due to the inherent fact that we are gridding a rectangular UV plane.

But then, why does the right image do worse in the power spectrum? When we grid on different scales, we are moving around contamination, but definitely not removing it. Perhaps the right image does worse just in the region that matters.

Potential solution: apply anti-aliasing filter (Tukey probably) on the UV plane.

Thoughts?

[bhazelton]

@nicholebarry can you also post the healpix images for these two? That's where I originally noticed the horizontal and vertical bars.

[isullivan]

I notice that the image on the right has a larger FoV, or at least a taper from the primary beam if I'm confused about that. I suspect that would perform some anti-aliasing for you. So, one additional suggestion would be to use the larger uv extent (ps_kspan) and higher uv resolution when gridding the cubes, take the FFT of that, and crop the field of view when converting to HEALPix.

[nicholebarry]

Ian, I was zooming in on the orthoslant images, there is no difference in the image extents. Both go out beyond the first sidelobe.

Healpix images:

[isullivan]

That makes perfect sense, since I didn't really think you had been varying the uv pixel size. However, I think that might not be a bad idea to check out. It will increase the time and memory, but if you double the uv resolution but crop the images back to the normal FoV it should drastically reduce any aliasing.

[nicholebarry]

Again, my units are all over the place. Here are the orthoslant weights.

I hesitate to zoom in the color bar since the units don't make sense....but I can if that's desired.