Step -1.3: vis_cpu validation

[steven-murray]

This test should be a formal validation of vis_cpu against pyuvsim. Note that parts of this test will be done as unit-tests in hera_sim.

• Simulation Component: point source, gaussian blob, GLEAM
• Simulators: vis_cpu, pyuvsim
• Pipeline Components: None
• Depends on: Ideally, this depends on a semi-automatic notebook infrastructure for all -1 tests, being developed by @piyanatk. It also depends on https://github.com/HERA-Team/hera_sim/pull/33 being merged.

Why this test is required

vis_cpu is a unique simulator, in that it is antenna-based instead of baseline-based. It could become useful for quicker simulations than other methods in the future. As such, it should be formally validated.

Summary

A brief step-by-step description of the proposed test follows:

• Construct visibilities from increasing-complexity sky models, both with vis_cpu and pyuvsim.
• Compare visibilities explicitly, and report on the level of absolute and relative error between simulators, especially as a function of frequency.

Simulation Details

• Freq. range:
• Channel width:
• Baseline/antenna configuration:
• Total integration time:
• Number of realisations:

Criteria for Success

• Visibilities match to 1% between simulators across all frequencies, baselines and LSTs.

[Jackmastr]

I've been using this PRISim vs pyuvsim memo as a starting point for visualizing the differences between the outputs of vis_cpu and pyuvsim. For obsparam_ref_1.1.yaml I'm seeing very similar results to the ones in the memo, where the "uvw" positions are close to the reference except for a small phase shift:

However, the visibilities themselves appear to be off by an appreciable amount:

Worse, with obsparam_ref_1.2_uniform.yaml the uvw positions don't seem to match at all and I'm not sure why:

The uvw positions in vis_cpu aren't changing as they are supposed to for Ntimes > 1.

[steven-murray]

This is great work!

I think it makes sense that vis_cpu should give different visibilities than pyuvsim in some regimes: it is after all an approximation. We just need to be able to identify the regimes in which it does well, and how well.

I might be wrong, but I feel like I recall that vis_cpu works differently, in that its uvw doesn't change with time, but the sky rotates with time. That should explain the last plot. So you'll only be able to compare visibilities in that ref sim.

[aelanman]

Internally, pyuvsim considers the sky to be moving and the baselines to be fixed. Any change in uvws over time can only be due to phasing. Depending on what file format you're using, that could explain the difference in UVWs. The uvfits format requires data to be phased, so the UVWs will change with time.

If this is the issue you can try reading both vis_cpu data and pyuvsim data using UVData, then apply unphase_to_drift to both.

[piyanatk]

To add to the list of things to test.

The new implementation in VisCPU (see HERA Memo #098) apply a correction to the source RA/Dec given a reference time to improve the source position accuracy in the Alt-AZ frame.

We should check how tolerant this correction is to the duration of the observation. That is, given a single reference time t_ref, compare accuracy of the output visibility from the VIsCPU simulations with the observation duration of 5 minutes, 1 hour, ..., and 24 hours, to the equivalent simulations from pyuvsim.

Phil Bull suggested that this test may be constructed by modifying one of the hera_sim unit test

The outcome of this test might be important to figure out how we may want to split the validation simulation in time.