search

EoRImaging / eppsilon

eppsilon - error propagated power spectrum with interleaved observed noise
BSD 2-Clause "Simplified" License
5 stars 4 forks source link

Add freq_dft option #125

Closed bhazelton closed 3 years ago

bhazelton commented 4 years ago

Add an option to do a frequency DFT even in the case of even frequency spacing to allow for using the true z values.

Cannot be merged until #119 is.

fixes #123

bhazelton commented 4 years ago

my testing showed worse outcomes with the DFT, so not sure we want to default it on, but I think we do want the option. @nicholebarry can you test this with your data?

nicholebarry commented 4 years ago

By being really pedantic with double precisions in cosmology_measures and z_mpc, along with fixing a double->float cast of the stored frequency array in the healpix cubes on the FHD side, I've been able to get the FFT and the regular DFT to be very similar.

This a difference on a zenith obs, FFT minus regular DFT. The color scale is floating, and at extremely low levels. EoR0_zenith_freqfix_doubles_noscale_nodft_minus_regdft

bhazelton commented 4 years ago

excellent! Thanks!

nicholebarry commented 4 years ago

Insofar as the "true" DFT (using the comoving distances directly, which are not regularly spaced), this is the difference that I get. I wouldn't say it's worse, but I'd like to hear your opinion.

I do expect the first kz mode to have a difference of 0. And I do expect the second kz mode to be different, because there is a curvature to the calculation of the comoving distances. redshift_diff

Here is the difference power spectra, regular redshift FFT minus the true redshift DFT. Blue: true redshift DFT has less power. Red: true redshift DFT has more power. Here, I suggest looking at the model only...the calibrated data and residual appear more complicated due to couplings with the foregrounds (i.e. the sidelobe structure). fhd_nb_gaussbeam_1fits_restrict1_fullimg_dencorr__zenithaveremove_swbh_minus_zenithdft_averemove_2dkpower

The power spectra has a strong response out to the third/fourth kz mode which I didn't quite expect. I think this can be explained by an expansion via the window function. For example, if I use a Hamming window (smaller "main" lobe, higher sidelobes compared to BH), the resulting spread past the second kz mode is different. fhd_nb_gaussbeam_1fits_dist_test_fullimg_dencorr__zenithaveremove_swham_minus_zenithdft_averemove_2dkpower

To me, this says that the true redshift DFT does not hurt the EoR window if the window function has low sidelobes, like the BH. I'd vote to include it by default since it is ideally the more correct thing to do.