EoRImaging / eppsilon

eppsilon - error propagated power spectrum with interleaved observed noise
BSD 2-Clause "Simplified" License
5 stars 4 forks source link

ave_removal increases noise ratio in first kpar bin #63

Closed nicholebarry closed 6 years ago

nicholebarry commented 6 years ago

Setting ave_removal increases the noise ratio in the first kpar bin.

We may not be incorporating the average back in correctly. There are also some multiplicative factors when adding back in the average that 1) may not be right and 2) may change depending on if using a Lomb-Scargle or FFT.

nicholebarry commented 6 years ago

If I remove n_freq from the multiplicative factors, I see much better results. The n_freq factor was a hold-over from removing the FFT IDL convention, and isn't applicable to removing the mean (which is never FFTed). Here are longruns with the Blackman-Harris window function.

It's hard to tell if this is right since I don't have anything to really compare to...comments welcome.

fhd_nb_2013longrun_autocal_pskspan100_beardsley_thesis_list_maxuv100_bh_ch0-127_averemove_swbh_dencorr_2dnnr

fhd_nb_2013longrun_autocal_pskspan100_beardsley_thesis_list_maxuv100_bh_ch0-127_averemove_swbh_dencorr_2dkpower

bhazelton commented 6 years ago

I made a branch with this change in (ave_remove_norm) but I'm worried it's still not quite right. The kpar=0 bin in the noise ratio is a bit too low with this change, and when I make difference power spectra (ave_removal - no ave_removal) the bottom 3 kpar bins all go down. I would expect the kpar=0 bin to go up while the other 2 go down to reflect the power being moved around, not disappearing.

I'm attaching the noise ratios without and with ave_removal and the difference power spectra: fhd_nb_aug2017_savedbp_w_cable_w_digjump_aug23_longrunstyle_swbh_dencorr_2dnnr fhd_nb_aug2017_savedbp_w_cable_w_digjump_aug23_longrunstyle_averemove_swbh_dencorr_2dnnr fhd_nb_aug2017_savedbp_w_cable_w_digjump_dencorr__aug23_longrunstyleswbh_minus_aug23_longrunstyleaveremove_2dkpower

bhazelton commented 6 years ago

I believe that PR #65 fixes this properly.

bhazelton commented 6 years ago

With the normalization in PR #65, the difference plots have the lowest 4 bins blue. Previously, my intuition was that the 0th bin should be red because the ave removal should be moving power from the next 3 bins into the 0th bin.

The noise ratios really make me think that this normalization is correct but I'm still getting my head around why the 0th bin drops along with the other bins. As I worked through this, I made a ratio plot that is interesting: This is the ratio of the dirty xx cube with ave_removal to the dirty xx cube without ave_removal. fhd_nb_aug2017_savedbp_w_cable_w_digjump_dencorr_dirty_xxaveremove_over_dirty_xxswbh_2dkpower