Closed mabitbol closed 4 years ago
Note: You can click on the figures for a bigger version. I have higher resolution ones if you want them (github has 10MB limit). Please let me know if you use any figures.
Results so far 2019-10-08 Data model 1
Other things:
Baseline constraints as printed by get_dist (can anyone help with the formatting here? Github inline latex or a different get_dist print function?)
\begin{tabular} { l c}
Parameter & 95\% limits\ \hline {\boldmath$A{lens} $} & $0.991^{+0.056}{-0.055} $\
{\boldmath$r $} & $< 0.00379 $\
{\boldmath$\betad $} & $1.591^{+0.032}{-0.032} $\
{\boldmath$\epsilon $} & $0.000^{+0.040}_{-0.040} $\
{\boldmath$\alpha^{EE}d $} & $-0.422^{+0.077}{-0.077} $\
{\boldmath$A^{EE}d $} & $10.03^{+0.71}{-0.66} $\
{\boldmath$\alpha^{BB}d $} & $-0.202^{+0.079}{-0.080} $\
{\boldmath$A^{BB}d $} & $5.01^{+0.37}{-0.35} $\
{\boldmath$\betas $} & $-3.005^{+0.065}{-0.067} $\
{\boldmath$\alpha^{EE}s $} & $-0.61^{+0.25}{-0.26} $\
{\boldmath$A^{EE}s $} & $4.02^{+0.43}{-0.40} $\
{\boldmath$\alpha^{BB}s $} & $-0.41^{+0.28}{-0.30} $\
{\boldmath$A^{BB}s $} & $2.01^{+0.26}{-0.24} $\ \hline \end{tabular}
Baseline (r pdf):
Bandpass marginalized (r pdf):
Angles marginalized (r pdf):
$\phi(\nu)$ in data but not model (r pdf):
Baseline full triangle plot:
All of the above triangle plot (CMB + FG params shown):
Update November 7th 2019. Updated foreground model with params drawn on 10-20% of the sky from Planck, SPASS etc. Used wider priors and made r two sided.
\begin{tabular} { l c}
Parameter & 95\% limits\ \hline {\boldmath$A{lens} $} & $1.000^{+0.064}{-0.062} $\
{\boldmath$r $} & $0.0002^{+0.0037}_{-0.0033}$\
{\boldmath$\betad $} & $1.530^{+0.015}{-0.015} $\
{\boldmath$\epsilon $} & $0.200^{+0.036}_{-0.036} $\
{\boldmath$\alpha^{EE}d $} & $-0.302^{+0.068}{-0.069} $\
{\boldmath$A^{EE}d $} & $40.1^{+2.7}{-2.5} $\
{\boldmath$\alpha^{BB}d $} & $-0.151^{+0.069}{-0.069} $\
{\boldmath$A^{BB}d $} & $20.0^{+1.3}{-1.3} $\
{\boldmath$\betas $} & $-3.201^{+0.045}{-0.046} $\
{\boldmath$\alpha^{EE}s $} & $-1.31^{+0.23}{-0.24} $\
{\boldmath$A^{EE}s $} & $8.03^{+0.76}{-0.70} $\
{\boldmath$\alpha^{BB}s $} & $-1.21^{+0.26}{-0.27} $\
{\boldmath$A^{BB}s $} & $4.02^{+0.44}{-0.41} $\ \hline \end{tabular}
r pdf
28/11/2019 update We have moved to a more aggressive foreground model that I am calling v3 with the following parameters: A_sync_BB = 5. EB_sync = 2. (this is just the ratio of EE to BB not EB) alpha_sync_EE = -0.8 alpha_sync_BB = -0.6 beta_sync = -3.1 nu0_sync = 23.
A_dust_BB = 20. EB_dust = 2. alpha_dust_EE = -0.4 alpha_dust_BB = -0.2 beta_dust = 1.53 temp_dust = 19.6 (fixed in all fits) nu0_dust = 353.
epsilon = 0.2 (synch - dust correlation) fg_eb = 0. if do_eb: fg_eb = 0.1 (this is the intrinsic EB fraction) Alens = 1. (no delensing)
I ran the following MCMCs: baseline, baseline + AHWP, baseline + sinuous, angles, shifts, gains + shifts, off center gains, combined gains angles, dphi1.
The dphi1 runs are not working right, bug.
The shifts also have some strange behavior where the first channel doesn't marginalize over the full prior but rather tends to one side.
There seems to be an unintended effect from marginalizing over symmetric gain priors: we can't seem to affect sigma r. Maybe things are canceling out by chance? Additionally, we apply the same gain to the E and B data which connects the FG E and B amplitudes. Running a B-only check now.
The phi(nu) and angle runs seem to show that we will be able to self-calibrate. The nominal AHWP and sinuous phi(nu) curves produce no bias and the self-calibration does not increase sigma r. Adding intrinsic FG EB does not effect our ability to self-calibrate until EE and BB are correlated at the 50% level (not observed and extremely unlikely if not impossible?).
The combined gains angles run has yet to converge.
TODO
Results None of the above runs produce any bias on r or increase on sigma r. Still trying.......
BB signals: foregrounds, noise, r=0.01 tensors (no lensing)
I have a collection of all the plots here: SO_plots
In particular you can check out the r pdfs, the triangle plots, and the constraints.txt's. The plots of the walkers and chains are for diagnostics.
Things to keep in mind
Update 05/12/2019 Here is an update on the estimates of biases from the gain, shift, and angle systematics.
I use the FGv3 model to make our dataset and then nearly the exact same model to with a maximum likelihood optimizer. The difference is that in the fitting model I include a fixed systematic effect. For example, if I want to test the impact of mis-modeling the LF1 gain by 5%, I would fix the value of the gain there to 1.05 and then perform the fit with this model. This will of course induce a bias in the params, in particular the FG SED spectral indices which would then presumably leak into r at some level. The dimension of the space is too large to explore with a grid (see note) search so instead I break this down into 3 cases: vary a given systematic on a per channel basis one at a time, vary the systematic for either the LF, MF, or HF bands pairwise in the same direction (e.g. both LF1 and LF2 get a gain of 1.05), or vary the systematic for LF, MF, or HF pairwise in opposite directions (e.g. LF1 gain=1.05 and LF2 gain = 0.95). I will call these runs "per channel", "symmetric", and "asymmetric" as a shorthand. All of the plots can be found here bias plots.
Note: 6 dimensions so if you only had 10 gain values you wanted to test for each of the frequencies you get a million runs... and then we have 4+ systematics. If anyone has any better ideas that would be great, I tried a weird thing where I fixed r in the MCMC and then sampled over the systematics but that didn't seem to work.
The most interesting thing I find is that the worst case runs are usually the asymmetric ones. In hindsight maybe this is obvious as this produces the biggest difference in the neighboring channels (for gains and shifts at least) which leads to incorrect SED indices which leads to a bias on r. The angle biases can be self-calibrated so no need to worry but in any case the results are >1 deg, 0.2 deg, and 0.4 deg for LF, MF, and HF with r bias < 2e-4. The gains and shifts are more worrisome with all the numbers at the 1% level and less for the HF. HOWEVER, before we panic, we are finding that we can easily marginalize over gain/shift uncertainties with negligible impact on sigma r. I am still trying to understand how this is possible and will post that below. Also I still need to run this with a finer resolution to get more exact numbers, now that we have a better idea of where to zoom in.
Gains Worst case limits for a bias of r < 2e-4: LF gains: ~2% (from asymmetric case) MF gains: ~1% (from asymmetric case) HF gains: <1% (from asymmetric case) -- need to check with finer resolution.
Shifts Worst case limits for a bias of r < 2e-4: LF shifts: <1% (from asymmetric case) -- need to check with finer resolution. MF shifts: <1% (from asymmetric case) -- need to check with finer resolution. HF shifts: <1% (from asymmetric case) -- need to check with finer resolution.
Angles LF gains: >1 deg (from asymmetric case) -- didn't go wider than 1 deg MF gains: 0.2 deg (from symmetric case) HF gains: 0.4 (from symmetric case) Remember that we can self calibrate better than this so these are not a problem even with FG EB.
dphi1
Gains 186 63 63 Per Channel gain r{bias}<1e-3 Channel Min Max 1 0.88 1.15 2 0.85 1.1 3 0.93 1.09 4 0.85 1.08 5 0.96 1.03 6 0.97 1.04 Per Channel gain r{bias}<2e-4 Channel Min Max 1 0.96 1.15 2 0.85 1.03 3 0.97 1.04 4 0.96 1.04 5 0.99 1.01 6 0.99 1.02 Asymmetric Delta gain r{bias}<1e-3 Channel Min Max lf 0.2 -0.1 mf 0.1 -0.08 hf 0.02 -0.04 Asymmetric Delta gain r{bias}<2e-4 Channel Min Max lf 0.18 -0.02 mf 0.04 -0.02 hf 0.0 -0.02 Symmetric gain r{bias}<1e-3 Channel Min Max lf 0.9 1.1 mf 0.9 1.1 hf 0.9 1.1 Symmetric gain r{bias}<2e-4 Channel Min Max lf 0.9 1.1 mf 0.93 1.1 hf 0.9 1.1
Shifts Per Channel shift r{bias}<1e-3 Channel Min Max 1 -0.1 0.04 2 -0.03 0.08 3 -0.05 0.05 4 -0.05 0.03 5 -0.01 0.01 6 -0.01 0.01 Per Channel shift r{bias}<2e-4 Channel Min Max 1 -0.06 0.01 2 -0.01 0.05 3 -0.02 0.02 4 -0.02 0.01 5 0.0 0.0 6 0.0 0.0 Asymmetric Delta shift r{bias}<1e-3 Channel Min Max lf 0.02 -0.08 mf 0.04 -0.04 hf 0.0 0.0 Asymmetric Delta shift r{bias}<2e-4 Channel Min Max lf 0.0 -0.04 mf 0.0 0.0 hf 0.0 0.0 Symmetric shift r{bias}<1e-3 Channel Min Max lf -0.1 0.1 mf -0.1 0.1 hf -0.05 0.06 Symmetric shift r{bias}<2e-4 Channel Min Max lf -0.05 0.1 mf -0.1 0.07 hf -0.02 0.02
Angles Per Channel angle r{bias}<1e-3 Channel Min Max 1 -1.0 1.0 2 -1.0 1.0 3 -0.6 0.6 4 -1.0 1.0 5 -1.0 1.0 6 -1.0 1.0 Per Channel angle r{bias}<2e-4 Channel Min Max 1 -1.0 1.0 2 -1.0 1.0 3 -0.3 0.3 4 -0.8 0.8 5 -1.0 1.0 6 -0.6 0.6 Asymmetric Delta angle r{bias}<1e-3 Channel Min Max lf 2.0 -2.0 mf 2.0 -2.0 hf 2.0 -2.0 Asymmetric Delta angle r{bias}<2e-4 Channel Min Max lf 2.0 -2.0 mf 1.0 -1.0 hf 2.0 -2.0 Symmetric angle r{bias}<1e-3 Channel Min Max lf -1.0 1.0 mf -0.4 0.4 hf -0.9 0.9 Symmetric angle r{bias}<2e-4 Channel Min Max lf -1.0 1.0 mf -0.2 0.2 hf -0.4 0.4
dphi1
TODO
Attempting a merge... Polangle lost compatibility with SACC.
I'm gonna close this one (but won't delete the branch), since this has been merged into the BBPower repo https://github.com/simonsobs/BBPower
Here we begin to address bandpass and polarization angle systematics. I will break the results into two categories (1) addressing biases from an incomplete model and (2) assessing the increase in sigma r from marginalizing over systematics.
1) Check for biases from incomplete models. (If we cannot include in the model or have a poor model then we would like to assess the potential for bias) (i) Impact of $\phi(\nu)$: the in-band frequency dependent polarization response/rotation. $\phi(\nu)$ can be generated from the broadband 3-layer AHWP or the sinuous antenna design. (ii) Other? Intrinsic foreground EB? (iii) Can we create any intuition or rules of thumb to help understand what shapes and levels of systematics are most critical?
TODO list
2) Assess impact on ability to constrain r. How much does our constraint on r degrade when marginalizing over additional foreground and systematic parameters. We are considering bandpass and polarization angle systematics here in particular. (i) Bandpass uncertainties. Gain, shift to parameterize uncertainty. +Jeff's model of slopes, shifts, ripples. (ii) Polarization angle uncertainties. Including $\phi(\nu)$ uncertainties. Intrinsic EB.
TODO list
General TODO list
Other issues:
Another important note: These results do not agree with the SO forecast paper because of the simplified foreground model. There is no residual FG marginalization here that drove up the errors in the forecast! Things might be more realistic and agree with the forecast when we go to data model 3.