Issue with TE and ET cross-frequencies spectra

[beringueb]

@xgarrido , @sgiardie and I found that the way mflike deals with TE and ET for cross frequency spectra is not clear and possibly not correct at the moment.

• ET cross-freqeuncy spectra are not stored during the theory constrution, instead they are defined as TE spectra with the "hasYX_xsp" argument being 'True'.
• In theoryforge.get_modified_theory(), cmbfg_dict is contructed with all different frequency and spectra combination as there is a loopfor exp1, exp2 in product(self.experiments, self.experiments): and one for s in self.requested_cls:. So for 3 frequencies with TT,TE and EE, cmbfg_dict has 12 entries.
• cmbfg_dict is then calibrated and rotated and this should work fine, becaus it contains all the crosses we are interested in.
• But when cmbfg_dict is transferred to dls_dict, the entry of dls_dict["te", m["t1"],m["t2"]] get written twice: once for the actual ["te", m["t1"],m["t2"]] and another time when hasYX_xsp == True when it gets: `cmbfgdict["te", m["t2"],m["t1"]].
• These two entries have the same CMB+FG but might have different calibrations.

Possible fixes include:

• include an extra key "hasXY_xsp" in dls_dict
• build the self.spec_meta dictionary using "pol" = TT/TE/ET etc and not the camb keys tt/te/ee

[HTJense]

Would a potential solution be to just be more consistent in swapping the keys in the dls_dict when distinguishing between TE/ET? We could change this check to

if m["hasYX_xsp"]:  # not symmetrizing
    dls_dict[p, m["t2"], m["t1"]] = cmbfg_dict[p, m["t2"], m["t1"]]
else:
    dls_dict[p, m["t1"], m["t2"]] = cmbfg_dict[p, m["t1"], m["t2"]]

(note the swapped indices on the left-hand side of the first case assignment). In other words, store E_1 x T_2 as T_2 x E_1 (which should be equivalent) in the dls_dict (which should, in my opinion, be simpler to read as well than adding an extra key in the dls_dict and ignoring half the possible entries).

The only other change needed then would be to add a check in the mflike._get_power_spectra function, to check that

for m in self.spec_meta:
    p = m["pol"]
    i = m["ids"]
    w = m["bpw"].weight.T
    dls_obs = DlsObs[p, m["t1"], m["t2"]]
    if m["hasYX_xsp"]: dls_obs = DlsObs[p, m["t2"], m["t1"]]
    cls = w @ dls_obs
    ps_vec[i] = clt

To make sure we also access the same entry.

[beringueb]

Yes, I think this would work too and might be simpler to implement and understand. I've done it in the TE_ET_quickfix branch.

@sgiardie can you look at the TE_ET_quickfix branch and see if it works ?

[sgiardie]

Hi Ben, if you use Hidde's solution then the extra flag m["hasYX_xsp"] should be removed from dls_dict[p, m["t1"], m["t2"], m["hasYX_xsp"]] in theoryforge

[sgiardie]

and also clt = w @ DlsObs[p, m["t1"], m["t2"]] --> clt = w @ dls_obs here

[beringueb]

Yes, sorry went a bit too fast here. Fixed now

[sgiardie]

The code works and now it makes sense how it handles the cross frequency spectra in the case without symmetrization. I have checked it makes sense also in the case with symmetrization. I added some comments in the code to make it a bit more transparent. We should decide if we want to stick with this version or make the code even clearer by setting the polarizations to TT/TE/ET/EE etc (instead of the camb ones tt/te/ee...) and eliminating the need of the flag "hasXY_xsp".

[xgarrido]

Fix https://github.com/simonsobs/LAT_MFLike/pull/71