search

simonsobs / SOLikeT

SO Likelihoods and Theories
https://soliket.readthedocs.io/
MIT License
12 stars 15 forks source link

MFLike #183

Closed cmbant closed 1 month ago

cmbant commented 3 months ago

Currently there are two different versions of MFLike (on separate git repos), SOLikeT and https://github.com/simonsobs/LAT_MFLike/, which is confusing and hard to maintain. As far as I can see, the other repo is more up to date, but the SOLikeT version has been converted into a set of Cobaya Theorys. However, not clear this gains anything in efficiency at the moment as there are no separate parameters for mflike (though in principle one could certainly split of e.g. calibration and foreground parameters). Can we just delete mflike from SOLikeT and pull from pypi? Or if we want the SOLikeT structure, merge refactor into LAT_MFLike?

(if they merge https://github.com/simonsobs/LAT_MFLike/pull/80 it would obviously be good to have in SOLikeT for testing, both for dragging sampling and fast use with cosmopower)

mgerbino commented 3 months ago

Hello Antony, yes, the idea behind the refactoring of MFLike into a set of Cobaya Theories was exactly to allow for possible independent parameters to be handled more efficiently, e.g., foregrounds and systematics (which might be shared by multiple likelihoods via fgspectra and syslib). Unfortunately, it has been always hard to stay up to date with the development of MFLike, which happens in the dedicated repo. My preference would be to keep the structure we have in SOLikeT (with improvements if needed), but I am open to re-discuss this if instead you/others think it is unnecessary

cmbant commented 3 months ago

Sure, makes sense. Perhaps we could make PR to merge changes into the main repo? I think the main thing is to avoid hard to track code duplication.

From: mgerbino @.> Sent: Monday, July 29, 2024 6:24:49 AM To: simonsobs/SOLikeT @.> Cc: Antony Lewis @.>; Author @.> Subject: Re: [simonsobs/SOLikeT] MFLike (Issue #183)

Hello Antony, yes, the idea behind the refactoring of MFLike into a set of Cobaya Theories was exactly to allow for possible independent parameters to be handled more efficiently, e.g., foregrounds and systematics (which might be shared by multiple likelihoods via fgspectra and syslib). Unfortunately, it has been always hard to stay up to date with the development of MFLike, which happens in the dedicated repo. My preference would be to keep the structure we have in SOLikeT (with improvements if needed), but I am open to re-discuss this if instead you/others think it is unnecessary

— Reply to this email directly, view it on GitHubhttps://github.com/simonsobs/SOLikeT/issues/183#issuecomment-2254971781, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AA6GQI6FLDGT46C64Z4EGKTZOXG2DAVCNFSM6AAAAABLQ2GAP6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJUHE3TCNZYGE. You are receiving this because you authored the thread.Message ID: @.***>

cmbant commented 3 months ago

I made a zeroth order attempt to merge the latest LAT_MFLike with the SOLikeT cobaya-theory structure (with ultimate objective of deleting from SOLikeT and importing). It didn't seem TheoryForge was really necessary once foregrounds were split off, so I merged it with mflike (they were mutually dependent anyway, and a lot of copy of attributes from one to the other, so don't think it was possible to use one without the other for systematics/calibrations?). For the moment I've simplified the number of theories so bandpasses are done as part of the BandpowerForeground class rather than separately. Calibration and nuisance parameters are now part of mflike, foreground and bandpass shifts are part of BandpowerForeground. BandpowerForeground seems to be all that is needed for pspipe to load the foreground model consistently as a separate class (without instantiating the likelihood). Thoughts on structure welcome of this (very rough, untested) draft:

https://github.com/simonsobs/LAT_MFLike/tree/restructure

cmbant commented 3 months ago

Now PR https://github.com/simonsobs/LAT_MFLike/pull/86. Unclear what exactly to do in SOLikeT - just delete mflike and corresponding tests, and assume it will be tested in the other repo? Or do we need to define an inherited class that multi-inherits from GaussianLikelihood?

ggalloni commented 3 months ago

Hello @cmbant, I was thinking of possible alternative structures wrt to what you propose in simonsobs/LAT_MFLike#86. How about something in between a TheoryForge and the new splitted-theories approach?

We could think of TheoryForge as an interface with a set of theories, which could be CAMB/CLASS plus Foreground (as they are now in your restructure branch) and something like "Systematics" containing for now all the operations on the output of the first two (calibration/rotation/templates). This means that each theory can have its parameters and TheoryForge will still provide a single set of spectra to MFLike as before so that the likelihood doesn't need to know about systematics and such. The interactions between these theories can be handled in TheoryForge of course.

This also implies that future development of systematics, or any of the other theories, can be done without touching the actual likelihood, which feels cleaner to me. Also, if someone for some reason wants to use a different version of some theory, that can be done easily through the interface.

Let me know what you think.

In the meantime, to test the idea and for fun, I tried to split also the systematics part as you did for foregrounds and it is kinda working (some devel is still needed).

cmbant commented 3 months ago

Thanks for looking at it. This sounds like the structure as it is now (pre-refactor) in SOLikeT? The way SoLikeT was written was quite logical and has some nice feature. But also had some slightly odd features (apart from being out of synch)

Another option would be to separate out systematics but leave calibration in likelihood (cf previous PRs).

ggalloni commented 3 months ago

Yes indeed, I was sloppy, that is essentially what SOLikeT does, except for the systematics which are handled within TheoryForge. Probably I would separate those as you suggest in the end. I agree with leaving calibration where it is.

For the single mode likelihoods, I opened a draft PR at simonsobs/LAT_MFLike#88. Locally tests are passing (pytest), I'll fix the ones failing there asap. Let me know if there is something else to do about it and eventually we can include that into the restructure branch.

cmbant commented 3 months ago

Looking at the cobaya code, I don't think at the moment there's any way for a Theory component to dynamically provide a list of default sampled parameters based on what is sent to must_provide (at most, a class can dynamically construct the parameters based on its own input parameters via get_class_options). Not easy to change, as parameters are assigned based on inputs and class-level defaults, before classes are instantiated and dependencies and providers are determined. So probably to separate the different foreground parameters for TT, TE, TTTEEE etc would require making separate Foreground classes (in addition to separate mflike classes). The classes can of course inherit from each other/import yaml files with !default as for the Planck likelihoods to avoid duplication.

An alternative would be to specify input parameters to the Foreground theory, which would have to list which polarization components are needed, and get_class_options() could then respond accordingly.

(a default Foreground model could also be constructed as a "Help theory" by mflike, where each mflike variation made its own foreground theory variation helper class; this way users would not need to specify the foreground component manually (as for CAMB's helper theory class), but would make it less easy to change the foreground model independently).