Seed for consistency - Githubissues

LSSTDESC / Smokescreen

A library of methods to mitigate experimenter bias in the LSST DESC cosmology analysis

https://lsstdesc.org/Smokescreen/

BSD 3-Clause "New" or "Revised" License

4 stars 0 forks source link

Seed for consistency #45

Closed m-aguena closed 3 months ago

m-aguena commented 4 months ago

If we ensure that a given seed always results in the same set of cosmological parameters, it can be used to blind different probes individually. This will remove the necessity of generating the data vectors of all your probes at the same time.

IMPORTANT reminder: save the seed on the blinded data vector file!

arthurmloureiro commented 3 months ago

I realised I was not keeping the original metadata in the blinded sacc!

This is now fixed :)

I also added a concealed = True metadata !

arthurmloureiro commented 3 months ago

String seed: `seed = "test"`

Locally: Shifts: {'Omega_c': 0.25876948503900465, 'sigma8': 0.7726193034315977} Sunrise: Shifts: {'Omega_c': 0.25876948503900465, 'sigma8': 0.7726193034315977}

Int Seed: 2112

Locally: Shifts: {'Omega_c': 0.21229098052498757, 'sigma8': 0.7142436174585977} Sunrise: {'Omega_c': 0.21229098052498757, 'sigma8': 0.7142436174585977}

arthurmloureiro commented 3 months ago

I also tested that if you give different number of parameters, the values are the same for the same parameters:

Locally (omega_c and sigma8): Shifts: {'Omega_c': 0.21229098052498757, 'sigma8': 0.7142436174585977}
Sunrise (only omega_c): Shifts: {'Omega_c': 0.21229098052498757}

arthurmloureiro commented 3 months ago

Now, of course... if you invert or change the order of the required shift parameters, the shifts will be different:

Locally: Shifts: {'sigma8': 0.7001923251481939, 'Omega_c': 0.22084394106175506}
Sunrise (inverted order): Shifts: {'Omega_c': 0.21229098052498757, 'sigma8': 0.7142436174585977}

Maybe this can be avoided by looping over CCL dictionaries of cosmological parameters instead of looping over the shifts dictionary...

arthurmloureiro commented 3 months ago

Now, of course... if you invert or change the order of the required shift parameters, the shifts will be different:

Locally: Shifts: {'sigma8': 0.7001923251481939, 'Omega_c': 0.22084394106175506}

Sunrise (inverted order): Shifts: {'Omega_c': 0.21229098052498757, 'sigma8': 0.7142436174585977}

Maybe this can be avoided by looping over CCL dictionaries of cosmological parameters instead of looping over the shifts dictionary...

yeah, that worked! Fixed by 53ba5afa96bec84445cac31ec5b699e8ed43348a