LSSTDESC / augur

DESC forecasting and inference validation tool
MIT License
3 stars 2 forks source link

Issues/16 #17

Closed fjaviersanchez closed 10 months ago

fjaviersanchez commented 2 years ago

This PR tries to address #16, implementing our own derivatives of the loglikelihood to get the Fisher matrix. It is still work in progress.

elisachisari commented 2 years ago

I tried to run this new version for the Y1 yaml example and I ran into issues towards the end. I copy the log.


Pipeline ran okay. Applying additional prior sigma = 0.2 to params--omega_c This will be assumed to be centered at the parameter center regardless of what the ini file says The limits of the parameter will also not be respected.

Applying additional prior sigma = 0.8 to params--w0 This will be assumed to be centered at the parameter center regardless of what the ini file says The limits of the parameter will also not be respected.

Applying additional prior sigma = 2.0 to params--wa This will be assumed to be centered at the parameter center regardless of what the ini file says The limits of the parameter will also not be respected.

Applying additional prior sigma = 0.3 to params--h This will be assumed to be centered at the parameter center regardless of what the ini file says The limits of the parameter will also not be respected.

Applying additional prior sigma = 0.14 to params--sigma8 This will be assumed to be centered at the parameter center regardless of what the ini file says The limits of the parameter will also not be respected.

Applying additional prior sigma = 0.08 to params--n_s This will be assumed to be centered at the parameter center regardless of what the ini file says The limits of the parameter will also not be respected.

Applying additional prior sigma = 3.9 to params--a0 This will be assumed to be centered at the parameter center regardless of what the ini file says The limits of the parameter will also not be respected.

Applying additional prior sigma = 0.5 to params--bias_lens0 This will be assumed to be centered at the parameter center regardless of what the ini file says The limits of the parameter will also not be respected.

Applying additional prior sigma = 0.5 to params--bias_lens1 This will be assumed to be centered at the parameter center regardless of what the ini file says The limits of the parameter will also not be respected.

Applying additional prior sigma = 0.5 to params--bias_lens2 This will be assumed to be centered at the parameter center regardless of what the ini file says The limits of the parameter will also not be respected.

Applying additional prior sigma = 0.5 to params--bias_lens3 This will be assumed to be centered at the parameter center regardless of what the ini file says The limits of the parameter will also not be respected.

Applying additional prior sigma = 0.5 to params--bias_lens4 This will be assumed to be centered at the parameter center regardless of what the ini file says The limits of the parameter will also not be respected.

Postprocessing... Generated covariance matrix was not positive definite - beware! /Users/chisa002/Documents/lsst_augur/augur/augur/postprocess.py:151: RuntimeWarning: invalid value encountered in sqrt heights = 2 np.sqrt(0.5 (C[0, 0] + C[1, 1] - D)) /Users/chisa002/Documents/lsst_augur/augur/augur/postprocess.py:161: RuntimeWarning: invalid value encountered in sqrt sig0 = np.sqrt(C[0, 0]) Traceback (most recent call last): File "/Users/chisa002/opt/anaconda3/envs/forecasting/bin/augur", line 7, in exec(compile(f.read(), file, 'exec')) File "/Users/chisa002/Documents/lsst_augur/augur/bin/augur", line 41, in run() File "/Users/chisa002/opt/anaconda3/envs/forecasting/lib/python3.9/site-packages/click/core.py", line 1137, in call return self.main(args, kwargs) File "/Users/chisa002/opt/anaconda3/envs/forecasting/lib/python3.9/site-packages/click/core.py", line 1062, in main rv = self.invoke(ctx) File "/Users/chisa002/opt/anaconda3/envs/forecasting/lib/python3.9/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, ctx.params) File "/Users/chisa002/opt/anaconda3/envs/forecasting/lib/python3.9/site-packages/click/core.py", line 763, in invoke return __callback(args, **kwargs) File "/Users/chisa002/Documents/lsst_augur/augur/bin/augur", line 37, in run augur.postprocess(_config) File "/Users/chisa002/Documents/lsst_augur/augur/augur/postprocess.py", line 62, in postprocess ax[j, i].set_xlim(-sig0+centers[i], sig0+centers[i]) File "/Users/chisa002/opt/anaconda3/envs/forecasting/lib/python3.9/site-packages/matplotlib/axes/_base.py", line 3522, in set_xlim left = self._validate_converted_limits(left, self.convert_xunits) File "/Users/chisa002/opt/anaconda3/envs/forecasting/lib/python3.9/site-packages/matplotlib/axes/_base.py", line 3439, in _validate_converted_limits raise ValueError("Axis limits cannot be NaN or Inf") ValueError: Axis limits cannot be NaN or Inf

elisachisari commented 2 years ago

Did you compare our own forecast with one from cosmosis for the same setup?

fjaviersanchez commented 2 years ago

hmmm, there's something going on with the updates. The matrix is not invertible anymore and for some reason the results don't agree 100% here is the comparison between diagonals (it might be related to the fact that the matrices are not invertible): image

fjaviersanchez commented 2 years ago

I thought it was the priors, but I added the priors in the calculation and they don't seem to make much of a difference. I am comparing here the matrices and they look alike, at least.

cosmosis

image

numdifftools

image

fjaviersanchez commented 10 months ago

This PR is stale now, closing. Will resume the work on a different branch/PR