search

lesgourg / class_public

Public repository of the Cosmic Linear Anisotropy Solving System (master for the most recent version of the standard code; GW_CLASS to include Cosmic Gravitational Wave Background anisotropies; classnet branch for acceleration with neutral networks; ExoCLASS branch for exotic energy injection; class_matter branch for FFTlog)
223 stars 291 forks source link

TT Chi2 minimization #399

Open fhenryco opened 3 years ago

fhenryco commented 3 years ago

hello,

Minuit fails to give me the parameter error on Omega_M when i fit a TT power spectrum simulated with Class. The Chi2 parabola as function of this parameter (all other parameters being fixed) is not smooth which is probably the reason why minuit fails. DG1

is such chaotic shape of the parabola actually expected with Class due limited precision of computations( rounding noises) ? Do other people see this ? When i try to minimize versus other parameters such as As or ns the parabola is on the other hand perfecly smooth and minuit succeedes!

lesgourg commented 3 years ago

Dear Frederic, this is indeed totally expected. CLASS can work at many different precision levels, and of course precision has a cost in terms of CPU time, so we always try to set the default precision to something fast, with just enough precision for the needs of most users. Most users do Bayesian inference with MCMC chains. For this, a relatively low precision is sufficient. Thus CLASS (and actually also CAMB) have default precision settings adjusted for those users. But it is very easy to increase the precision in roder to do profile likelihoods, frequentist analyses and minimisation with MINUIT. For instance, the CAMEL team (a sub-team of Planck specialised on frequentists analyses and based in Orsay) did a massive use of CLASS with MINUIT, always with enhanced precision settings (that still keep the code relatively fast - I think the slow down is of the order of 50% at most). You might find these settings written in their papers. I am also sure that there is one issue dedicated to this on forum, with some clear recommendation for the precision settings. I will try to dig it out and come back to you.

lesgourg commented 3 years ago

Indeed this has been touched at least in #156. Quoting from there:

Let me give below a set of precision parameters that I trust for lowering considerably the numerical noise. It has been used in the paper https://arxiv.org/pdf/1311.1657.pdf which performed a frequentist analysis and had to use a high-precision minimisation algorithm instead of an MCMC. As stated in their Table 1, they found that they should use:

tol_background_integration = 1.e-3 
tol_thermo_integration = 1.e-3
tol_perturb_integration = 1.e-6
reionization optical depth tol = 1.e-5
l_logstep = 1.08 
l_linstep = 25
perturb_sampling_stepsize = 0.04
delta_l_max = 800

Since this dates back from 2013, a few things may have change. Please let us know if you find that these settings are not sufficient (or on the contrary, that you can get the precision you want with less extreme settings). Note that the meaning of tol_background_integration and tol_thermo_integration will become different in the forthcoming CLASS v3.0 release, thus these parameters will take different values (in the default and in the high-precision settings). Then we will try to remember to update this chat!

fhenryco commented 3 years ago

Thank you very much for your reactivity and this set of precision parameters. I actually had tried to modify some of the parameters in precision.h but apparently not the best ones...thanks again

lesgourg commented 3 years ago

No need to change any .h and recompile... just pass your precisions parameters in input as you would do with the cosmological parameters!

fhenryco commented 3 years ago

OK , Minuit works much better now, thanks!