search

gammacombo / gammacombo

A framework to perform a statistical combination of measurements.
GNU General Public License v3.0
14 stars 30 forks source link

Problem with interval extraction for PLUGIN scan #228

Closed matthewkenzie closed 4 years ago

matthewkenzie commented 4 years ago

I'm seeing a weird little bug which is that the new CL interval maker is doing something very fishy for plugin scans. Look at the plot below in which it clearly finds the wrong upper value for the estimate.

@tmombaecher do you have an idea what can cause that?

gammacharm_empty+101+152+107_g_plugin

tmombaecher commented 4 years ago

Hm, so it's the new CL interval maker, i.e. the one you have to choose with a -qh option? I think I have not looked at this part of the code yet. Just guessing: it might be an issue with starting values combined with the interpolation fit. For the default CL maker code I had some months ago added also fit point uncertaintie to the interpolation. But that worked very well (and you seem to use the other method here). I would therefore check a) starting values b) interpolation fits (you might already get them with the --controlplots option, at least I added the plots for the default case some months ago) d) the amount of points taken into account in the interpolation. Unfortunately I am off for the next days and can only take a look really next week.

tmombaecher commented 4 years ago

do you have a command to reproduce the error?

matthewkenzie commented 4 years ago

Ah no it's not done with a --qh command so perhaps it doesn't use the new maker? Let me try and look into a bit more and see if I can provide something which will reproduce it.

tmombaecher commented 4 years ago

I found a way to reproduce and corner the error. It seems related to the quadratic interpolation (independent of the new CLIntervalMaker or the standard interpolate() method in the MethodAbsScan class) at a low number of scan points. It's not unproblematic, as it does not converge with a high number of toys. I suspect that for few scan points the interpolation uses points including both flanks of the peak, which leads to a bad fit.

Command to reproduce: bin/tutorial -c 1 --var a_gaus --npoints 26 --ntoys 1000 --ps 1 --npointstoy 26 --scanrange -7.0:7.0 -a plugin tutorial_tutorial1_a_gaus_plugin

tmombaecher commented 4 years ago

Found a solution. The interpolation fit was not working because the start parameters were not set properly. Redefining the quadratic function as interpolation around the scan point close to the interval boundary and choosing starting values with a linear function before seems to work (for 2 very different examples). Testing a bit more the datasets case before ready to merge. tutorial_dataset_branchingRatio_plugin tutorial_tutorial1_a_gaus_plugin

tmombaecher commented 4 years ago

I merged the fix and consider this solved. ✅

matthewkenzie commented 4 years ago

Thanks @tmombaecher !