search

anderkve / gambit_np

0 stars 1 forks source link

Dev: pass lnlike cutoff #4

Open fzeiser opened 3 years ago

fzeiser commented 3 years ago

Started on how to pass the lnlike cutoff parameter from gambit to gledeli

@anderkve: Could you have a look at the NuclearBit.cpp (and headerfile) and code up the rest, so how one gets the lnlike cutoff parameter from the settings?

anderkve commented 3 years ago

This turned out to be a bit more involved than I thought, because passing info from the "KeyValues" section in the YAML file to specific GAMBIT module functions really broke the GAMBIT design. So in the end I figured it was easier just to add a YAML option "lnlike_cutoff" specific for our NuclearBit function. This is now demonstrated in NuclearBit_demo.yaml. I've commented out the standard GAMBIT option "model_invalid_for_lnlike_below". (This means that internally this value is set to a default which is 0.9, so that should never affect anything.)

So now it's completely up to NuclearBit / gledeli to invalidate points with too low lnlike. This should now work as long as the get_results function in gledeliBE.py always includes a boolean entry results["valid_point"] in the results dictionary it sends back to GAMBIT. I didn't implement this in gledeliBE, as I had some other issues preventing me from testing it. Can you try adding that and test that the cutoff works as expected?

*We are sort of breaking the GAMBIT design by having a single backend code evaluating a bunch of independent likelihood contributions, and tracking internally when/if a point falls below the cutoff. The design of GAMBIT rather expects that individual likelihood contributions are reported back to GAMBIT and that the GAMBIT core keeps track of when/if a point falls under the threshold. Hence the values in KeyValues were never intended to be shared with the module functions in the different Bits.

fzeiser commented 3 years ago

Thanks, I'll test it!

I see you point about breaking GAMBIT's design, but I would propose to continue. Unless we want to turn fully GAMBIT, we're for now just using it to demonstrate gledeli. I also think that this is a likely usecase scenario for "external" users who may just want to use a very limited set of GAMBIT's functionality.

anderkve commented 3 years ago

I see you point about breaking GAMBIT's design, but I would propose to continue. Unless we want to turn fully GAMBIT, we're for now just using it to demonstrate gledeli. I also think that this is a likely usecase scenario for "external" users who may just want to use a very limited set of GAMBIT's functionality.

I completely agree -- I just wanted to explain why something that should have been simple turned out not be. :) It's actually something I'll take note of for future GAMBIT development. It might be nice to have a simple way to give all parts of GAMBIT access to all info in the yaml file, even though (for safety/consistency) it typically shouldn't be used.