search

anderkve / gambit_np

0 stars 1 forks source link

How to perform a prior scan? #2

Open fzeiser opened 4 years ago

fzeiser commented 4 years ago

How can I perform a prior scan? I though I could simply use random as a scanner, to randomly go through the priors distribution specified in the model. If there is a way to avoid calculating the likelihood it'd be computationally cheaper, but that's not a major problem right now.

So I tried this and I'm confused by the output of random . I ran with the attached yaml file, here a short description NuclearBit_162Dy_22.yaml.txt

Parameters:

  GSFModel20:
    gsf_p1:
      prior_type: flat
      range: [11, 13]
    gsf_p2:
      prior_type: flat
      range: [100, 400]
    gsf_p3:
      prior_type: flat
      range: [2., 5]
[...]

  scanners:

    random:
      plugin: random
      point_number: 10000
      like: LogLike

I ran it with mpi -np 8. So I would have expected that the points sampled for gsf_p1 are flat between 11 and 13. However,

This is unexpected: I get ~1/8th of the samples at 11.0?

Similar behavior for the other parameters.

fzeiser commented 4 years ago

It might be due to a cutoff (although I don't quite understand this, because the likelihood cutoffs should come "after" selecting the prior point") - I'll check this by removing the cutoff in gledeli!

anderkve commented 4 years ago

The list of targets that GAMBIT compute (likelihoods and observables) are listed in the ObsLikes section of the yaml file. I think you need to have something here for the scan to run at all, but we can just add a dummy function "dummy_loglike" or something in NuclearBit that simply always returns loglike = 0. If we make that the only entry in the ObsLikes list, we get a fast prior scan.

I wonder if the weird sampling distribution implies that there is something wrong with the MPI setup... Could it be that it has effectively run 8 independent GAMBIT runs that all used the same prior?

Unfortunately I don't have time to dig into it today -- can look at it tomorrow.

fzeiser commented 4 years ago

I will check with and without MPI and post the results. For the moment I run again without a cutoff for the likelihood.

fzeiser commented 4 years ago

Good spot! Infact running without MPI gives the expected behavior!

fzeiser commented 4 years ago

I'll leave the issue open, as this might be something you want to comment on in GAMBIT. But from my point it's good enough now. The dummy_loglike is not needed - actually it's interesting to get the D0 and Gg model results, so I can have gledeli calculate them. But it's a very good idea if one gets a more expensive likelihood function later.

anderkve commented 4 years ago

I quickly added (https://github.com/anderkve/gambit_np/commit/aafbb8be311015d41984591c7836bb184ee23d07) a dummy zero-loglike function in NuclearBit: function name: getZeroLogLike, capability zeroLogLike I've also included it comment-out in the ObsLikes section of NuclearBit_demo.yaml.

It's nice to have, because it means we can do prior-only scans with any scanner we prefer, and using the same plotting scripts as we would for a regular posterior plot (since with loglike=0 the usual posterior weights will simply be the prior weights).

If we want, we still save the output of gledeliLogLike during our prior scan. All we need to do is to change the purpose: LogLike to purpose: Observable for the gledeliLogLike entry in the yaml file.