search

joshspeagle / dynesty

Dynamic Nested Sampling package for computing Bayesian posteriors and evidences
https://dynesty.readthedocs.io/
MIT License
357 stars 77 forks source link

UserWarnings: Random number generation inefficient? #113

Closed jvines closed 5 years ago

jvines commented 5 years ago

Hi, I'm getting a huge number of these UserWarnings: UserWarning: Random number generation appears to be extremely inefficient. Bounding distributions might be sub-optimal.

from line 200 of sampling.py

What could be causing this?

I'm using the pip version of dynesty and python 2.7

Thanks!

joshspeagle commented 5 years ago

Hi Jose,

Are you on the most recent version (v0.9.3)? If not, try updating and see if that fixes your problem.

jvines commented 5 years ago

Yes, I am on the most recent version. I changed from using pip to directly installing from the repository, I'm checking to see if the same error occurs again.

jvines commented 5 years ago

Yes, the same warnings appear even when installing from the repo

joshspeagle commented 5 years ago

Okay, great. Could you provide some additional information on your problem and configuration?

joshspeagle commented 5 years ago

Any update on this? Some additional information would be great to try and diagnose the possible problem.

joshspeagle commented 5 years ago

Closing this for now.

jvines commented 5 years ago

Hi, I'm very sorry but currently, I'm not in a location with stable internet access :s

What kind of information do you need? The problem, in particular, has 29 dimensions.

I'm using the dynamic sampler with a single bound

I noticed that the efficiency drops to ~9.5% before it gets stuck on the warnings of inefficient random number generator and random walks.

Once again, I'm sorry for the delayed response.

joshspeagle commented 5 years ago

Could you include a snippet of your code (if it's a model that I can run), the exact commands you're using to initialize the sampler, and a few snapshots of the output?

jvines commented 5 years ago

sampler = dynesty.DynamicNestedSampler(loglike, prior, 29, nlive=100, bound ='single', sample='rwalk')

loglike is a very complicated function that deals with dictionaries and stuff, and prior is a function that returns priors (e.g. Normal, Uniform, Jeffreys)

For more context I'm using a code named Juliet (https://github.com/nespinoza/juliet) and when I wrote to Nestor about the dynesty warnings outputs he redirected me to you.

Saldy I can't provide screenshots at the moment but from my correspondence with Nestor, I could recall this line:

/jvines/anaconda2/lib/python2.7/site-packages/dynesty/sampling.py:200: UserWarning: Random number generation appears to be extremely inefficient. Bounding distributions might be sub-optimal. warnings.warn("Random number generation appears to be "

I tried reinstalling from the repository but the same warnings kept appearing.

jvines commented 5 years ago

I'm now running a case with 16 free parameters and the same happens after this iteration:

iter: 1788 | batch: 0 | bound: 1 | nc: 25 | ncall: 17544 | eff(%): 9.909 | loglstar: -inf < 4752.189 < inf | logz: 4744.410 +/- 0.119 | dlogz: 124.089 > 0.010

Should it be expected that dynesty get stuck for long on this:?

/Users/jvines/anaconda2/lib/python2.7/site-packages/dynesty-0.9.3-py2.7.egg/dynesty/sampling.py:224: UserWarning: Random walk proposals appear to be extremely inefficient. Adjusting the scale-factor accordingly.

joshspeagle commented 5 years ago

Okay, that's helpful. First, for the error you list above:

/jvines/anaconda2/lib/python2.7/site-packages/dynesty/sampling.py:200: UserWarning: Random number generation appears to be extremely inefficient. Bounding distributions might be sub-optimal.

This springs up when proposals exceed the boundaries of the unit n-cube that dynesty samples within (which gets transformed to the parameters of interest). This should only be happening if a good chunk of the bounding ellipsoid exceeds the bounds of the unit n-cube, which generally only happens if:

  1. the ellipsoid is constructed when the live points are still pretty diffuse in one or more dimensions and/or
  2. a lot of live points are piled up near the edge

You can check the scenarios by terminating the sampler and examining the bound object stored (either in the sampler or the results object). Some examples for how to visualize that can be found here.

I'll also just note that 100 points for a 29-dimensional problem is smaller than I'd recommend and leads to an underconstrained solution for the bounding ellipsoid (you generally want ~n^2-style scaling to ensure stable covariances for the ellipsoids). It's also a higher dimensionality than I'd recommend for rwalk (see the FAQ for additional discussion on sampling options and tuning).

As for your second test case, it looks like dynesty is hitching as soon as it constructs the first bounding ellipsoid (which by default happens after the efficiency hits 10%; see the first_update argument for additional details). The warning here suggests the ellipsoid itself (which dynesty uses to make random walk proposals) is much larger than the distribution of points. Have you tried running with 'multi' instead? Or just checking to make sure all your points aren't either (1) dispersing everywhere in some dimensions or (2) running up against the boundary in others?

jvines commented 5 years ago

Okay, so I tried giving it 1000 live points and changed the bound to multi and it got past the 10% efficiency mark! And is continuing normally.

Thank you very much for your help!!!

joshspeagle commented 5 years ago

Great! I'll close this for now, but feel free to comment with any other issues if you're having more problems.