search

joshspeagle / dynesty

Dynamic Nested Sampling package for computing Bayesian posteriors and evidences
https://dynesty.readthedocs.io/
MIT License
357 stars 77 forks source link

Derived Parameters #98

Closed joezuntz closed 5 years ago

joezuntz commented 6 years ago

It's extremely useful to be able to record and output a set of derived parameters for each point in parameter space, along with the sample point itself. We often want to plot contours of these derived parameters.

Emcee calls this feature "blobs", and enables it by checking how many outputs the loglikelihood function returns - if it returns two objects the second is assumed to be some generic output to be returned to the user along with the sample.

I'd be happy to provide a PR for this if you would consider it.

Many thanks, Joe

joshspeagle commented 6 years ago

Happy to take a PR for this if you'd like to contribute. Otherwise, I'll try and implement this when I get around to resolving the other open issues in ~a week or so.

joshspeagle commented 5 years ago

Alright, it's...uh...a little past a week, but I've now spent a good amount of time looking at this. I originally thought this would be straightforward, similar to its implementation in emcee. Essentially, I just planned to check whether the loglikelihood returns more than 1 output, partition them into logl and logl_extra, and continue on as normal (saving logl_extra as appropriate).

Turns out there are several structural reasons why this is difficult to implement in dynesty, in the same vein as the difficulties trying to implement a previous feature request. The two biggest reasons are:

  1. The way that sampling is set up involves a lot of passing things back and forth between various modules of the code. Coherently tracking extra log-likelihood outputs then requires modifying all of those individual methods to check this, which turns out to be really difficult to keep track of.
  2. I hadn't appreciated just how much nested sampling manipulates the likelihoods as part of the algorithm itself. There's a lot of places where I'm operating over the current set of live or dead points, and all of those also required associated checks.

Most of these problems just comes down to me having not properly structured the code to enable these types of add-ons. Tackling this without adding a billion safety checks would probably require significant rewriting of the internals, so I'm just going to put this on hold for now.

I know that this is disappointing (I am currently working on a problem where keeping track of additional parameters returned from the loglikelihood would be extremely useful), and apologize both for the delay in getting back to you and failing to properly structure the code to enable this. It's a good lesson I hope to carry forward to future projects.

joezuntz commented 5 years ago

Hi Josh,

Many thanks for considering this and letting me know, and sorry I never got around to attempting it myself, though it sounds like this would be pretty hard in this case.

Cheers, Joe