search

tensorflow / probability

Probabilistic reasoning and statistical analysis in TensorFlow
https://www.tensorflow.org/probability/
Apache License 2.0
4.24k stars 1.1k forks source link

Laplace approximation #570

Open dionman opened 5 years ago

dionman commented 5 years ago

Is there any function computing Laplace approximation (similar to ed.Laplace) supported at tensorflow probability at the moment?

jeffpollock9 commented 5 years ago

I'm not aware of any Laplace approximation functionality but would definitely be keen to help with contributing it.

brianwa84 commented 5 years ago

Building a cookie-cutter laplace_approx(target_density, location) shouldn't be too hard. Could probably do something like tfd.MVNLinearOperator(LinearOperatorInvert(LinearOperstorFull(tf.hessians(target_density(xs), xs)))) A couple thoughts: tf.hessians d/n work in tf2, here's a workaround: https://github.com/tensorflow/tensorflow/issues/29781#issuecomment-504980912 The hessian is not necessarily PD unless the location is in a local minimum.

Bigger challenge may be how are you going to find the location.

Brian Patton | Software Engineer | bjp@google.com

On Thu, Sep 26, 2019 at 1:51 PM Jeff notifications@github.com wrote:

I'm not aware of any Laplace approximation functionality but would definitely be keen to help with contributing it.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/tensorflow/probability/issues/570?email_source=notifications&email_token=AFJFSI7N622KMV63T42PSUDQLTZDJA5CNFSM4I2X5PMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7WNR5I#issuecomment-535615733, or mute the thread https://github.com/notifications/unsubscribe-auth/AFJFSI6DPVKUDU2G3E2EL7TQLTZDJANCNFSM4I2X5PMA .

junpenglao commented 5 years ago

I would try using lbgfs to find the location - it should be even possible to find multi mode using tfp.optimizer.converged_all when the log_prob is batch-friendly.

Padarn commented 4 years ago

Would it be valuable to have laplace_approx(target_density, location) exposed, or should this look more like the Edward version that does the location finding.

From looking at the Edward implementation, there is a Base class Inference for which tfp doesn't seem to have an equivalent of - or am I missing something looking through the code?

jeffpollock9 commented 3 years ago

If at all useful to anyone coming across this issue, I coded up some Laplace approximation stuff recently. Usage is in this comment, which also contains a gist of the implementation: https://github.com/tensorflow/probability/pull/1178#issuecomment-737183260

Would be happy to spend some time on this making a more formal proposal for adding something like this to TFP if that was of interest.