search

EmuKit / emukit

A Python-based toolbox of various methods in decision making, uncertainty quantification and statistical emulation: multi-fidelity, experimental design, Bayesian optimisation, Bayesian quadrature, etc.
https://emukit.github.io/emukit/
Apache License 2.0
602 stars 128 forks source link

Model calibration #222

Open ghost opened 5 years ago

ghost commented 5 years ago

Dear Emukit developper(s),

I recently found the Emukit library and I really like it. There is one feature that would make it 100% complete for what I need: Bayesian calibration (as in https://www.asc.ohio-state.edu/statistics/comp_exp/jour.club/KennedyOHagan_2002.pdf)

Do you know of any example/application of this method in Emukit or any other package based on GPFlow?

javiergonzalezh commented 5 years ago

Hi @szboksteen,

Thanks for your interest in Emukit. Can you provide some details about what you are trying to do? Having an specific example will help us to identify better what is missing.

Also, have you try to start with a wrapper for GPflow like the one we have for GPy? This will give you direct access to all the stuff already implemented. Definitely something nice to have,

javiergonzalezh commented 5 years ago

Note that for the GPflow model wrapper you don't need to start with all the interfaces. IModel and IDifferentiable shpuld give you good coverage already.

ghost commented 5 years ago

Hello, thanks for replying. What I would like to do is retrieve posterior distributions of uncertain model parameters based on experimental observations. The statistical model:

y = eta(x,theta) + delta(x) + epsilon,

where y - output (measurable by experiment) eta - original model output x - settings (measurable by experiment) theta - unknown model parameters delta - systematic model bias epsilon - noise.

The result of calibration is P(theta | x_observed,y_observed) = posterior. This can then be used as en emulator to make predictions P(y_new | x_new, theta_posterior).

Thanks for the advise about creating a wrapper.

apaleyes commented 5 years ago

Hey @szboksteen , did you end up using emukit? If so, we would love to know your use case!

ghost commented 5 years ago

Hello Andrei,

I have used emukit only for DOE at the moment. I have directly used GPFlow for building the emulators. If I find a way to do Bayesian Calibration (Kennedy and O'Hagan type) with emukit, then I will definitely use it more. Any progress on this?

Regards Sowande

On Thu, Sep 5, 2019 at 10:19 AM Andrei Paleyes notifications@github.com wrote:

Hey @szboksteen https://github.com/szboksteen , did you end up using emukit? If so, we would love to know your use case!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/amzn/emukit/issues/222?email_source=notifications&email_token=AJWG7ZPNAZ6YZV5NJHYA64DQIC6I5A5CNFSM4HXFGH42YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD56IQMQ#issuecomment-528255026, or mute the thread https://github.com/notifications/unsubscribe-auth/AJWG7ZMAGK7S4EQBVACN4TLQIC6I5ANCNFSM4HXFGH4Q .

-- Met vriendelijke groet/Kind regards,

Dr. ir. S.Z. Boksteen | Directeur | AMELA Process Optimization B.V.| Moliereweg 576 | 3076 GV Rotterdam | Tel.: +316-49137876 | Web: www.amela-po.com

KvK-nr.: 73900583

apaleyes commented 5 years ago

@szboksteen no, not really. we are currently focusing on other applications of emukit, i guess mainly because we haven't encountered use case for calibration ourselves