search

SciML / Surrogates.jl

Surrogate modeling and optimization for scientific machine learning (SciML)
https://docs.sciml.ai/Surrogates/stable/
Other
330 stars 69 forks source link

Information theory criteria for design space exploration #267

Open ArnoStrouwen opened 3 years ago

ArnoStrouwen commented 3 years ago

Taken from a discussion with @ludoro on SciML slack:

As an alternative to acquisition functions, such as expected improvement, information theory based criteria could be used to pick new points to evaluate the objective at. Such as Fisher information or Shannon information. Some of these methods are described in: https://link.springer.com/chapter/10.1007/978-1-4939-8847-1_6 These information theory based methods have mostly been used to improve prediction variance over the entire design space, but recently they have also been used to find the maximum of the objective. https://www.tandfonline.com/doi/abs/10.1080/16843703.2018.1542965

atiyo commented 3 years ago

To take this idea a bit further: the whole notion of efficient data acquisition for surrogates is essentially the same as active learning.From this perspective, one might dip into the active learning literature for a whole set of applicable techniques (including the information theoretic driven ones).

Edit: somehow submitted comment before I finished typing!

ArnoStrouwen commented 2 years ago

https://bayesoptbook.com/ Overview of many interesting techniques such as entropy search.