PyBADS is a Python implementation of the Bayesian Adaptive Direct Search (BADS) algorithm for solving difficult and mildly expensive optimization problems, originally implemented in MATLAB. BADS has been intensively tested for fitting a variety of computational models, and is currently being used in many computational labs around the world (see Google Scholar for many example applications).
In a benchmark with real model-fitting problems from computational and cognitive neuroscience, BADS performed on par or better than many other common and state-of-the-art optimizers, as shown in the original NeurIPS paper [1].
PyBADS requires no specific tuning and runs off-the-shelf like other Python optimizers (e.g., scipy.optimize.minimize
).
Note: If you are interested in estimating posterior distributions (i.e., uncertainty and error bars) over model parameters, and not just point estimates, you might also want to check out Variational Bayesian Monte Carlo for Python (PyVBMC), a package for Bayesian posterior and model inference which can be used in synergy with PyBADS.
The full documentation is available at: https://acerbilab.github.io/pybads/
BADS is effective when:
D = 20
or so.PyBADS is available via pip
and conda-forge
, and requires Python version 3.9 or newer.
python -m pip install pybads
or:
conda install --channel=conda-forge pybads
jupyter
executable is found on your path then import errors may arise.
conda install jupyter
If you are running Python 3.11 and get an UnsatisfiableError
you may need to install Jupyter from conda-forge
:
conda install --channel=conda-forge jupyter
The example notebooks can then be accessed by running
python -m pybads
If you wish to install directly from latest source code, please see the instructions for developers and contributors.
The typical workflow of PyBADS follows four steps:
Running the optimizer in step 3 only involves a couple of lines of code:
from pybads import BADS
# ...
bads = BADS(target, x0, lower_bounds, upper_bounds, plausible_lower_bounds, plausible_upper_bounds)
optimize_result = bads.optimize()
with input arguments:
target
: the target function, it takes as input a vector and returns its function evaluation;x0
: the starting point of the optimization problem. If it is not given, the starting point is randomly drawn from the problems bounds;lower_bounds
and upper_bounds
: hard lower and upper bounds for the optimization region (can be -inf
and inf
, or bounded);plausible_lower_bounds
and plausible_upper_bounds
: plausible lower and upper bounds, that represent our best guess at bounding the region where the solution might lie;non_box_cons
(optional): a callable function that denotes non-box constraint violations.The outputs are:
optimize_result
: a OptimizeResult
which presents relevant information about the solution and the optimization problem. In particular:
"x"
: the minimum point found by the optimizer;"fval"
: the value of the function at the given solution.For a full list and description of the entries of the optimize_result
object, see the OptimizeResult class documentation.
Once installed, example Jupyter notebooks can be found in the pybads/examples
directory. They can also be viewed statically on the main documentation pages. These examples represent a full tutorial that will walk you through the basic usage of PyBADS as well as some if its more advanced features, such as noisy targets.
For practical recommendations, such as how to set lower_bounds
, upper_bounds
and the plausible bounds, check out the FAQ on the BADS wiki. Even though the FAQ refers to the MATLAB version of BADS, most of the concepts apply equally to PyBADS.
PyBADS/BADS follows a mesh adaptive direct search (MADS) procedure for function minimization that alternates poll steps and search steps (see Fig 1).
Fig 1: BADS procedure
See here for a visualization of several optimizers at work, including BADS.
See the original BADS paper for more details (Acerbi and Ma, 2017).
PyBADS is under active development. The original BADS algorithm has been extensively tested in several benchmarks and published papers, and the some benchmarks have been replicated with PyBADS. However, as with any optimization method, you should double-check your results.
If you have trouble doing something with PyBADS, spot bugs or strange behavior, or you simply have some questions, please feel free to:
Singh, S. G. & Acerbi, L. (2024). PyBADS: Fast and robust black-box optimization in Python. Journal of Open Source Software, 9(94), 5694, https://doi.org/10.21105/joss.05694
Acerbi, L. & Ma, W. J. (2017). Practical Bayesian Optimization for Model Fitting with Bayesian Adaptive Direct Search. In Advances in Neural Information Processing Systems 31: 8222-8232. (paper + supplement on arXiv, NeurIPS Proceedings)
Please cite both references if you use PyBADS in your work (the 2017 paper introduced the framework, and the latest one is its Python library). You can cite PyBADS in your work with something along the lines of
We optimized the log likelihoods of our models using Bayesian adaptive direct search (BADS; Acerbi and Ma, 2017), via the PyBADS software (Singh and Acerbi, 2024). PyBADS alternates between a series of fast, local Bayesian optimization steps and a systematic, slower exploration of a mesh grid.
Besides formal citations, you can demonstrate your appreciation for PyBADS in the following ways:
Note: If you are interested in estimating posterior distributions (i.e., uncertainty and error bars) over model parameters, and not just point estimates, you might also want to check out Variational Bayesian Monte Carlo for Python (PyVBMC), a package for Bayesian posterior and model inference which can be used in synergy with PyBADS.
@article{singh2024pybads,
title={{PyBADS}: {F}ast and robust black-box optimization in {P}ython},
author={Gurjeet Sangra Singh and Luigi Acerbi},
publisher = {The Open Journal},
journal = {Journal of Open Source Software},
year = {2024},
volume = {9},
number = {94},
pages = {5694},
url = {https://doi.org/10.21105/joss.05694},
doi = {10.21105/joss.05694},
}
@article{acerbi2017practical,
title={Practical {B}ayesian Optimization for Model Fitting with {B}ayesian Adaptive Direct Search},
author={Acerbi, Luigi and Ma, Wei Ji},
journal={Advances in Neural Information Processing Systems},
volume={30},
pages={1834--1844},
year={2017}
}
PyBADS is released under the terms of the BSD 3-Clause License.
PyBADS was developed by members (past and current) of the Machine and Human Intelligence Lab at the University of Helsinki. Work on the PyBADS package was supported by the Research Council of Finland Flagship programme: Finnish Center for Artificial Intelligence FCAI.