This is the PhD thesis of Yarin Gal, who is associate professor at Oxford now.
The main goal of this thesis is to develop practical tools to reason about uncertainty in deep learning.
epistemic(reducible) uncertainty can be reduced with more data, while aleatoric(irreducible) uncertainty can not.
Applications of model uncertainty:
(1)active learning
(2)efficient exploration in deep reinforcement learning
Bayesian deep learning:
uncertainty is of great importance in deep reinforcement learning- with uncertainty information an agent can decide when to exploit rewards it knows of, and when to explore its environment. To make use of the dropout Q-network's uncertainty estimates, we use Thompson sampling instead of epsilon greedy.
PILCO uses a Gaussian process which can model the
dynamics’ output uncertainty, but cannot scale to high dimensional observation spaces. In
contrast, Deep PILCO uses a deep neural network capable of scaling to high dimensional
observations spaces. Like PILCO, our policy-search algorithm alternates between fitting
a dynamics model to observed transitions data, evaluating the policy using dynamics
model predictions of future states and costs, and then improving the policy.
Replacing PILCO’s GP with a deep network is a surprisingly complicated endeavour
though, as we wish our dynamics model to maintain its probabilistic nature, capturing
1) output uncertainty, and 2) input uncertainty.
This is the PhD thesis of Yarin Gal, who is associate professor at Oxford now.
The main goal of this thesis is to develop practical tools to reason about uncertainty in deep learning. epistemic(reducible) uncertainty can be reduced with more data, while aleatoric(irreducible) uncertainty can not. Applications of model uncertainty: (1)active learning (2)efficient exploration in deep reinforcement learning
Bayesian deep learning:
uncertainty is of great importance in deep reinforcement learning- with uncertainty information an agent can decide when to exploit rewards it knows of, and when to explore its environment. To make use of the dropout Q-network's uncertainty estimates, we use Thompson sampling instead of epsilon greedy.