Formulation

[oyamad]

The current code solves dynamic programs with

• reward function f(s, x) and
• state transition function g(s, x, e)

where

1. s is an N-dimensional continuous state variable
2. x is a 1-dimensional continuous action variable and
3. e is a discrete random variable, whose distribution is state independent.

Discussion:

1. Do we want to allow discrete state variables in some dimensions?
2. We want to allow discrete actions (#5) and multidimensional actions.
3. Do we want to allow state dependent distributions?

[sglyon]

1. I think we should allow for discrete states

2. yes and yes

3. Is this different from discrete states?