Discrete-state discrete-action MDPs

[jlperla]

https://julia.quantecon.org/dynamic_programming/discrete_dp.html and also https://julia.quantecon.org/multi_agent_models/aiyagari.html also uses it

• Uses a relatively standard MDP algorithm with discrete states/values with VFI and PFI and modified policy iteration
• Stationary distribution of the solution
• Simulations after solution calculated.
• Most are probably available in external libraries for MDPs with a full model. Algorithms are simple but tricky to get high-performance without inplace operations/etc.

[jlperla]

https://github.com/JuliaPOMDP/DiscreteValueIteration.jl is one algorithm in that package.