Reinforcement Learning

[tbreloff]

I'm starting to work on a framework for reinforcement learning in https://github.com/tbreloff/Reinforce.jl, and I was thinking it would be nice to include the core abstractions in LearnBase so that I could have standalone environments or agents without requiring a dependency on Reinforce.jl.

I propose adding this to LearnBase:

abstract AbstractEnvironment
abstract AbstractAgent

# `r, s, A = observe!(env)` should return `(reward, state, actions)`
# Note: most environments will not implement this directly
function observe!(env::AbstractEnvironment)
    reward!(env), state!(env), actions(env)
end

# `r = reward!(env)` returns the current reward, optionally updating it first
function reward end
function reward! end

# `s = state!(env)` returns the current state, optionally updating it first
function state end
function state! end

# `A = actions(env)` returns a list/set/description of valid actions
function actions end

# `a = action(agent, r, s, A)` should take in the last reward `r`, current state `s`, 
#      and set of valid actions `A`, then return an action `a`
function action end

Any thoughts/objections?

[ahwillia]

Looks good to me. I've been wanting to reproduce the basic examples from Sutton and Barto book as a learning exercise - it would be great if I could use what you're working on!

Edit: After giving a little more thought... I like this API, but can't think of a reason why they would have to live in LearnBase rather than have Reinforce as a standalone package. I guess it doesn't hurt to have it in LearnBase though.

[tbreloff]

I'm going to hold off on this, and keep everything in Reinforce for now. The interface isn't stable enough and anyone using it would want other things in Reinforce anyways.