search

JuliaML / Reinforce.jl

Abstractions, algorithms, and utilities for reinforcement learning in Julia
Other
200 stars 35 forks source link

Is the state part of the environment? #5

Closed zsunberg closed 7 years ago

zsunberg commented 7 years ago

Hi guys, excited to see a reinforcement learning interface for Julia! I have a question about the interface. Is the state part of the environment?

If the state is part of the environment, then what is the reason for having it as an additional argument to step!? If the state is not part of the environment, then why would the environment be mutated in a call to step!?

The only RL package that I am really familiar with is OpenAI Gym, where the state is part of the environment. Perhaps it would make sense to follow their example since people are familiar with it, it has been successful, and it would allow simple interaction between environments and solvers written in julia and python.

tbreloff commented 7 years ago

Hi @zsunberg and thanks for the interest. You should probably review #2 and join the gitter chat to get more background on decisions that have been made. Really the design is to allow for maximum flexibility, and easier use than the gym. For example, you can do iteration like:

env = ...
policy = ...
for sars in Episode(env,policy)
    # optionally do something?
end

In practice this is much simpler and elegant than the gym api (IMO of course). You can look at the code of OpenAIGym.jl to get a feel of how they relate.

zsunberg commented 7 years ago

(continuing on gitter)