search

TheButlah / makrl

makrl - modular algorithm kit for reinforcement learning
4 stars 1 forks source link

Figure out what the relationship is between `Environment` and OpenAI environment class #15

Closed TheButlah closed 5 years ago

TheButlah commented 5 years ago

Our Environment class was originally just going to wrap the OpenAI environments. However, I can't help but feel that it might be better for Environment to extend from them. What are people's thoughts? I can think of the following benefits of incorporating the Environment class into an OpenAI compatible system:

  1. It would work in other people's OpenAI code, in a format they expect
  2. It might make how we deal with the OpenAI envs easier to implement
  3. It might make sure that we stay consistent with OpenAI's state representation

Drawbacks:

  1. It might conflict with other libraries' environments, such as Facebook ELF, Microsoft Malmo, etc
  2. It might prevent us from modifying OpenAI's state representation
TheButlah commented 5 years ago

Quite a while ago we determined that there will be no formal Environment class as part of our framework. It will instead be structured in a way independent of the particular environment class. Because we need the agent to have some set API it can depend on, it currently seems most likely that we will either interact with the environment using the OpenAI environment api, or else the experiment code will handle the interaction of the agent and environment (less likely)

TheButlah commented 5 years ago

After building BatchedEnv, it became clear that only duck typing is necessary. As long as an environment can expose the same basic API as a Gym environment, the code will work. If it didn't, you would get an exception and it would crash.