search

tensorflow / agents

TF-Agents: A reliable, scalable and easy to use TensorFlow library for Contextual Bandits and Reinforcement Learning.
Apache License 2.0
2.78k stars 719 forks source link

Multi-dimensional action DQN #97

Open zh4ngx opened 5 years ago

zh4ngx commented 5 years ago

I see this TODO in the code for this file:

https://github.com/tensorflow/agents/blob/master/tf_agents/agents/dqn/dqn_agent.py

    # TODO(oars): Get DQN working with more than one dim in the actions.

Is there an ETA on this? I have a use case right now that this would enable.

kbanoop commented 5 years ago

No one is actively working on this AFAIK, so contributions are welcome.

A simple way to get this to work without modifying the agent is to create an environment wrapper that flattens the actions from your environment to 1 dimension. https://github.com/tensorflow/agents/blob/master/tf_agents/environments/wrappers.py

e.g. if your env returns actions in {0, 1, 2} x {0, 1, 2}, the wrapper would modify the actions to be in the range {0, 1, ... 8}

This will allow your environment to work with the current DQN agent.

seungjaeryanlee commented 5 years ago

I will try to implement this wrapper!

axitkhurana commented 5 years ago

Hi @seungjaeryanlee, are you still working on this?