search

PacktPublishing / Deep-Reinforcement-Learning-Hands-On

Hands-on Deep Reinforcement Learning, published by Packt
MIT License
2.83k stars 1.28k forks source link

Multi-agent D4PG #41

Closed canteli closed 5 years ago

canteli commented 5 years ago

Thank you for these useful examples. I am trying to implement D4PG in multiple agents that interact with each other, share the same reward, but each agent takes its own actions. I wonder if you had any tips on how I could modify the code to achieve this. Thank you in advance.

Shmuma commented 5 years ago

Hi!

Sorry, don't have much experience with multi-agent RL, just have heard that this is a cool and promising direction :).

In terms of sharing parts of the network among the agents this is doable in pytorch even in separate subprocesses. For example, you can check my article about pong speed up here: https://medium.com/mlreview/speeding-up-dqn-on-pytorch-solving-pong-in-30-minutes-81a1bd2dff55

In case of several agents living in the same processes, this is even simpler -- just pass output from common net to specific nets.

canteli commented 5 years ago

Hi Max,

Thank you for your feedback. Actually, I just want multiple RL controllers to take different actions on a same environment, which will output a common reward to all of them. However, in the code on Github the functions take an agent and the environment as inputs, so I'm not sure how train multiple agents to control different action variables of the same environment.

In my case, the environment is a community of buildings, the reward is the total peak energy demand, and I want every building to have a RL controller that takes actions on its building but gets rewards from the community. (Then I'll make them share states or weights with each other, but for now I just want multiple agents to interact with the same environment).

Thank you for your quick reply. Best,

José Ramón Vázquez-Canteli

Graduate Research Assistant

On Thu, Mar 14, 2019 at 11:51 AM Max Lapan notifications@github.com wrote:

Hi!

Sorry, don't have much experience with multi-agent RL, just have heard that this is a cool and promising direction :).

In terms of sharing parts of the network among the agents this is doable in pytorch even in separate subprocesses. For example, you can check my article about pong speed up here: https://medium.com/mlreview/speeding-up-dqn-on-pytorch-solving-pong-in-30-minutes-81a1bd2dff55

In case of several agents living in the same processes, this is even simpler -- just pass output from common net to specific nets.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/PacktPublishing/Deep-Reinforcement-Learning-Hands-On/issues/41#issuecomment-472959523, or mute the thread https://github.com/notifications/unsubscribe-auth/AcLQD3E_uZs2A4QoeULPU27dL6BAUF8Pks5vWn3tgaJpZM4btwjW .