search

PKU-MARL / Multi-Agent-Transformer

323 stars 59 forks source link

Transformer memory #17

Closed rsanchezmo closed 7 months ago

rsanchezmo commented 1 year ago

Hi! Have you considered to add previous observations from the agents to the feed the transformer? The encoder should pay attention to relations between agents by also considering the temporal dimension, which may lead to a better action planning.

As far as I know, the current implementation of MAT does not support previous observations and because of that, it has no memory.

morning9393 commented 1 year ago

hiya, thanks for your attention! yep, we used to considered adding an extra attention block at the beginning of encoder for modeling history information, and we believe it is very likely to improve performance further especially in partially observable scenarios.

actually, I've been meaning to implement this feature to this repo, but haven't done it yet due to procrastinations. (struggling for this.....)

rsanchezmo commented 1 year ago

Hi!

Do you plan to keep the same input [joint observations history]? Do you think now passing previous actions as a new input would improve the model, as well as previous rewards, etc [based on Decision Transformer or ODT]?

Thanks in advance!

MPHarryZhang commented 1 year ago

Hi!

Do you plan to keep the same input [joint observations history]? Do you think now passing previous actions as a new input would improve the model, as well as previous rewards, etc [based on Decision Transformer or ODT]?

Thanks in advance!

Hello, I also found this problem in the research, and I am trying it recently. Could I discuss this problem with you?

rsanchezmo commented 1 year ago

@MPHarryZhang reach me by email!