Algorithmic issues

[AHPUymhd]

{ "base_config": "configs/HighwayEnv/agents/DQNAgent/ddqn.json", "model": { "type": "EgoAttentionNetwork", "embedding_layer": { "type": "MultiLayerPerceptron", "layers": [64, 64], "reshape": false, "in": 7 }, "others_embedding_layer": { "type": "MultiLayerPerceptron", "layers": [64, 64], "reshape": false, "in": 7 }, "self_attention_layer": null, "attention_layer": { "type": "EgoAttention", "feature_size": 64, "heads": 2 }, "output_layer": { "type": "MultiLayerPerceptron", "layers": [64, 64], "reshape": false } }, "gamma": 0.99, "batch_size": 64, "memory_capacity": 15000, "target_update": 512 } Hello, may I ask if this ettention code is run with the algorithm of the attention mechanism? But I see that it is an algorithm that inherits DQN, and I see that there are only DQN, DDQN and DOUBLE DQN in the algorithm library, and whether there are PPO and other algorithms, I am very much looking forward to your reply．

[AHPUymhd]

@eleurent

[eleurent]

Hi, DQN (and its variants Dueling DQN and Double Dueling DQN) is the learning algorithm (just like PPO is), while Attention/Transformer is the network architecture, which is being trained by the RL algorithm. You fill find the attention implementation in agents/common/models.py.

While I wanted to implement PPO, I never found the time.

But I wrote this script which combines StableBaselines3's PPO with my implementation of attention as a CustomPolicy.

[AHPUymhd]

Hi, DQN (and its variants Dueling DQN and Double Dueling DQN) is the learning algorithm (just like PPO is), while Attention/Transformer is the network architecture, which is being trained by the RL algorithm. You fill find the attention implementation in agents/common/models.py.

While I wanted to implement PPO, I never found the time.

But I wrote this script which combines StableBaselines3's PPO with my implementation of attention as a CustomPolicy.

Wow, I am really appreciate your help, I'm going to learn your code!