search

marlbenchmark / off-policy

PyTorch implementations of popular off-policy multi-agent reinforcement learning algorithms, including QMix, VDN, MADDPG, and MATD3.
MIT License
386 stars 67 forks source link

Bug with idx_range, causing error with Prioritized ER #3

Open Maxtoq opened 2 years ago

Maxtoq commented 2 years ago

Describe the bug When using PER with QMIX, an issue arises with the idx_range returned by the insert function of RecPolicyBuffer:

line 267, in insert for idx in range(idx_range[0], idx_range[1]): IndexError: index 1 is out of bounds for axis 0 with size 1`.

The reason seems to be that the insert function takes as first parameter the number of episodes to insert, instead of the number of steps (as the function description explains it).

To try fixing the issue, I computed the number of steps to insert, from the number of episodes, as such: from line 164:

episode_length = acts.shape[0]
assert episode_length == self.episode_length, ("different dimension!")
# My line
number_insert_steps = num_insert_episodes * episode_length

And then I replace num_insert_episode by num_insert_steps in the rest of the function.

This seems to work. However, I am not completely sure that it was intended like that.

Tell me if I am wrong and the issue is related to something else.

zhouweiqing-star commented 2 years ago

Describe the bug When using PER with QMIX, an issue arises with the idx_range returned by the insert function of RecPolicyBuffer:

line 267, in insert for idx in range(idx_range[0], idx_range[1]): IndexError: index 1 is out of bounds for axis 0 with size 1`.

The reason seems to be that the insert function takes as first parameter the number of episodes to insert, instead of the number of steps (as the function description explains it).

To try fixing the issue, I computed the number of steps to insert, from the number of episodes, as such: from line 164:

episode_length = acts.shape[0]
assert episode_length == self.episode_length, ("different dimension!")
# My line
number_insert_steps = num_insert_episodes * episode_length

And then I replace num_insert_episode by num_insert_steps in the rest of the function.

This seems to work. However, I am not completely sure that it was intended like that.

Tell me if I am wrong and the issue is related to something else.

Hello, may I ask you if you have run this program? I have encountered some problems. I wonder if you can help me. that:Failed to detect the name of this notebook, you can set it manually with the WANDB_NOTEBOOK_NAME environment variable to enable code saving. wandb: Currently logged in as: zhouweiqing (use wandb login --relogin to force relogin) wandb: ERROR Error while calling W&B API: project not found (<Response [404]>) requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://api.wandb.ai/graphql

zhouweiqing-star commented 2 years ago

Hello, I have encountered some problems. I wonder if you can help me. that:Failed to detect the name of this notebook, you can set it manually with the WANDB_NOTEBOOK_NAME environment variable to enable code saving. wandb: Currently logged in as: zhouweiqing (use wandb login --relogin to force relogin) wandb: ERROR Error while calling W&B API: project not found (<Response [404]>) requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://api.wandb.ai/graphql i can not solve it

Maxtoq commented 2 years ago

Hi, this is not related to my problem, so you should make another issue. This would allow people with the same issue finding a solutions as well. I'm creating another issue to answer you.