elliottower
commented
1 year ago Hi, thanks for the bug report. Your code is a bit complicated, so I tried to reproduce the problem in as simple an example as possible, and I found the rewards were working fine. The following code prints any rewards which are nonzero.
env = pong_v3.env()
env.reset(seed=42)
turn = 0
for agent in env.agent_iter():
observation, reward, termination, truncation, info = env.last()
if reward != 0:
print(f"TURN {turn} | {agent}: {reward}")
if termination or truncation:
break
else:
if "action_mask" in info:
mask = info["action_mask"]
elif isinstance(observation, dict) and "action_mask" in observation:
mask = observation["action_mask"]
else:
mask = None
action = env.action_space(agent).sample(mask) # this is where you would insert your policy
env.step(action)
turn +=1
env.close()This should have an output similar to this:
TURN 266 | first_0: 1 TURN 267 | second_0: -1 TURN 548 | first_0: 1 TURN 549 | second_0: -1 TURN 830 | first_0: 1 TURN 831 | second_0: -1
However, I see that our average_total_reward utility doesn't actually work with zero sum games, which is most of the environments, so I think I will make a PR adding a utility to track the scores and return it after running games. As a basic way to track scores, try this code:
env = pong_v3.env()
env.reset(seed=42)
total_rewards = {agent: 0 for agent in env.agents}
for agent in env.agent_iter():
observation, reward, termination, truncation, info = env.last()
# When the enemy scores, your agent gets -1 return, but the total score for your agent stays the same
# The environment ends when one player reaches 20 points
if reward > 0:
total_rewards[agent] += reward
if termination or truncation:
print(f"Final rewards: {total_rewards}")
break
else:
if "action_mask" in info:
mask = info["action_mask"]
elif isinstance(observation, dict) and "action_mask" in observation:
mask = observation["action_mask"]
else:
mask = None
action = env.action_space(agent).sample(mask) # this is where you would insert your policy
env.step(action)
env.close()This will tally up all scores received by agents over the course of the game:
Final rewards: {'first_0': 14, 'second_0': 20}
xiezhipeng-git
Describe the bug
pettingzoo.atari pong_v3 reward need >0 Name: pettingzoo Version: 1.23.1
Code example
System info
Describe how PettingZoo was installed (pip, source, ...) pip Version of (by pettingzoopettingzoo.version) 1.23.1 What OS/version you're using. Note that while we will accept PRs to improve Window's support, we do not officially support it. wsl:ubuntu-22.04 (windows11) Python version 3.10.6
Additional context
No response
Checklist