[Bug Report] Getting Error from "D:\PongDQN_RL\venv\lib\site-packages\gym\wrappers\compatibility.py"

[creativekids11]

The Error Message:

logger.warn(f"{pre} is not within the observation space.") Traceback (most recent call last): File "E:\PongDQN_RL\02_dqn_pong.py", line 141, in agent = Agent(env, buffer) File "E:\PongDQN_RL\02_dqn_pong.py", line 63, in init self._reset() File "E:\PongDQN_RL\02_dqn_pong.py", line 66, in reset self.state, = self.env.reset() File "E:\PongDQN_RL\venv\lib\site-packages\gym\core.py", line 379, in reset obs, info = self.env.reset(kwargs) File "E:\PongDQN_RL\lib\wrappers.py", line 117, in reset return self.observation(self.env.reset()) File "E:\PongDQN_RL\venv\lib\site-packages\gym\core.py", line 379, in reset obs, info = self.env.reset(kwargs) File "E:\PongDQN_RL\venv\lib\site-packages\gym\core.py", line 379, in reset obs, info = self.env.reset(**kwargs) File "E:\PongDQN_RL\lib\wrappers.py", line 20, in reset (obs, reward, terminated, truncated, info) = self.env.step(1) File "E:\PongDQN_RL\lib\wrappers.py", line 43, in step obs, reward, terminated, truncated, info = self.env.step(action) File "E:\PongDQN_RL\venv\lib\site-packages\gym\wrappers\order_enforcing.py", line 37, in step return self.env.step(action) File "E:\PongDQN_RL\venv\lib\site-packages\gym\wrappers\env_checker.py", line 37, in step return env_step_passive_checker(self.env, action) File "E:\PongDQN_RL\venv\lib\site-packages\gym\utils\passive_env_checker.py", line 214, in env_step_passive_checker result = env.step(action) File "E:\PongDQN_RL\venv\lib\site-packages\gym\wrappers\compatibility.py", line 105, in step obs, reward, done, info = self.env.step(action) ValueError: too many values to unpack (expected 4)

Details

I saw everywhere on internet but no solutions I'm searching from a week, tried all fixes on internet but no update I think its kinda an installation bug or an not fixed bug

If anyone thinking, I'm training pongnoframeskip-v4 DQN from atari set According to me i did the installation correctly in the venv, virtual environment(cmd i used to create it - python -m venv venv) gym(0.26.2), gym[atari] and autorom version is the latest i also did: AutoRom command in the venv

[pseudo-rnd-thoughts]

Could you provide a minimal working example to test with? It is unclear from your stack trace what the issue is

[creativekids11]

PongDQN_RL.zip This is the code (02_dqn_pong.py) to train (where the error occured)

[creativekids11]

BTW, pytorch with version 2.2.0 cuda 11.8 is also installed in the same env. IDK if it can cause any problem.

[pseudo-rnd-thoughts]

If you using gym v0.26.2 and the latest atari version then you shouldn't be using apply_api_compatibility

[creativekids11]

If you using gym v0.26.2 and the latest atari version then you shouldn't be using apply_api_compatibility

Didn't work after removing it

[creativekids11]

The error changed to:

E:\PongDQN_RL\venv\lib\site-packages\gym\utils\passive_envchecker.py:233: DeprecationWarning: np.bool8 is a deprecated alias for `np.bool`. (Deprecated NumPy 1.24) if not isinstance(terminated, (bool, np.bool8)): Traceback (most recent call last): File "E:\PongDQN_RL\02_dqn_pong.py", line 141, in agent = Agent(env, buffer) File "E:\PongDQN_RL\02_dqn_pong.py", line 63, in init self._reset() File "E:\PongDQN_RL\02_dqn_pong.py", line 66, in reset self.state, = self.env.reset() File "E:\PongDQN_RL\venv\lib\site-packages\gym\core.py", line 379, in reset obs, info = self.env.reset(kwargs) File "E:\PongDQN_RL\lib\wrappers.py", line 117, in reset return self.observation(self.env.reset()) File "E:\PongDQN_RL\venv\lib\site-packages\gym\core.py", line 379, in reset obs, info = self.env.reset(kwargs) File "E:\PongDQN_RL\venv\lib\site-packages\gym\core.py", line 379, in reset obs, info = self.env.reset(**kwargs) File "E:\PongDQN_RL\lib\wrappers.py", line 20, in reset (obs, reward, terminated, truncated, info) = self.env.step(1) ValueError: not enough values to unpack (expected 5, got 4)

[creativekids11]

Have you got the error replicated with the code. Or else i can share name and versions of all the libraries for you to try in an venv.

[pseudo-rnd-thoughts]

Can you make a minimal working example of the code with the bug, there is too much for me to look through. Try testing commenting out a wrapper each time to see which one causes the issue

[creativekids11]

I managed to fix this error but now the input format is not approving, to the model.

[creativekids11]

Could you kindly provide me with an latest code with dqn which can solve pong. As there are no updates online, plus no available latest code.

[pseudo-rnd-thoughts]

There are a host of projects that implement DQN for you to see: CleanRL, Stablebaselines3, Tianshou, etc

[creativekids11]

CleanRL docs was very helpful, thanks: https://docs.cleanrl.dev/rl-algorithms/dqn/