sail-sg / envpool

C++-based high-performance parallel environment execution engine (vectorized env) for general RL environments.
https://envpool.readthedocs.io
Apache License 2.0
1.07k stars 99 forks source link

[BUG] Using `env.reset()` and `env.async_reset()` results in illegal instructions #173

Closed vwxyzjn closed 1 year ago

vwxyzjn commented 2 years ago

Describe the bug

env.reset() and env.async_reset() results in illegal instructions. This warrants a better error message.

To Reproduce

import envpool
import numpy as np

# make asynchronous
num_envs = 64
batch_size = 16
env = envpool.make("Pong-v5", env_type="gym", num_envs=num_envs, batch_size=batch_size)
action_num = env.action_space.n
env.reset()
env.async_reset()  # send the initial reset signal to all envs
while True:
    obs, rew, done, info = env.recv()
    env_id = info["env_id"]
    action = np.random.randint(action_num, size=batch_size)
    env.send(action, env_id)
    print(action)
[0 4 0 3 1 1 2 1 0 3 0 3 3 4 4 5]
[2 1 0 2 1 2 4 2 5 0 1 0 2 4 2 2]
[1 5 2 0 5 4 1 2 0 1 2 0 0 3 2 0]
[5 5 4 3 5 0 3 5 5 5 2 5 0 2 0 2]
[1 3 5 3 4 2 4 1 1 5 3 5 1 5 5 1]
[5 4 4 1 4 3 2 0 4 1 1 5 2 4 5 5]
[1 0 1 0 1 3 3 3 2 0 0 3 4 5 0 2]
Illegal Instruction! 69
Illegal Instruction! 8a
Illegal Instruction! 2
Illegal Instruction! 2
Illegal Instruction! 2
Illegal Instruction! 2
Illegal Instruction! 2
Illegal Instruction! [4 1 4 1 2 1 2 2 3 2 4 0 5 4 0 2]
2
Illegal Instruction! e8
Illegal Instruction! 30
Illegal Instruction! 2
Illegal Instruction! 38
Illegal Instruction! 84
Illegal Instruction! a
Illegal Instruction! b9
[1 0 1 4 4 5 3 3 2 1 1 4 3 1 0 0]
[2 5 2 1 0 2 5 0 2 3 1 4 4 0 3 3]
[4 0 3 2 5 2 2 4 4 0 2 2 4 4 0 3]
Illegal Instruction! b2
[3 2 3 0 2 1 0 1 4 2 1 0 3 3 2 3]
[4 5 2 3 1 0 2 5 5 2 2 2 5 5 5 4]
[4 1 2 0 5 2 0 2 4 1 4 2 4 0 4 4]
[0 4 2 0 5 4 4 1 2 4 0 0 1 0 4 4]
[2 1 5 4 3 3 2 5 2 2 0 4 5 4 5 0]
[2 0 5 2 3 3 2 4 1 2 4 2 0 0 0 0]
[0 3 1 2 0 5 2 5 1 3 0 3 2 0 4 5]
[1 0 0 3 2 3 2 1 2 4 0 4 4 4 4 2]
[4 2 5 5 4 1 3 2 4 4 4 0 1 3 0 5]
[5 5 0 3 3 4 0 1 3 4 4 3 2 1 5 3]
[2 1 2 2 4 4 2 4 4 3 3 5 2 1 1 3]
Illegal Instruction! 2

Expected behavior

A better error message should be available like saying env.reset() and env.async_reset() cannot be used at the same time.

SobhanMP commented 1 year ago

is this safe to ignore?

Trinkle23897 commented 1 year ago

No, it is an error you need to care about. By design, envpool never allows users to use multiple reset function calls. You should do either one of the following patterns:

env.async_reset()
env.recv()
env.send(...)
env.recv()
env.send(...)
...

or

env.reset()
env.step(...)
env.step(...)
...
vwxyzjn commented 1 year ago

Makes sense. Closing the issue.