Bug Report. gym wrapper returns np._bool causes error when using stable-baselines3

[andykim0723]

I was trying to connect RLBench with stable-baselines3 and found a minor error. I used the code from #103, and changed state to vision.

import gym
import rlbench.gym
import stable_baselines3.common.env_checker
from stable_baselines3 import PPO

env = gym.make('reach_target-vision-v0')

print(stable_baselines3.common.env_checker.check_env(env))
model = PPO('MlpPolicy', env, verbose=1)
model.learn(total_timesteps=10000)

obs = env.reset()
for i in range(1000):
    action, _states = model.predict(obs, deterministic=True)
    obs, reward, done, info = env.step(action)
    print(obs.shape)
    print(reward)
    if done:
      obs = env.reset()

env.close()

When I run this code, such error occured from stable-baselines3 env checker:

  assert isinstance(done, bool), "The `done` signal must be a boolean"
AssertionError: The `done` signal must be a boolean

This was due to the incompatibility bewteen np._bool and bool. In rlbench/gym/rlbench.env.py line107, terminate is a np._bool type, which makes isinstance(done, bool) False. To fix it, I simply typecasted and it works:

def step(self, action) -> Tuple[Dict[str, np.ndarray], float, bool, dict]: 
        obs, reward, terminate = self.task.step(action) 
        terminate = bool(terminate) 
        return self._extract_obs(obs), reward, terminate, {}