[Question] TypeError: 'NoneType' object is not callable in Rendering Fetch Environments

[aalmuzairee]

Hi,

I'm getting this error at the end of rendering the Fetch environments https://github.com/Farama-Foundation/Gymnasium-Robotics in "human" mode:

Exception ignored in: <function WindowViewer.del at 0x7fb4ea7d2680> Traceback (most recent call last): File "miniconda/envs/gym/lib/python3.7/site-packages/gymnasium/envs/mujoco/mujoco_rendering.py", line 335, in del File "miniconda/envs/gym/lib/python3.7/site-packages/gymnasium/envs/mujoco/mujoco_rendering.py", line 328, in free TypeError: 'NoneType' object is not callable

I'm able to render the entire scene but when the program closes I get this error. Would appreciate more insight into whether it is a glfw based issue, mujoco environment based issue, or another package, thanks!

[rodrigodelazcano]

Can you share the code you are using to replicate the issue?

Also, we are working in a similar issue related with the mujoco renderer implemented in Gymnasium https://github.com/Farama-Foundation/Gymnasium/issues/325

[aalmuzairee]

Sure, here is the code snippet:

import gymnasium as gym

env = gym.make('FetchReach-v2', render_mode="human")
observation, info = env.reset(seed=42)

for _ in range(100):
   action = env.action_space.sample()  
   observation, reward, terminated, truncated, info = env.step(action)
   if terminated or truncated:
      observation, info = env.reset()

env.close()

Working on: Ubuntu 18.04 mujoco 2.3.2 gymnasium 0.28.1 gymnasium-robotics 1.2.1 python 3.7.16

[aalmuzairee]

This seems to have been fixed in https://github.com/Farama-Foundation/Gymnasium/issues/438 , and has been merged into the main gymnasium repo. Any idea it will be updated in a new release @rodrigodelazcano?

[Kallinteris-Andreas]

@aalmuzairee can you try using the main git version of Gymansium.

[rodrigodelazcano]

@aalmuzairee yes it will be updated once Gymnasium makes the next release. As @Kallinteris-Andreas mentioned if you can check that it works with the source code of Gymasaium that would be great

[aalmuzairee]

Seems to be working for me, thank you!

[aalmuzairee]

Hi @rodrigodelazcano @Kallinteris-Andreas,

It seems that it wasn't working as ideally as I expected.

In this code:

import gymnasium as gym

# env = gym.make('CartPole-v1', render_mode="rgb_array")
env = gym.make('FetchReach-v2', render_mode="rgb_array")
observation, info = env.reset(seed=42)

for i in range(4):
   action = env.action_space.sample()  # this is where you would insert your policy
   observation, reward, terminated, truncated, info = env.step(action)
   if terminated or truncated:
      observation, info = env.reset()

   # Render the rgb_arrays
   if i % 1 == 0:
      print("Frame", i , ":", env.render()[0,0])
      env.close() 
env.close()

When I render the frames, this is the output for the

CartPole-v1:

Frame 0 : [255 255 255]
Frame 1 : [255 255 255]
Frame 2 : [255 255 255]
Frame 3 : [255 255 255]

any Fetch Environment (FetchReach-v2 here):

Frame 0 : [114 218 145]
Frame 1 : [0 0 0]
Frame 2 : [0 0 0]
Frame 3 : [0 0 0]

When rendering the CartPole environment, the frames are normal even after calling "env.close()" after rendering. However, for the fetch environments, this code causes every frame after the first one to be completely black (array of zeros). This is caused by the close() function in each environment. The CartPole uses PyGame to close the viewer, while the Fetch environments use the mujoco_rendering.py file to close. Any ideas on how to make the behaviour similar to the CartPole environment? Appreciate your help.

[Kallinteris-Andreas]

You are using close incorrectly, you are supposed to close at the very end not in every step

[aalmuzairee]

I see, but the problem is it is used in this way in https://github.com/Farama-Foundation/Gymnasium/blob/main/gymnasium/wrappers/monitoring/video_recorder.py#L148.

Every video recording wrapper ( From Gymnasium or SB3 ), calls the "close()" function from video_recorder.py to save the video, which also closes the running env. This causes trouble when a user is trying to record multiple videos from one agent.

[Kallinteris-Andreas]

The renderer is supposed to be reopened after each env.reset()

[aalmuzairee]

Are you able to get more than one video from the same agent using any of the video wrappers above on the Fetch Environment?

[Kallinteris-Andreas]

What do mean "from the same agent" the policy should be irrelevant here, show us how you use the video_recorder

[aalmuzairee]

Sure thing, here is an example using SB3:

(make sure to install SB3 > 2.0 so that it supports gymnasium, you can do that with: pip install "stable_baselines3>=2.0.0a1" --upgrade)

from stable_baselines3 import PPO
from stable_baselines3.common.monitor import Monitor
from stable_baselines3.common.vec_env import DummyVecEnv, VecVideoRecorder
import gymnasium as gym

env_id = "FetchReach-v2"

def make_env():
    env = gym.make(env_id, render_mode="rgb_array")
    env = Monitor(env)
    observation, info = env.reset(seed=42)
    return env

env = DummyVecEnv([make_env])
env = VecVideoRecorder(env, f"videos", record_video_trigger=lambda x: x % 500 == 0, video_length=50)

model = PPO("MultiInputPolicy", env, verbose=1)

model.learn(total_timesteps=2000)

You should now have four videos in your "videos" folder, only the first one of them is a normal video, the others are all black. Please let me know if you have a different results, thanks. (video_recorder is being called in VecVideoRecorder)

[Kallinteris-Andreas]

Does the same problem persist with env_id='Hopper-v4'

[aalmuzairee]

I'm not able to run Hopper-v4 because of a mujoco-xml-versioning error, however I'm able to run 'Ant-v4' and it works normally, all videos are good. Hope this helps.

[Kallinteris-Andreas]

well then this is unrelated to the original issue, create a new isuee i am not familiar with how franka envs render ping rodrigo

[aalmuzairee]

You're right, this is a bit different from the original issue. I found a hacky fix for it for now, appreciate your help.

[rodrigodelazcano]

Hey @aalmuzairee was your issue solved?

[aalmuzairee]

Hi @rodrigodelazcano, yes thank you.