[Bug Report] PyGame MAgent VideoRecorder not working

[murtazarang]

Describe the bug gym.wrappers.RecordVideo not compatible with MAgent Pygame. This is also referenced and potentially fixed in the issue referenced here. The error is reproducible across both gym version, 0.22 and 0.23.

Code example

import time
import os
import pyvirtualdisplay

_display = pyvirtualdisplay.Display(visible=False, size=(1400, 900)) # use False with Xvfb
_ = _display.start()
os.environ["SDL_VIDEODRIVER"]="x11"

import pygame
pygame.init()
pygame.display.list_modes()
screen = pygame.display.set_mode((1400, 900))

import gym
import supersuit as ss

def main():
    num_envs = 1
    env = tiger_deer_v4.parallel_env(map_size=45, minimap_mode=False,
                                     tiger_step_recover=-0.1, deer_attacked=-0.1, max_cycles=50, extra_features=False)
    env = ss.pad_observations_v0(env)
    env = ss.pad_action_space_v0(env)
    env = ss.pettingzoo_env_to_vec_env_v1(env)
    env = ss.concat_vec_envs_v1(env, num_envs, num_cpus=4, base_class='gym')
    obs = env.reset(seed=1)
    env = gym.wrappers.RecordVideo(env, f"videos/{'check_test3'}")
    env.is_vector_env = True
    env.render(mode='human')

Traceback

Traceback (most recent call last):
  File "/mnt/g/My Drive/Research/Code/Github/latent-comm-traj/env_tests.py", line 72, in <module>
    main()

  File "/mnt/g/My Drive/Research/Code/Github/latent-comm-traj/env_tests.py", line 50, in main
    new_step = env.step(step_actions)

  File "/home/tago/anaconda3/envs/latent/lib/python3.7/site-packages/gym/wrappers/record_video.py", line 110, in step
    self.start_video_recorder()

  File "/home/tago/anaconda3/envs/latent/lib/python3.7/site-packages/gym/wrappers/record_video.py", line 75, in start_video_recorder
    self.video_recorder.capture_frame()

  File "/home/tago/anaconda3/envs/latent/lib/python3.7/site-packages/gym/wrappers/monitoring/video_recorder.py", line 157, in capture_frame
    frame = self.env.render(mode=render_mode)

  File "/home/tago/anaconda3/envs/latent/lib/python3.7/site-packages/stable_baselines3/common/vec_env/base_vec_env.py", line 281, in render
    return self.venv.render(mode=mode)

 File "/home/tago/anaconda3/envs/latent/lib/python3.7/site-packages/supersuit/vector/concat_vec_env.py", line 90, in render
    return self.vec_envs[0].render(mode)

  File "/home/tago/anaconda3/envs/latent/lib/python3.7/site-packages/supersuit/vector/markov_vector_wrapper.py", line 93, in render
    return self.par_env.render(mode)

  File "/home/tago/anaconda3/envs/latent/lib/python3.7/site-packages/pettingzoo/utils/conversions.py", line 162, in render
    return self.aec_env.render(mode)

  File "/home/tago/anaconda3/envs/latent/lib/python3.7/site-packages/pettingzoo/utils/wrappers/order_enforcing.py", line 63, in render
    return super().render(mode)

File "/home/tago/anaconda3/envs/latent/lib/python3.7/site-packages/pettingzoo/utils/wrappers/base.py", line 82, in render
    return self.env.render(mode)

  File "/home/tago/anaconda3/envs/latent/lib/python3.7/site-packages/pettingzoo/utils/wrappers/order_enforcing.py", line 63, in render
    return super().render(mode)

  File "/home/tago/anaconda3/envs/latent/lib/python3.7/site-packages/pettingzoo/utils/wrappers/base.py", line 82, in render
    return self.env.render(mode)

  File "/home/tago/anaconda3/envs/latent/lib/python3.7/site-packages/pettingzoo/utils/wrappers/order_enforcing.py", line 63, in render
    return super().render(mode)

  File "/home/tago/anaconda3/envs/latent/lib/python3.7/site-packages/pettingzoo/utils/wrappers/base.py", line 82, in render
    return self.env.render(mode)

  File "/home/tago/anaconda3/envs/latent/lib/python3.7/site-packages/pettingzoo/utils/conversions.py", line 299, in render
    return self.env.render(mode)

  File "/home/tago/anaconda3/envs/latent/lib/python3.7/site-packages/pettingzoo/magent/magent_env.py", line 144, in render
    ), "mode must be consistent across render calls"

AssertionError: mode must be consistent across render calls
free(): invalid pointer
Aborted (core dumped)

System Info OS: Windows WSL2 Ubuntu 20.04

Environment: Conda

Relevant Packages:

• pyvirtualdisplay 3.0

• supersuit 3.4.0

• pyglet 1.5.23

• pygame 2.1.0

• pettingzoo 1.18.1

• multi-agent-ale-py 0.1.11

• gym 0.23.1

• freetype 2.11.0

• python 3.7.11

• [x] I have checked that there is no similar issue in the repo (required)

[jjshoots]

Hi @murtazarang, we've just had a brief discussion among the devs, and gym.wrappers is actually not compatible with PZ. We also don't have any wrappers in SuperSuit for video recording.

However, should you be interested, you can do something like this

for agent in env.agent_iter():
    observation, reward, done, _ = env.last()
    action = (model.predict(observation, deterministic=True)[0] if not done else None)
    env.step(action)
    i += 1
    if i % (len(env.possible_agents) + 1) == 0:
        video_log.append(Image.fromarray(env.render(mode="rgb_array")))

total_reward = total_reward / n_agents

print("writing gif")
video_log[0].save(
   "./optimization_gifs/" + policy + "_" + str(total_reward)[:5] + ".gif",
    save_all=True,
    append_images=video_log[1:],
    optimize=False,
    duration=int(1000 / 15),
    loop=0,
)

[jjshoots]

Hi @murtazarang did that solve your problem? Can this issue be closed?

[jkterry1]

Hey, I'm closing this due to inactivity, please let us know if you need any additional help.

[Yong-Hsu]

Hi @murtazarang, we've just had a brief discussion among the devs, and gym.wrappers is actually not compatible with PZ. We also don't have any wrappers in SuperSuit for video recording.

However, should you be interested, you can do something like this

for agent in env.agent_iter():
    observation, reward, done, _ = env.last()
    action = (model.predict(observation, deterministic=True)[0] if not done else None)
    env.step(action)
    i += 1
    if i % (len(env.possible_agents) + 1) == 0:
        video_log.append(Image.fromarray(env.render(mode="rgb_array")))

total_reward = total_reward / n_agents

print("writing gif")
video_log[0].save(
   "./optimization_gifs/" + policy + "_" + str(total_reward)[:5] + ".gif",
    save_all=True,
    append_images=video_log[1:],
    optimize=False,
    duration=int(1000 / 15),
    loop=0,
)

The mode parameter in the render function of env becomes deprecated. Is there another way we can save the videos?

[elliottower]

Closing this as we don't have MAgent in this repo anymore nor is there a current video recorder. For those who want it for future reference, this script can be used to record videos but it is by no means the most efficient way--we are working on porting functionality from Gymnasium for video recording

import os

from moviepy.video.io.ImageSequenceClip import ImageSequenceClip
from moviepy.video.fx.all import resize
from PIL import Image as im

from pettingzoo.classic import tictactoe_v3

def save_video(
    frames: list,
    video_folder: str,
    i,
    name,
    fps=60,
):
    video_folder = os.path.abspath(video_folder)
    os.makedirs(video_folder, exist_ok=True)
    video_path = f"{video_folder}/{name}{i}.mp4"
    clip = ImageSequenceClip(frames, fps=fps)
    # making sure even dimensions
    width = clip.w if (clip.w % 2 == 0) else clip.w - 1
    height = clip.h if (clip.h % 2 == 0) else clip.h - 1
    clip = resize(clip, newsize=(width, height))
    clip.write_videofile(video_path)

def save_last_frame(frame, image_folder, i, name):
    image_folder = os.path.abspath(image_folder)
    os.makedirs(image_folder, exist_ok=True)
    image_path = f"{image_folder}/{name}{i}.png"
    # Saving as a PNG file
    img = im.fromarray(frame)
    img.save(image_path)

if __name__=="__main__":
    for i, env_module in enumerate([tictactoe_v3]): # gin_rummy_v4, leduc_holdem_v4, texas_holdem_no_limit_v6, texas_holdem_v4, tictactoe_v3, rps_v2
        render_frames = []
        env = env_module.env(render_mode="rgb_array")

        name = env.metadata.get("name")

        env.reset()

        render_frames.append(env.render())

        for agent in env.agent_iter():
            observation, reward, termination, truncation, info = env.last()
            if termination or truncation:
                action = None
            else:
                if "action_mask" in info:
                    mask = info["action_mask"]
                elif isinstance(observation, dict) and "action_mask" in observation:
                    mask = observation["action_mask"]
                else:
                    mask = None
                action = env.action_space(agent).sample(mask)
            env.step(action)
            render_frames.append(env.render())

        video_name = save_video(render_frames, "game_video", i, name, fps=1)
        last_frame_location = save_last_frame(render_frames[-1], "last_frame", i, name)

[Butanium]

If like me, you struggled using a video recorder from sb3 / gymnasium you may have forgotten to specify render_mode="rgb_array". MWE for sb3 video recorder:

from pettingzoo.atari import pong_v3
import supersuit as ss
from stable_baselines3.common.vec_env import VecVideoRecorder
def get_env():
    env = pong_v3.parallel_env(render_mode="rgb_array")
    env = ss.pettingzoo_env_to_vec_env_v1(env)
    envs = ss.concat_vec_envs_v1(
        env, 1 , num_cpus=0, base_class="stable_baselines3"
    )
    envs.render_mode = "rgb_array"
    envs = VecVideoRecorder(envs, f"videos/", capped_cubic_video_schedule)
    envs.single_observation_space = envs.observation_space
    envs.single_action_space = envs.action_space
    return envs

Adding envs.render_mode = "rgb_array" is mandatory otherwise the video recorder fails. In case you need it:

def capped_cubic_video_schedule(episode_id: int) -> bool:
    """The default episode trigger.

    This function will trigger recordings at the episode indices 0, 1, 8, 27, ..., :math:`k^3`, ..., 729, 1000, 2000, 3000, ...

    Args:
        episode_id: The episode number

    Returns:
        If to apply a video schedule number
    """
    if episode_id < 1000:
        return int(round(episode_id ** (1.0 / 3))) ** 3 == episode_id
    else:
        return episode_id % 1000 == 0

[elliottower]

Thanks for the heads up, good to see you can do it with SB3's video recorder