Regarding the visualization error issue and the parallelization problem in Maniskill3.

[yeshenpy]

Amazing work! However, I encountered some problems while using it.

The first problem is, if I open more than one environment, i.e. num_env > 1, is it unreasonable for visualization? The code below can reproduce this issue; the robotic arm does not move in the visualization window, only the gripper moves.

import gymnasium as gym
import mani_skill.envs
envs = gym.make(
    "PickCube-v1", # there are more tasks e.g. "PushCube-v1", "PegInsertionSide-v1", ...
    num_envs=2,
    obs_mode="state", # there is also "state_dict", "rgbd", ...
    control_mode="pd_ee_delta_pose", # there is also "pd_joint_delta_pos", ...
    render_mode="human")
print("Observation space", envs.observation_space)
print("Action space", envs.action_space)
obs, _ = envs.reset(seed=0)
for i in range(100):
    action = envs.action_space.sample()
    obs, reward, terminated, truncated, info = envs.step(action)
    envs.render()

The second problem is that Maniskill3 directly supports multi-environment parallelization, but how does it handle resets in this context? If I open multiple environments and one of them ends (succeeds early), will it reset immediately, or will it continue executing actions after success? Both approaches can ensure that the lengths of the collected trajectories are equal. Is there still a need to use gym.vector.AsyncVectorEnv for re-encapsulation? Since I am training RL, the details of parallelization are very important.

Thanks for your valuable time.

[StoneT2000]

What do you mean if only the robot arm moves, not the gripper? When I run the code I see the whole robot moving. Note that if yhou have parallel environments there isn't a recommended way to visualize it other than to record videos using the RecordEpisode wrapper (which will record videos in parallel, e.g. if you run the baseline PPO code the evaluation videos are always multiple videos together).

For environment parallelization you should not use any wrappers/async vector envs from gymnasium or other libraries if you want to use GPU simulation. (We still support asyncvectorenv for CPU simulation). To use GPU simulation setting num_envs > 1 will automatically turn it on when using gym.make. To make it look like AsyncVectorEnv, you should use the ManiSkillVectorEnv wrapper which wraps the env created by gym.make into a object that is an instance of gymnasium VectorEnv.

Relevant imports:

from mani_skill.utils.wrappers.record import RecordEpisode
from mani_skill.vector.wrappers.gymnasium import ManiSkillVectorEnv

I highly recommend looking at the PPO code https://github.com/haosulab/ManiSkill/blob/main/examples/baselines/ppo/ppo.py to see how it is used for RL workflows. I will be adding documentation about how to do certain types of ML workflows with ManiSkill soon.

[StoneT2000]

Oh and regarding environment reset, by default the environment created by gym.make does not come with auto reset, so it only resets if you call env.reset. If you wrap ManiSkillVectorEnv around the environment, then you get auto reset (you can also turn it off). There is also an argument for ignoring terminations (terminated = True if environment is success or fail state). If you ignore terminations then resets always happen whenever the environment is truncated (e.g. reaching the max episode steps). If you don't ignore terminations then there is partial resets, where at each step not all environments reset at the same time (which can sometimes speed up RL training).

See https://github.com/haosulab/ManiSkill/blob/main/mani_skill/vector/wrappers/gymnasium.py for the code.

[StoneT2000]

Hi @yeshenpy is this still an issue?