Is there any tricks to accelerate the running speed of the environment?

[alexxchen]

The running speed is slow for me. I don't know whether it is normal to cost 1.3 seconds to run per episode on single task, maybe it varies with the computational power of cpu, but the scale is still large. The agent inference time can be ignored compared to the environment step time. I use stable-baselines3 SubprocVecEnv to create one thread for each simulator. I think it's slow because I am using the ML10 as inner loop. I wonder what the official running time is.

I found the old version meta-world, based on gym rather than gymnasium (I don't sure if it is the cause), it cost only a third of the time for each episode.

Here is the evaluation code

from metaworld.envs import (ALL_V2_ENVIRONMENTS_GOAL_OBSERVABLE,
                            ALL_V2_ENVIRONMENTS_GOAL_HIDDEN)
import time

door_open_goal_observable_cls = ALL_V2_ENVIRONMENTS_GOAL_OBSERVABLE["door-open-v2-goal-observable"]

env = door_open_goal_observable_cls(seed=5)

env.reset()
start = time.time()
for _ in range(500):
    a = env.action_space.sample()
    next_obs2, _, _, _ = env.step(a)

print("run time", time.time() - start)

[reginald-mclean]

This example code takes 1.1 seconds to run on a server I have access to. Can you share more about your use case? I'm not quite sure if 'agent inference time can be ignored.' If you load a GPU with many jobs then agent inference time could be the bottleneck.

[Kallinteris-Andreas]

Just want to add for context the performance I am getting for the Humanoid environment is 1.7k steps/sec

>>> import gymnasium
>>> from gymnasium.utils.performance import benchmark_step
>>> env = gymnasium.make("Humanoid-v5")
>>> benchmark_step(env, target_duration=60, seed=456)
1747.5079450369717

you are getting 455 steps/sec which is reasonable considering the environment is more complex than the humanoid, and you may have a slower CPU

@alexxchen what CPU does your computer have?

Can you benchmark_step for "Humanoid-v5" and all metaworld environments and share the results with us?

What environment runtime performance expectation (steps per second) do you have?

[alexxchen]

Thanks for your reply @reginald-mclean! According to your result, the difference is within the expected range. I am meta-learning the hyperparameters, taking the reinforcement leaning as inner loop. To train the 10 environment for 500 steps in inner loop during each update of the outter loop, I find the bottleneck is the inner loop training speed (the inner loop agent is small enough). I'm just afraid of making a mistake, since the paper did not mention the wall time.

By the way, is it posssible to transplant meta-world to something like isaac gym?

@Kallinteris-Andreas Thanks for your information! The benchmark result is almost the same as you. I'm just being greedy. The faster the simulation runs, the better.

[Kallinteris-Andreas]

We are working on adding support for MJX But that is a while away.

[alexxchen]

We are working on adding support for MJX But that is a while away.

That sounds very exciting. Thank you for your work!