FPS varies enormously

[leo2r]

I created a custom Gym env and am using Stable Baselines to train agents on it. I've noticed that the FPS, for some unknown reason, varies greatly sometimes even though I am only changing some variables for how the reward or the value of the reward. It's usually around 25-30 FPS but then on some runs it's down to 3 FPS which makes it incredibly slow to run. Any idea what may happen? I looked at the documentation but couldn't find anything about FPS... thanks!

[Miffyli]

This depends on your environment, not SB. Some environments may slow down when the agent goes to specific areas of the environment. E.g. in video games with first-person perspective, the FPS may increase when agent looks up, as there is nothing to render and simulation speed improves.

SB has mostly stable affect on the FPS, apart from the arguments like learning_starts for DQN. Before this, agent only collects samples, which is fast. After that point, agent trains on every step, which slows down the whole process quite a bit.

(PS: we recommend moving to stable-baselines3 as it is more up-to-date)

[leo2r]

Thanks for the quick reply @Miffyli ! I meant to post this in stable-baselines3 oops 😅 Ah I see what you mean, thanks for the explanation. My env is a 200 by 200 array with values ranging from 0 to 6, is it correct to define the observation space as:

self.HEIGHT = 200
self.WIDTH = 200

# Define a 2-D observation space
self.observation_shape = (self.HEIGHT, self.WIDTH)
self.observation_space = spaces.Box(shape = self.observation_shape,
                                    low = np.zeros(self.observation_shape, dtype=int), 
                                    high = np.full(self.observation_shape, 6, dtype=int),
                                    dtype = int)

Thanks

EDIT: I realise this is a Gym question not Stable Baseline, so no worries if you can't answer it/you can delete this issue if it is inappropriate for this repo.

[Miffyli]

It is hard to say more just based on the observation spaces, I would have to know more about the environment :). However, if you are using A2C or PPO, then SB3 should not slow down considerably during training. If you are using TD3/SAC/DQN, check the learning_starts parameter.

I recommend studying how your agent behaves during training, and specifically how things change when FPS suddenly drops. I also recommend studying how fast your environment runs, specifically.

I am closing this one as "no tech support" (sorry, we do not have time to debug things, apart for bugs for the library itself). If you believe there is a bug in SB3, please open a new issue there.

[leo2r]

Thanks for the help @Miffyli ! 🙏🏽