search

eugenevinitsky / sequential_social_dilemma_games

Repo for reproduction of sequential social dilemmas
MIT License
387 stars 132 forks source link

Rollout.py slows down after 150 steps #96

Open eugenevinitsky opened 5 years ago

eugenevinitsky commented 5 years ago

For reasons currently unclear to me, rollout.py is slowing down after 150 steps. If this is a general issue, this will make training the experiments take a good deal longer. This is worth profiling.

eugenevinitsky commented 5 years ago

Confirmed, the increase in time is due to the line: if save_path is not None: self.env.render(filename=save_path + 'frame' + str(i).zfill(6) + '.png') in rollout.py which takes longer and longer at each time-step. Therefore, this won't be an issue during training but should be resolved anyways.

eugenevinitsky commented 5 years ago

@natashamjaques do you have any clue why this might be happening? I have no idea at the moment.

natashamjaques commented 5 years ago

Yes, there are two rendering modes: 'fast' and 'pretty'. It's set to pretty by default -> this uses matplotlib to render each frame to an image on disk, and then stitches them together later. It's going to be necessary if we want fancy annotated videos with actions, rewards, (plots), etc later.

Fast rendering just saves the RGB matrices themselves and makes that into a video at the end. It doesn't look as good but it shouldn't slow things down.

I've tried to separate the code for 'rollout' from 'render_rollout'. I'm assuming during training we'll only use 'rollout', and not render it except every so many thousands of steps for evaluation purposes.

eugenevinitsky commented 5 years ago

Ah, so during training RLlib actually has its own internal rollout code that it uses. Also, I'm still unclear why 'pretty' mode would cause each rendering step to take longer over time e.g. the 100th step takes half as long as the 200th to render?

internetcoffeephone commented 4 years ago

Rollout.py no longer slows down in https://github.com/eugenevinitsky/sequential_social_dilemma_games/pull/179. I presume the reason this would happen previously was because of the following: when rendering with matplotlib, if you don't clear the plot inbetween rendering separate images, the time to render increases linearly per frame - which is an exponential increase overall.

I also added code that calculates/renders a rollout for all checkpoints in the ray_results directory.