duplicated and missing frames

[sitmo]

In the standard sf environment, I receive a list of 3 frames after each step.

When I plot the frames like below (each row shows the 3 frames I receive in that step) I see a pattern of 2 repeated duplicated frames. I was expecting three unique frames in each step. It looks like there is a bug in processing the frames? Maybe an indexing error?

Any idea how I can fix this?

[sitmo]

I'v gone through the code visually, and I can't see anything that might cause this.. but I know to little still.

• Maybe it has to do that the MAME frame rate and the game frame rate is different? That might cause frames to skip/repeat due to aliasing of rate differences?

Another thing related to this that puzzles me is :

• each environment step returns a list of 3 frames instead of a single frame.

In the datapipe code I see np.frombuffer(ptr).reshape(hight,width,channels) and that should return a single RBG frame, not a list of 3 frames? Or is it intended to return 3 frames in each step?

[M-J-Murray]

Sorry for the slow response. Something you have to bear in mind when using this framework is that you need to match up the framework with the games frame rate. There are 3 parameters to the environment which affect frame delivery: frame_ratio, frames_per_step, frame_skip. Frame skip is used internally within the emulator, it can increase the play speed of the game, as only every nth frame is returned. Frame ratio is similar, but internal to this library. So the frames will still be rendered to screen, but the libarary will only interact with MAME every nth frame. The reason for the distinction is to allow human viewing without skipping rendered frames. Frames per step is used to determine how many emulator steps should be taken before the next action is sent to the emulator. So if this is set to 3, then the emulator will collect 3 frames, and on the 3rd frame send your action to the game. The reason you might want to capture groups of frames at a time is so that you can show time progression to your reinforcement learning algorithm.

The problem you are seeing with the duplicated frames is kind of complicated. The game itself is running at 60 fps, however the gameplay seems to run at around 30 frames a second. And then on top of that the character animations are done at around 12 frames per second. So to get the optimum performance without losing gameplay information your want to set: frame_ratio=1, frames_per_step=2, frame_skip=5. This would return 12 frames per second, and capture two frames at a time before acting.