Is it possible to guarantee to get a fixed number of video frames per tick using overclock?

[emailweixu]

Without this, using different overclock cannot give the equivalent video streams. It will be better if the video frames can be exactly aligned with each tick.

[DaveyBiggers]

Minecraft's render speed varies hugely depending on the machine's capabilities and the scene that is currently being rendered. (Frame rate drops substantially while things are exploding, for example.) While the render can be throttled to a maximum fps, it's impossible to enforce a minimum. Even without varying the overclocking, you're unlikely to get precisely equivalent video streams from run to run, due to the stochasticity in the system.

The best approach seems to be for Malmo to push out frames as fast as possible, and have the user match the frames and actions themselves. Some of the python samples contain example code for this - see robust_frames.py, for instance.

[emailweixu]

Is it possible to control the advance of each tick in Minecraft? This way can have precise control over the observations.

[emailweixu]

Deepmind's Lab environment allows the precise control over the number of frames using step() function (https://github.com/deepmind/lab/blob/master/docs/python_api.md). With continuous movement and complex environment, it's very difficult to do what robust_frames.py does. With manual control of the advancement of each frame, we don't need to worry about the timing of frames and observations.

I really like Minecraft as a learning environment because of its generality. But currently, its non-deterministic behavior makes me uneasy.

[tambetm]

I kinda agree, but on the other hand - in actual robotics you cannot "step the environment". You have to take into account that time is passing by and if your calculations take too long, you might crash in meantime. Or sometimes your sensors are laggy or noisy. Minecraft prepares you for that.

On Fri, Jan 6, 2017 at 3:04 PM, emailweixu notifications@github.com wrote:

Deepmind's Lab environment allows the precise control over the number of frames using step() function (https://github.com/deepmind/ lab/blob/master/docs/python_api.md). With continuous movement and complex environment, it's very difficult to do what robust_frames.py does. With manual control of the advancement of each frame, we don't need to worry about the timing of frames and observations.

I really like Minecraft as a learning environment because of its generality. But currently, its non-deterministic behavior makes me uneasy.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Microsoft/malmo/issues/437#issuecomment-271034308, or mute the thread https://github.com/notifications/unsubscribe-auth/ADo7grPE3fcz0tALvfyFZwQ22MY3N9ziks5rPsh2gaJpZM4LbOFv .

[emailweixu]

It's true for running in the real physical world or real game. But for simulation, we want to train as fast as possible, and possibly with multiple minecraft instances on one single machine. With current overclocking mechanism, it's really hard to control the frame rate.

[tambetm]

The way I do it: https://github.com/tambetm/gym-minecraft/blob/master/gym_minecraft/envs/minecraft_env.py#L281-L289

Basically I wait till at least one observation is available. If there isn't I wait for 1ms and try again. I run Minecraft with 5ms ticks and getting 150-160 fps. Why not 200 as supposed with 5ms ticks? Firstly if episode ends, resetting takes time and draws down the fps. But occasionally I also miss an observation. This hasn't prevented learning though (using recurrent policy).

Also when running with 5ms ticks you should also increase the maxFps in options.txt, as suggested here. If you set maxFps 200, then you nicely get one frame per observation, most of the time. In other times I just return the previous frame. Also setting video resolution to 40x30 helps to keep rendering costs low. Again, hasn't prevented from learning.

[DaveyBiggers]

Eliminating all non-determinism from Minecraft would be a significant engineering effort - if it's even possible at all - and I'm not sure how useful the resulting platform would be: adding code to enforce deterministic behaviour is only going to make everything slower, and Minecraft is already a heavy-weight environment, even when over-clocked.

I'm going to close this issue - it may be that we revisit the idea of adding a step() function at some point in the future, but for now you'll just have to cope with the non-determinism using the sort of workarounds that @tambetm suggests.