Speeding up MalmoEnv rendering speed

[darius-lam]

The MalmoEnv gym environment is currently bottlenecked by the socket communication between the python client and java server. Before optimizations I was getting ~10 fps (even using 5 MS/tick and PrioritiseOffscreenRendering)

By adding

socket.setTcpNoDelay(true);

to the java serverSocket and

sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)

to the python client socket (in malmoenv/core.py)

I was able to get ~130fps (using the same settings).

Discussion on NoDelay

I'm not sure how MineRL and MinecraftGym do it, but MalmoEnv gets data from the Minecraft Client via socket communication. In particular, MalmoEnvServer.java is responsible for sending data to the Python Env

        dout.writeInt(obs.length);
        dout.write(obs);

        dout.writeInt(BYTES_DOUBLE + 2);
        dout.writeDouble(reward);
        dout.writeByte(done ? 1 : 0);
        dout.writeByte(sent ? 1 : 0);

As can be seen from the code above, at every step, Java has to make 6 write calls to the socket. This can cause latency issues because of the DelayedAck-Nagle'sAlgorithm interaction. I ran some latency tests and a lot of time is spent clientside waiting for data from the server. To resolve this, we can use the workaround I described above, by setting TCP_NODELAY on the clientside and more importantly serverside, so that the socket sends data as soon as it is written to the TCP write buffer.

I haven't tested the nodelay flag yet for unseen effects so I'm including this blurb here in case it comes in handy for debugging later.

[balloch]

This should be a pull request. 130 fps would be a game changer

[balloch]

socket.setTcpNoDelay(true);

In what file is this? I don't see any socket object in the Malmo subdirectory under Minecraft

[balloch]

also, can this be implemented with the prebuilt libraries? I assume not...