Too many end of file errors when reading from socket

[davidyu]

Why is this happening?

[davidyu]

Here's a typical message updating a player's position:

Cy24:gibber.components.PosCmpcy10:utils.Vec2y1:xd923.4748795138904y1:yd247.15199999999987gcR1R2d-9.881312916824931e-324R3d-9.881312916824931e-324gg

When node decides to write that message through to a TCP server, the OS beneath it decides to split that up across into multiple packets. So on the client reading from the socket, I have a message spanning, say 146 characters, but I only have the first chunk with about 72 characters for me to read. I read try to read 146 characters as indicated by the header, and I run into an EOF error.

What's worse, this throws off ALL future reads, because now when I get the next chunk of the data, I'm trying to parse it as I would a new message, and the opcode is gibberish! But by this time, because I don't have control over how TCP buffering works, I may already have a part of the next update message, so I read all of that too, and now I have to start over the cycle again with the next half of the message, which is again gibberish. This continues until I magically get to a point where I read to the end of a message. This rarely happens, so I'm just stuck in a read limbo.

Solving the problem

• I can use shorter messages and hope the OS never chops up my messages. I can round the position vector to the nearest int whenever possible or force rounding before serializing (unpredictable and scary).
• I can make a stateful client parser. When it doesn't have enough to parse, it should stop and resume on the next update iteration (692fa1a EDIT: bugfixes in 2af02b4).
• I might try waiting longer periods between subsequent reads. This isn't guaranteed to solve the problem.
• I need to verify that it IS the OS splitting up my data, and not just node doing something dumb to optimize performance It is something underneath node.
• I can't use datagrams because Flash is stupid

Using telnet for diagnosis, it looks as if I'm getting the entire message at once (they're not sent in chunks), so is this an AS3 socket implementation problem?

[davidyu]

Read/write expectation robustness

Even with a stateful parser, something is wrong. We should ALWAYS be able to resume from a paused read, no matter what. We should never get empty strings or strings prepended with a character we're not expecting (which is what's happening, we're getting a mysterious "?" character popping in every once in a while and sometimes we just get a blank string from readUTFBytes()). The causes to these issues must be found and fixed before we move on to other orthogonal solutions.

For example, we should be able to start over from a bad state by clearing all socket read buffers.

UPDATE The issue recurs with position updates with length of exactly 194. I wonder what the combination of byte 2, short 194 is and why this seems to append the following string with a '?' character.

UPDATE 2 Someone is lying! The actual length of the string is ~140, but we are reading an extra byte (or something) when we read the unsigned short (or UInt16) so we are getting 194! Why...

See 4966f72 for repro of this issue. I think this is a bug in AS3 or Haxe, but it might be a gotcha of UTF-8 I don't yet understand.

Observe in fbb09e8 that it is generally a bad idea to transform a ByteArray into a string and expect it to round-trip with no artifacts. Chances are, the UTF-8 implementation will ignore certain bits that comprise a character and replace them with arbitrary ones when you transform it back into bytes. Likewise with ByteArray.toString. God knows what sorts of design decisions Adobe made with it. To do this safely, just sample from the ByteArray directly by using Socket.writeBytes( ByteArray, ... ).

This issue is fixed in a3e45a1. What an adventure.

[davidyu]

We can probably remove the pause-resume functionality, seeing as we only needed it because of a bug that's now been fixed.