Update to node-minecraft-protocol 0.16 (major changes including using protodef)

deathcap commented 8 years ago

https://github.com/deathcap/wsmc/issues/19

deathcap commented 8 years ago

Current status, makes some progress:

wsmc $ npm start

> wsmc@0.0.1 start /Users/admin/games/voxeljs/wsmc
> NODE_DEBUG=mc-proto node wsmc.js

WS(0.0.0.0:24444) <--> MC(localhost:25565)
websocket received 14 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   6D 63 77 65 62 63 68 61 74 75 73 65 72 58         mcwebchatuserX
WS requested username: mcwebchatuserX
MC-PROTO: 19012 writing packet handshaking.set_protocol
MC-PROTO: 19012 { protocolVersion: 47,
  serverHost: 'localhost',
  serverPort: 25565,
  nextState: 2 }
MC-PROTO: 19012 writing packet login.login_start
MC-PROTO: 19012 { username: 'webuser-1' }
Successfully connected to MC
compress { threshold: 256 }
mc received 3 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   03 80 02                                          ...
lengthField= <Buffer 03>
writing to ws 4 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   03 03 80 02                                       ....
mc received 48 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   02 24 39 61 38 34 63 31 37 30 2D 30 65 36 62 2D   .$9a84c170-0e6b-
000010   33 38 62 61 2D 62 31 61 65 2D 64 32 35 38 64 37   38ba-b1ae-d258d7
000020   64 65 33 30 31 63 09 77 65 62 75 73 65 72 2D 31   de301c.webuser-1
lengthField= <Buffer 31>
writing to ws 50 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   31 00 02 24 39 61 38 34 63 31 37 30 2D 30 65 36   1..$9a84c170-0e6
000010   62 2D 33 38 62 61 2D 62 31 61 65 2D 64 32 35 38   b-38ba-b1ae-d258
000020   64 37 64 65 33 30 31 63 09 77 65 62 75 73 65 72   d7de301c.webuser
000030   2D 31                                             -1
mc received 18 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   01 00 00 00 00 00 00 02 14 07 64 65 66 61 75 6C   ..........defaul
000010   74 00                                             t.
lengthField= <Buffer 13>
writing to ws 20 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   13 00 01 00 00 00 00 00 00 02 14 07 64 65 66 61   ............defa
000010   75 6C 74 00                                       ult.
mc received 10 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   39 00 3D 4C CC CD 3D CC CC CD                     9.=LÌÍ=ÌÌÍ
lengthField= <Buffer 0b>
writing to ws 12 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   0B 00 39 00 3D 4C CC CD 3D CC CC CD               ..9.=LÌÍ=ÌÌÍ
mc received 26 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   3F 08 4D 43 7C 42 72 61 6E 64 0B 47 6C 6F 77 73   ?.MC|Brand.Glows
000010   74 6F 6E 65 2B 2B 00 00 00 00                     tone++....
lengthField= <Buffer 1b>
writing to ws 28 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   1B 00 3F 08 4D 43 7C 42 72 61 6E 64 0B 47 6C 6F   ..?.MC|Brand.Glo
000010   77 73 74 6F 6E 65 2B 2B 00 00 00 00               wstone++....
mc received 34 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   08 C0 65 60 00 00 00 00 00 40 53 80 00 00 00 00   .Àe`.....@S.....
000010   00 C0 69 20 00 00 00 00 00 00 00 00 00 00 00 00   .Ài ............
000020   00 00                                             ..
lengthField= <Buffer 23>
writing to ws 36 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   23 00 08 C0 65 60 00 00 00 00 00 40 53 80 00 00   #..Àe`.....@S...
000010   00 00 00 C0 69 20 00 00 00 00 00 00 00 00 00 00   ...Ài ..........
000020   00 00 00 00                                       ....
mc received 8 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   12 00 00 00 00 00 00 00                           ........
lengthField= <Buffer 09>
writing to ws 10 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   09 00 12 00 00 00 00 00 00 00                     ..........
mc received 10 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   06 41 A0 00 00 14 00 00 00 00                     .A .......
lengthField= <Buffer 0b>
writing to ws 12 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   0B 00 06 41 A0 00 00 14 00 00 00 00               ...A .......
mc received 10 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   39 00 3D 4C CC CD 3D CC CC CD                     9.=LÌÍ=ÌÌÍ
lengthField= <Buffer 0b>
writing to ws 12 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   0B 00 39 00 3D 4C CC CD 3D CC CC CD               ..9.=LÌÍ=ÌÌÍ
mc received 2 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   09 00                                             ..
lengthField= <Buffer 03>
writing to ws 4 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   03 00 09 00                                       ....
mc received 7 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   1F 00 00 00 00 00 00                              .......
lengthField= <Buffer 08>
writing to ws 9 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   08 00 1F 00 00 00 00 00 00                        .........
mc received 7 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   1F 00 00 00 00 00 00                              .......
lengthField= <Buffer 08>
writing to ws 9 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   08 00 1F 00 00 00 00 00 00                        .........
mc received 7 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   1F 00 00 00 00 00 00                              .......
lengthField= <Buffer 08>
writing to ws 9 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   08 00 1F 00 00 00 00 00 00                        .........
mc received 10 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   06 41 A0 00 00 14 00 00 00 00                     .A .......
lengthField= <Buffer 0b>
writing to ws 12 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   0B 00 06 41 A0 00 00 14 00 00 00 00               ...A .......
mc received 10 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   06 41 A0 00 00 14 00 00 00 00                     .A .......
lengthField= <Buffer 0b>
writing to ws 12 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   0B 00 06 41 A0 00 00 14 00 00 00 00               ...A .......
mc received 10 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   39 00 3D 4C CC CD 3D CC CC CD                     9.=LÌÍ=ÌÌÍ
lengthField= <Buffer 0b>
writing to ws 12 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   0B 00 39 00 3D 4C CC CD 3D CC CC CD               ..9.=LÌÍ=ÌÌÍ
mc received 10 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   39 00 3D 4C CC CD 3D CC CC CD                     9.=LÌÍ=ÌÌÍ
lengthField= <Buffer 0b>
writing to ws 12 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   0B 00 39 00 3D 4C CC CD 3D CC CC CD               ..9.=LÌÍ=ÌÌÍ
mc received 10 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   39 00 3D 4C CC CD 3D CC CC CD                     9.=LÌÍ=ÌÌÍ
lengthField= <Buffer 0b>
writing to ws 12 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   0B 00 39 00 3D 4C CC CD 3D CC CC CD               ..9.=LÌÍ=ÌÌÍ
mc received 10 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   39 00 3D 4C CC CD 3D CC CC CD                     9.=LÌÍ=ÌÌÍ
lengthField= <Buffer 0b>
writing to ws 12 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   0B 00 39 00 3D 4C CC CD 3D CC CC CD               ..9.=LÌÍ=ÌÌÍ
websocket received 1 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   04                                                .
Failed to parse varint uncompressed length in raw buffer from ws: <Buffer 04> offset 1
websocket received 4 bytes
Offset   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

000000   00 09 00 00                                       ....

Looks like the last packet is fragmented (varint length 04, followed by 4 byte payload: 00 09 00 00), expect to receive it in whole in one ws frame.

deathcap commented 8 years ago

Debugging on the client (mcwebchat example), for now editing node-minecraft-protocol/src/debug.js to always log (then npm run-script prepublish to recompile), until can discover how to best set process.env.NODE_DEBUG to mc-proto within the browserify environment (should be possible somehow; browserify does automatically scan for globals including process).

After sending the username, the first MC packet sent over the websocket is for the held_item_slot, with params { slotId: 0}. Serialized in protodef/src/serializer.js _transform, uses a node transform stream in object mode, serializing to 09 00 00:

> chunk
Object {name: "held_item_slot", params: Object}name: "held_item_slot"params: ObjectslotId: 0__proto__: Object__proto__: Object
> this.createPacketBuffer(chunk)
[9, 0, 0]

Length is not added at this point yet. Has node-minecraft-protocol changed to write the packet length and packet data in two write calls?

deathcap commented 8 years ago

Maybe this? https://github.com/PrismarineJS/node-minecraft-protocol/blob/master/src/transforms/framing.js#L17-L24

  _transform(chunk, enc, cb) {
    var buffer = new Buffer(sizeOfVarInt(chunk.length));
    writeVarInt(chunk.length, buffer, 0);
    this.push(buffer);
    this.push(chunk);
    return cb();
  }
}

buffer is the encoded varint length [4], chunk is the data [0, 9, 0, 0]. This matches the two separate websocket writes I am seeing on the websocket server side in the wsmc proxy:

websocket received 1 bytes

000000   04                                                .
Failed to parse varint uncompressed length in raw buffer from ws: <Buffer 04> offset 1
websocket received 4 bytes

000000   00 09 00 00                                       ....

How to ensure the ws client only sends full mc packets in each ws frame?

deathcap commented 8 years ago

https://nodejs.org/api/stream.html#stream_writable_cork "Forces buffering of all writes." - I would think this would work to do only one write:

  _transform(chunk, enc, cb) {
    var buffer = new Buffer(sizeOfVarInt(chunk.length));
    writeVarInt(chunk.length, buffer, 0);
    this.cork();
    this.push(buffer);
    this.push(chunk);
    this.uncork();
    return cb();
  }

but it seems to have no effect (still sends two writes) at least in browserify - maybe a browser module limitation or something? Manually combining the two buffers and calling this.push() once works as expected:

  _transform(chunk, enc, cb) {
    var buffer = new Buffer(sizeOfVarInt(chunk.length) + chunk.length);
    writeVarInt(chunk.length, buffer, 0);
    chunk.copy(buffer, sizeOfVarInt(chunk.length));
    this.push(buffer);
    return cb();
  }

With this change in node-minecraft-protocol/src/transforms/framing.js, browserify mineflayer is back again! mcwebchat demo works at least, able to send/receive chat messages to/from mc server.

Although, copying the buffer into another buffer is undesirable, maybe there is a better way to fix this (try cork/uncork again? have wsmc buffer the packets itself?!).

deathcap commented 8 years ago

Perhaps better, if possible: disable MC packet framing on the WS<->MC stream. Websockets have their own framing: http://lucumr.pocoo.org/2012/9/24/websockets-101/#framing - so MC's framing is redundant in this context. Ensure each WS frame is one MC packet.

update: disabled framing with commits below, works well

deathcap commented 8 years ago

deathcap / wsmc

Update to node-minecraft-protocol 0.16 (major changes including using protodef) #21