Broken with minecraft-protocol 0.16, missing protocol property

[deathcap]

Using the latest releases minecraft-protocol 0.16.5 and mineflayer 1.5.3, wsmc fails with:

node wsmc.js 
wsmc/wsmc.js:4
var readVarInt = minecraft_protocol.protocol.types.varint[0];
                                            ^

TypeError: Cannot read property 'types' of undefined
    at Object.<anonymous> (wsmc/wsmc.js:4:45)
    at Module._compile (module.js:398:26)
    at Object.Module._extensions..js (module.js:405:10)
    at Module.load (module.js:344:32)
    at Function.Module._load (module.js:301:12)
    at Function.Module.runMain (module.js:430:10)
    at startup (node.js:141:18)
    at node.js:980:3

[deathcap]

Works with minecraft-protocol@0.13.4, fails with minecraft-protocol@0.14.0

bisected to: https://github.com/PrismarineJS/node-minecraft-protocol/commit/1a9e08cbd8597d7359d5a0b9fd37e312dccb75b7 move createPacketBuffer and parsePacketData functions to serializer, also move protocol's exports to serializer

Looks like minecraft_protocol.protocol. changed to just minecraft_protocol. Works up to 0.15.0, but then in the next release 0.16.0, changed again:

wsmc/wsmc.js:4
var readVarInt = minecraft_protocol.types.varint[0];
                                         ^

TypeError: Cannot read property 'varint' of undefined
    at Object.<anonymous> (wsmc/wsmc.js:4:42)
    at Module._compile (module.js:398:26)
    at Object.Module._extensions..js (module.js:405:10)
    at Module.load (module.js:344:32)
    at Function.Module._load (module.js:301:12)
    at Function.Module.runMain (module.js:430:10)
    at startup (node.js:141:18)
    at node.js:980:3

[deathcap]

Began to update to 0.15.0 on https://github.com/deathcap/wsmc/compare/nmp0.15.0 - but node examples/mcwebchat/mcwebchat.js fails with this mysterious error:

wsmc/node_modules/prismarine-block/index.js:5
  var mcData=require('minecraft-data')(mcVersion);
                                      ^

TypeError: require(...) is not a function
    at loader (wsmc/node_modules/prismarine-block/index.js:5:39)
    at Object.<anonymous> (wsmc/node_modules/mineflayer/lib/plugins/blocks.js:5:40)
    at Module._compile (module.js:398:26)
    at Object.Module._extensions..js (module.js:405:10)
    at Module.load (module.js:344:32)
    at Function.Module._load (module.js:301:12)
    at Module.require (module.js:354:17)
    at require (internal/module.js:12:17)
    at Object.<anonymous> (wsmc/mineflayer-stream.js:11:15)
    at Module._compile (module.js:398:26)

Probably ought to just update straight to minecraft-protocol 0.16.0. But what version specifier should I use for API compatibility with this module? ^0.15.0 would pickup ^0.16.0. since "^" only locks the major version, accepts changes minor and patch - it may be acceptable to use ~0.15.0, to lock major and minor, only allowing updates from the patch version number. Or stricter, '0.15.0' (or 0.16.5, etc.) to lock to an exact version.

[deathcap]

Caused by ancient minecraft-data (0.0.1), updated to ~0.16.3, fixed above 'is not a function' error with minecraft-data but now hitting an incompatibility with mineflayer (1.5.3):

$ node examples/mcwebchat/mcwebchat.js 
You are using a pure-javascript implementation of RSA.
Your performance might be subpar. Please consider installing URSA
module.js:328
    throw err;
    ^

Error: Cannot find module 'mineflayer/lib/block'
    at Function.Module._resolveFilename (module.js:326:15)
    at Function.Module._load (module.js:277:25)
    at Module.require (module.js:354:17)
    at require (internal/module.js:12:17)
    at Object.<anonymous> (wsmc/mineflayer-stream.js:41:10)
    at Module._compile (module.js:398:26)
    at Object.Module._extensions..js (module.js:405:10)
    at Module.load (module.js:344:32)
    at Function.Module._load (module.js:301:12)
    at Module.require (module.js:354:17)

Block and Biome moved to prismarine-block and prismarine-biome in https://github.com/PrismarineJS/mineflayer/commit/ca365c3c1bbdb39cf6632b71e7f3d6363d36c2ae

---

update: Block and Biome are still accessible via the mineflayer exports; fixed in d94181283b588b127269d1aed90d5d40d9964915

mcwebchat now errors out in the browser:

Error: Cannot find module './minecraft-data/1.8/enums/blocks'
s_prelude.js:1
(anonymous function)_prelude.js:1
mcVersionToMcDataindex.js:30
exportsindex.js:9
(anonymous function)version.js:3
s_prelude.js:1
(anonymous function)_prelude.js:1
(anonymous function)browser.js:2

dynamic require in ./node_modules/minecraft-protocol/node_modules/minecraft-data/index.js:

        function mcVersionToMcData(mcVersion)
        {
            var dir = "./minecraft-data/" + mcVersion;
            return {
                blocks: require(dir + '/enums/blocks'),

static requires were added in https://github.com/PrismarineJS/node-minecraft-data/commit/eb92ebd02cc2a02699d940efa0fb13551cd6fb92 as of minecraft-data 0.15.0 (right after 0.14.0). minecraft-protocol 0.15.0 uses minecraft-data 0.11.0, so probably have to update straight to minecraft-protocol 0.16.

[deathcap]

Starting update to minecraft-protocol 0.16.6, failure on nmp0.15.0 branch is:

wsmc/wsmc.js:4
var readVarInt = minecraft_protocol.types.varint[0];
                                         ^

TypeError: Cannot read property 'varint' of undefined
    at Object.<anonymous> (wsmc/wsmc.js:4:42)
    at Module._compile (module.js:435:26)
    at Object.Module._extensions..js (module.js:442:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:313:12)
    at Function.Module.runMain (module.js:467:10)
    at startup (node.js:136:18)
    at node.js:963:3

major change is move to use protodef, relevant commit: https://github.com/PrismarineJS/node-minecraft-protocol/commit/f45c6dff49cbc12903ac2c1b9d408e23d029c290

---

packet IDs also moved:

wsmc.js:24
var ids = minecraft_protocol.packetIds.play.toClient;
                                      ^

TypeError: Cannot read property 'play' of undefined
    at Object.<anonymous> (wsmc/wsmc.js:24:39)
    at Module._compile (module.js:435:26)
    at Object.Module._extensions..js (module.js:442:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:313:12)
    at Function.Module.runMain (module.js:467:10)
    at startup (node.js:136:18)
    at node.js:963:3

in https://github.com/PrismarineJS/node-minecraft-protocol/commit/1a9e08cbd8597d7359d5a0b9fd37e312dccb75b7#diff-1fdf421c05c1140f6d71444ea2b27638L4

then to minecraft-data: https://github.com/PrismarineJS/node-minecraft-protocol/commit/6b6303b853457636a8bf9532e5fa57a820770cdc#diff-1fdf421c05c1140f6d71444ea2b27638L7

[deathcap]

wsmc/JavaScript server now runs, but the mcwebchat example (browserified) fails in a dynamic require:

        function mcVersionToMcData(mcVersion) // mcVersion = '1.8'
        {
            var dir = "./minecraft-data/" + mcVersion;
            return {
                blocks: require(dir + '/enums/blocks'),

since browserify compiles requires statically, unable to resolve at runtime:

require('./minecraft-data/1.8/enums/blocks') Error: Cannot find module './minecraft-data/1.8/enums/blocks'

fixed in node-minecraft-data 0.15.0: https://github.com/PrismarineJS/node-minecraft-data/commit/eb92ebd02cc2a02699d940efa0fb13551cd6fb92 - but mineflayer was using ~0.5.0. PR to mineflayer to update: https://github.com/PrismarineJS/mineflayer/pull/354

---

gets further, until mineflayer/index.js using requireindex:

var requireIndex = require('requireindex');
var plugins = requireIndex(path.join(__dirname, 'lib', 'plugins'));

requireindex calls FS.readdirSync - not defined in browserify: (I thought it used to be? can be static)

                var files = FS.readdirSync(dir);

update: https://www.npmjs.com/package/bulkify browserify transform

---

brfs transform supports fs.readdirSync: https://github.com/substack/brfs/issues/19 (PR merged)

[deathcap]

Added brfs transform but now running into https://github.com/rom1504/node-mojangson browserify incompatibility:

 mcwebchat@0.0.1 start wsmc/examples/mcwebchat
> wzrd mcwebchat.js:bundle.js -- --global-transform brfs

server started at http://localhost:9966
{"url":"/bundle.js","type":"bundle","command":"browserify mcwebchat.js --global-transform brfs","elapsed":"1609ms","time":"2016-01-09T19:59:38.656Z"}
TypeError: callee.apply is not a function while parsing file: mineflayer/node_modules/node-mojangson/grammar.js
    at walk (wsmc/examples/mcwebchat/node_modules/brfs/node_modules/static-module/node_modules/static-eval/index.js:89:27)
    at walk (wsmc/examples/mcwebchat/node_modules/brfs/node_modules/static-module/node_modules/static-eval/index.js:92:23)
    at walk (wsmc/examples/mcwebchat/node_modules/brfs/node_modules/static-module/node_modules/static-eval/index.js:77:26)
    at walk (wsmc/examples/mcwebchat/node_modules/brfs/node_modules/static-module/node_modules/static-eval/index.js:85:25)
    at module.exports (wsmc/examples/mcwebchat/node_modules/brfs/node_modules/static-module/node_modules/static-eval/index.js:114:7)
    at traverse (wsmc/examples/mcwebchat/node_modules/brfs/node_modules/static-module/index.js:256:23)
    at walk (wsmc/examples/mcwebchat/node_modules/brfs/node_modules/static-module/index.js:208:13)
    at walk (wsmc/examples/mcwebchat/node_modules/brfs/node_modules/static-module/node_modules/falafel/index.js:49:9)
    at wsmc/examples/mcwebchat/node_modules/brfs/node_modules/static-module/node_modules/falafel/index.js:46:17
    at forEach (wsmc/examples/mcwebchat/node_modules/brfs/node_modules/static-module/node_modules/falafel/node_modules/foreach/index.js:12:16)
    at walk (wsmc/examples/mcwebchat/node_modules/brfs/node_modules/static-module/node_modules/falafel/index.js:34:9)

unable to repro in isolation (wzrd example.js in node-mojangson, the example parses), maybe an unintended interaction with global brfs browser transform or something else. repro with: browserify mcwebchat.js -g brfs

https://github.com/substack/static-eval/blob/c702b3e00091e0d351303796ff41c991e7babe62/index.js#L89 - static-eval walking CallExpression

       else if (node.type === 'CallExpression') {
            var callee = walk(node.callee);
            if (callee === FAIL) return FAIL;

            var ctx = node.callee.object ? walk(node.callee.object) : FAIL;
            if (ctx === FAIL) ctx = null;

            var args = [];
            for (var i = 0, l = node.arguments.length; i < l; i++) {
                var x = walk(node.arguments[i]);
                if (x === FAIL) return FAIL;
                args.push(x);
            }
            return callee.apply(ctx, args);
        }

the callee is an object: { resolve: [Function: resolver] }, args are [ "path" ]

ah, node-mojangson has a dynamic readFileSync in the generated grammar.js:

var source = require('fs').readFileSync(require('path').normalize(args[1]), "utf8");

but this shouldn't crash. Proposed a fix in static-eval (-> static-module -> static-module -> brfs):

https://github.com/substack/static-eval/pull/12 Fix exception instead of failure when parsing CallExpressions

and removing the affected (unused) unbrowserifiable code in node-mojangson:

https://github.com/rom1504/node-mojangson/pull/4

[deathcap]

mojangson 0.2.2 is in mineflayer 2570e20, testing with it able to brfs global transform now with no parse error - but the transformed requireindex crashes here:

          var files = FS.readdirSync(dir);

with no defined 'FS' (?) - this likely cannot be statically replaced. But mineflayer has had:

        var plugins = requireIndex(path.join(__dirname, 'lib', 'plugins'));

for a while, since at least https://github.com/PrismarineJS/mineflayer/commit/bd4eba1765bcc94e1f5cfd489f10a29d28c91849 - not sure how I had this working in browserify previously

[deathcap]

Mineflayer plugins were statically require()'d in wsmc/mineflayer-stream.js: https://github.com/deathcap/wsmc/blob/57ceb6a6fa9c7f637daf9f62bcfd5b1e1fefa02b/mineflayer-stream.js#L8-L31 - this plugin list (via the dynamic requireIndex) wasn't supposed to be used, but require('mineflayer') is called from the change to use require('mineflayer').Block (etc.) instead of require('mineflayer/lib/block') I had to in order to get prismarine-block: https://github.com/deathcap/wsmc/commit/d94181283b588b127269d1aed90d5d40d9964915

This puts browserified mineflayer in a bit of a pickle:

• I can't change back to require('mineflayer/lib/block') because block.js no longer exists in mineflayer (and neither does biome.js): https://github.com/PrismarineJS/mineflayer/tree/master/lib - they were moved to separate modules: https://github.com/PrismarineJS/mineflayer/commit/ca365c3c1bbdb39cf6632b71e7f3d6363d36c2ae - prismarine-block and prismarine-biome.
• If I change to require('mineflayer').Block to require("prismarine-block")(mc.minecraftVersion) (and likewise for biome), I'll have to include prismarine-block in my project, and ensure it matches the same version as mineflayer uses. If other objects move to their own modules, then I'll have to update my project to the new modules accordingly, increasing maintenance difficulty and version fragility.
• If I use require('mineflayer').Block then mineflayer/index.js will execute and hit the unbrowserifiable requireindex (maybe it only worked before due https://github.com/substack/static-eval/issues/10 Doesn't handle var statements with multiple declarations?)

I think any solution will require changes to mineflayer, but I'm not sure what is the correct fix going forward. Some ideas:

• Restore mineflayer/lib/block.js and biome, but have them just require prismarine-block, restoring require('mineflayer/lib/block') compatibility
• Shim requireindex (sidenode: this is https://www.npmjs.com/package/requireindex not https://www.npmjs.com/package/require-index) for browserify so it does nothing (since the plugins list is mineflayer's index.js is not used here)
• Change mineflayer/index.js to not use requireindex and just list the plugins statically like I do in wsmc/mineflayer-stream.js. Requires some more maintenance in mineflayer, as new plugins are added, but makes it directly compatible with browserify. Straightforward easy change.

update: above mineflayer change works, PR'd for consideration: https://github.com/PrismarineJS/mineflayer/pull/356

• Add a mineflayer browser.js alternative replacement for index.js with the static requires. Basically https://github.com/deathcap/wsmc/blob/d94181283b588b127269d1aed90d5d40d9964915/mineflayer-stream.js instead of https://github.com/PrismarineJS/mineflayer/blob/c3f67f98c7002c8aad720fde9ad280bde85dd4ac/index.js - but another change I have is using WS instead of TCP. There is some interesting in WS support in node-minecraft-protocol: https://github.com/PrismarineJS/node-minecraft-protocol/issues/312 - this could be a part of it. Would make wsmc simpler and easier to maintain if all the WS stuff was in NMP.
• Something else?

[deathcap]

After solving the above issue, next step is to update wsmc/minecraft-protocol-stream.js for the changes in node-minecraft-protocol/createClient.js.

[deathcap]

Current status: wsmc runs, but on ws connection, logs a bunch of errors (probably not transitioning states correctly — the ws is supposed to mainly be in the PLAY state):

Successfully connected to MC
compress { threshold: 256 }
Skipping state [object Object] packet:  <Buffer 03 80 02>
Skipping state [object Object] packet:  <Buffer 02 24 34 34 63 63 61 36 61 34 2d 34 30 36 65 2d 33 38 36 36 2d 39 35 63 64 2d 64 34 63 65 38 33 64 62 38 34 63 61 09 77 65 62 75 73 65 72 2d 36>
…

no chat messages received, attempting to send a message in mcwebchat fails with:

WebSocket error: Error: Serialization error for .toServer : SizeOf error for name : chat is not in the mappings value

stack trace points to ProtoDef but doesn't say where it is called:

exception.stack "Error: Serialization error for .toServer : SizeOf error for name : chat is not in the mappings value at ProtoDef.sizeOfMapper (http://localhost:9966/bundle.js:95584:34) at ProtoDef.sizeOf (http://localhost:9966/bundle.js:95985:33) at http://localhost:9966/bundle.js:95484:21 at tryCatch (http://localhost:9966/bundle.js:96077:12) at tryDoc (http://localhost:9966/bundle.js:96084:10) at http://localhost:9966/bundle.js:95483:19 at Array.reduce (native) at ProtoDef.sizeOfContainer (http://localhost:9966/bundle.js:95479:23) at ProtoDef.sizeOf (http://localhost:9966/bundle.js:95892:65) at ProtoDef.sizeOf (http://localhost:9966/bundle.js:95985:33)"

[rom1504]

"chat is not in the mappings value" means it's not in the correct state indeed. That error definitely doesn't say enough though.

[rom1504]

I think that's happening somewhere you do a .write('chat',...) and you are not in PLAY state.

[deathcap]

It's throwing the exception here:

mappings is only {0: "login_start", 1: "encryption_begin"}, so it's definitely in the wrong state.

Receiving the connect event, changes client.state in onConnect() to LOGIN, but not getting the success event which calls onLogin() to set client.state to PLAY. wsmc is logging:

    if (state !== 'play' && state !== 'login') {
      console.log('Skipping state '+state+' packet: ',buffer);
      return;
    }

state is now an object { size: 6, name: 'keep_alive', state: 'play' } - need to update this check.

[deathcap]

Tracking further updates in the pull request: https://github.com/deathcap/wsmc/pull/21

[deathcap]

Fixed in https://github.com/deathcap/wsmc/pull/21 https://github.com/deathcap/wsmc/commit/c44de84435dd22eb28fcbfa32dda0b158ccdd1f8