Run chunk loading translation in web worker

[deathcap]

Handle the chunkColumnLoad-event-triggered chunk loading, translating the MC/mineflayer chunks to voxel.js chunk data (ndarray), in a web worker to avoid blocking the UI during loading (currently takes ≈500 ms/chunk)

---

optimization idea: have the worker accumulate cubic voxeljs chunks (32x32x32) from mc's chunks (16x256x16), and send the raw chunk data as an ArrayBuffer (as a transferrable to avoid copying) back to the main thread, where it does game.showChunk on the completed chunk - similar to voxel-land (worker, listener)

[deathcap]

The forEachBlock(/*origin*/{ x, y, z}, /* size */{x, y, z}, function (block)) api for mineflayer proposed in https://github.com/deathcap/voxel-clientmc/issues/5#issuecomment-85180048 is appealing, would avoid the repeated chunk calculation lookups in mineflayer lib/plugins/blocks.js:

  function blockAt(absolutePoint) {
    var loc = new Location(absolutePoint);
    var key = columnKeyXZ(loc.chunkCorner.x, loc.chunkCorner.z);
    var column = columns[key];
    // null column means chunk not loaded
    if (! column) return null;
    var blockType = bite(column.blockType);
    var nibbleIndex = loc.blockIndex >> 1;
    var lowNibble = loc.blockIndex % 2 === 1;

    var biomeId = column.biome.readUInt8(loc.biomeBlockIndex);

    var block = new Block(blockType >> 4, biomeId);
    block.metadata = blockType & 0x0f;
    block.light = nib(column.light);
    block.skyLight = nib(column.skyLight);
    block.add = nib(column.add);
    block.position = loc.floored;
    block.signText = signs[loc.floored];
    block.painting = paintingsByPos[loc.floored];

    return block;

but, workers can only be passed serialized data, preferably ArrayBuffers (transferred, zero copy), so the main thread would have to iterate in forEachBlock, build the buffer, then pass to the worker to iterate again and translate to a voxel.js chunk array? Would like to move all of the iteration into the worker if possible.

new idea: run mineflayer entirely in a separate web worker, post messages to/from main thread

[deathcap]

Run with webworkify, pipe a workerstream to a websocket-stream… but, there's a problem loading mineflayer in a worker environment, which is more restricted than the main browser context. Surprisingly, lodash crashes, of all places:

ErrorEvent {error: null, colno: 29, lineno: 44571, filename: "blob:http%3A//localhost%3A9966/c24cb815-eeca-4f9b-80e8-8afc63eb7e33", message: "Uncaught TypeError: Cannot read property 'prototype' of undefined"…}bubbles: falsecancelBubble: falsecancelable: truecolno: 29currentTarget: WorkerdefaultPrevented: falseerror: nulleventPhase: 2filename: "blob:http%3A//localhost%3A9966/c24cb815-eeca-4f9b-80e8-8afc63eb7e33"lineno: 44571message: "Uncaught TypeError: Cannot read property 'prototype' of undefined"path: NodeList[0]returnValue: truesrcElement: Workertarget: WorkertimeStamp: 1427253552623type: "error"proto: ErrorEvent

    /** Used for native method references */
    var objectProto = Object.prototype; // TypeError: Cannot read property 'prototype' of undefined

Is Object really not supposed to be available in workers? https://github.com/lodash/lodash/issues/223

edit: updating lodash in mineflayer fixes this: https://github.com/andrewrk/mineflayer/pull/238

[deathcap]

The worker gets the packet data but there's a slight impedance mismatch with workerstream:

https://github.com/maxogden/workerstream/blob/8313b7c82deb1b46f75093688c54bc1ab7e20d15/parent.js#L21-L23

``javascript ParentStream.prototype.parentMessage = function(e) { var data = e.data; this.emit('data', data, e) }

from workerGlobal.parent.onmessage http://www.w3.org/TR/workers/#communicating-with-a-dedicated-worker - event .data is an Uint8Array, typed array, not augmented to a nodejs `Buffer`. Can be converted with https://github.com/feross/typedarray-to-buffer, modifying workerstream as follows:

```javascript
var toBuffer = require('typedarray-to-buffer');

  if (!Buffer.isBuffer(data)) data = toBuffer(data); // in parentMessage

websocket-stream had a similar fix, converting in https://github.com/maxogden/websocket-stream/commit/0089ece2411d576910e7b5a42aaf27c82a1ff10e#diff-168726dbe96b3ce427e7fedce31bb0bcR58

    if (data instanceof ArrayBuffer) data = new Buffer(new Uint8Array(data))

without this conversion, crashes in node-minecraft-protocol/src/client.js on the 'data' event:

  self.socket.on('data', function(data) {
    if (self.encryptionEnabled) data = new Buffer(self.decipher.update(data), 'binary');
    incomingBuffer = Buffer.concat([incomingBuffer, data]); // data is expected to be a Buffer
    prepareParse()
  });

question is, where should this be fixed? workerstream (to always emit Buffers), buffer (to accept concat Uint8Array), mineflayer (likely too web-specific), mf-worker (stream to augment Uint8Arrays to Buffers), or elsewhere? update: went with https://github.com/maxogden/workerstream/pull/11

[deathcap]

Merged in GH-9