Change to use mineflayer for loading chunks

[deathcap]

There is already a fully-functional JavaScript-based MC client available at https://github.com/andrewrk/mineflayer in an NPM module. voxel-clientmc should use it, where possible, instead of reimplementing the client. Requires https://github.com/deathcap/mineflayer/commit/3f0c7e84629a7b0675e5486a82a250ee3ff8f4a5 for websocket support with https://github.com/deathcap/wsmc

As of 49fa436008557a08269e44b85d7c9ef72abaf407 voxel-clientmc uses mineflayer, but only chat messages are supported. Chunks should render, too. mineflayer has awareness of surrounding blocks, need to somehow bridge it to voxel.js.

[deathcap]

Some progress on this, chunks are now visible as of handling the spawn position packet https://github.com/deathcap/voxel-clientmc/commit/dcc1a0cfcec12e268a73e700d01b9b025cdb5a41 - but they are not properly completely decoded:

[roblabla]

I seriously don't recommend using mineflayer, since it's going to take a long while before it can become usable for 1.7/1.8

[deathcap]

Looking into mineflayer again, that is unfortunate it is no longer maintained as of late last year https://github.com/andrewrk/mineflayer/commit/3de8393da5c84fed23ee5fa68453dc9b95add330, stuck on MC 1.6.2. I see node-minecraft-protocol now supports 1.7.10, and 1.8 is in the works on a branch, cool. Yet Minecraft protocol compatibility seems to be a moving target, rarely (if ever?) is everything updated simultaneously and working on the same version, and once >1.8 is released many projects will probably require more updating :/

Considering whether it would be worth it to base voxel-clientmc on an older MC version, at least temporarily, to unblock development, stepping off the rat race of updating for a while. mineflayer does have a nice API, seems generic and high-level enough, at least from a quick glance, abstracted away from low-level protocol details, so maybe voxel-clientmc could be built on top of it, for 1.6.x (or even 1.5?), a solid foundation for testing. Then later, or in parallel, a (new project?) for 1.7 or 1.8 or >1.8 could be written to implement the parts of the API voxel-clientmc uses. API for reference: https://github.com/andrewrk/mineflayer/blob/master/doc/api.md#methods

Another point in favor of starting on an older version, there is a large collection of client/server modifications and server packs for pre-1.7, and certainly 1.8 is still in its early stages; e.g., FTB recently released their flagship pack early this month, for 1.7, and from what I can surmise there are still a decent number of people playing older packs, for 1.6, 1.5, or even 1.4, so developing for those versions wouldn't be a complete waste, and hopefully much of the code could be reused with a new updated 1.7/8/X API. I am interested in exploring support for modded / Forge servers, granted that's a ways off.

To investigate further. Maybe mineflayer isn't the way to go and I'm placing too much importance on it, but it does appear to be the closest project doing what I'm looking for, implementing a fully-fledged MC client in JavaScript. Any alternatives?

[roblabla]

The biggest roadblock for updating mineflayer is the fact that it's based on Burger to extract biome/block/etc... information from the minecraft jar, which has been dropped at 1.7, and it turns out to be a major pain to update. Mineflayer has a very nice and flexible API, but putting all the biome and block information by hand in it is very slow and error-prone. Besides, it's probably missing a lot of logic.

node-minecraft-protocol is very bare and provide very little abstraction, so creating a project based on it will generate a lot of maintenance work each protocol update (and that's after node-minecraft-protocol itself is updated... which, as proven by 1.8, becomes increasingly difficult each new update). Besides, projects based on node-minecraft-protocol are written for one version of minecraft, and will probably never be able to be truly compatible accross minecraft version.

[deathcap]

Hm.. having the biome and block and other enums https://github.com/andrewrk/mineflayer/tree/master/lib/enums in mineflayer doesn't seem too important, at least for a basic client. The biome data (id, color, height, name, rainfall, temperature) is nice to have but clients ought to be able to ignore it right? And just show the numeric biome id, worst case. I guess the block data is more important: stack size, diggable, harvestable tools, bounding box, etc., for clients to properly interact with each type of block. I see what you mean. No easy answers :/

Perhaps starting with a node-minecraft-client client example https://github.com/CraftJS/node-minecraft-protocol/tree/master/examples and expanding it as needed is the way to go.

Trying the packets-1.8 branch, examples/client_echo.js, but seems to get stuck on large packets: https://gist.github.com/deathcap/4624e58ca86311797a10

[roblabla]

packets-1.8 branch isn't finished yet. Lots of bugs to fix.

https://github.com/CraftJS/node-minecraft-protocol/wiki/Walking-in-straight-line could interest you.

I think we should be writing a newer abstraction on top of the node-minecraft-protocol project. One that doesn't depend on external tools, as to avoid the problem we currently have with mineflayer. I'll see if I can write a new tool to extract block information from the current 1.7/1.8 jars. I had already started doing that, but didn't find the time to finish.

In the meantime, we could just make a very crude categorization of the blocks, manually. Kinda like the node-minecraft-protocol packets.

[deathcap]

Update: minecraft-protocol 0.13 released for MC 1.8, updated development versions of mineflayer and wsmc now available for 1.8. voxel-clientmc can now begin to update..

mineflayer is essentially a headless MC client, so we'll get a fair amount of functionality for free. The challenge is then to expose it graphically in voxel-clientmc

[deathcap]

Chat works (in 1.8, just as it did in 1.6, back to parity) but the chunk format changed (individual fields no longer compressed, data/extended fields merged into 16-bit block ids: https://github.com/andrewrk/mineflayer/commit/dbe6143b0fd10793b0acd29d6e0614be19e5e3b2), the manual parsing code in voxel-clientmc is no longer usable anyways. Instead, should listen for chunkColumnLoad from mineflayer, iterate from the chunk corner using blockAt vec3 x,y,z (note: might be slow since blockAt calculates the chunk Location from the point on each call, but can be optimized later), then populate the voxel.js chunk ndarray with the translated block ids accordingly

[roblabla]

As I've said before, if you want to make a "forEachBlock" function in mineflayer, I'm all for it. something like forEachBlock(/*origin*/{ x, y, z}, /* size */{x, y, z}, function (block)) ? PRs welcome. That would avoid the need to expose chunks, while having the possibility of being more optimized than repeated blockAt calls.

[deathcap]

Yeah I'll have to see about optimizing block iteration using something like that, probably in mineflayer but maybe also in voxel.js and voxel-clientmc. I'm thinking in any case chunk loading should occur in a web worker to not block the main thread. Using a simple loop over blockAt + setBlock, sort of works:

[deathcap]

Some decent optimizations on the voxel.js side in https://github.com/deathcap/voxel-clientmc/commit/0ba600a5eeba2a8b5b17d7133e4609f182e8201e and https://github.com/deathcap/voxel-clientmc/commit/27291ea60d912d362cd48e7bcb59e70fc4fb2cc0 yet still around 500 ms/chunk. But I am using mineflayer, so I'll close this issue, opening another one for more optimization: https://github.com/deathcap/voxel-clientmc/issues/8