search

ProtoDef-io / node-protodef

Describe your protocol, and read it with ease.
MIT License
31 stars 18 forks source link

Add js compiler protodef implementation, that is 10x faster #102

Closed Karang closed 4 years ago

Karang commented 4 years ago

My attempt at https://github.com/ProtoDef-io/node-protodef/issues/86

The compiler can generate reading and writing code. It follows the protodef standard and can be extended with custom native types. It can compile a code that is able to read the whole nbt spec.

I also included an experimental function that can optimize the generated code using the closure compiler. It is not always faster and, when it is, not by much.

Running examples/compiled.js:

Running 1000000 tests
write / read compiled: 195.35 ms (5119.09k packet/s)
serializer / parser: 18095.48 ms (55.26k packet/s)
write / read compiled (+closure): 123.37 ms (8105.85k packet/s)

Running examples/compiled_nbt.js:

Running 10000 tests
read compiled: 282.89 ms (35.35k packet/s)
parser: 2237.11 ms (4.47k packet/s)
read compiled (+closure): 288.36 ms (34.68k packet/s)
rom1504 commented 4 years ago

That's cool. Did you try yet to read minecraft protocol.json with this or are there missing features that prevent it ?

On Mon, May 11, 2020, 21:57 Karang notifications@github.com wrote:

My attempt at #86 https://github.com/ProtoDef-io/node-protodef/issues/86

For now the compiler can only generate reading code, writing will be done later. It follows the protodef standard and can be extended with custom native types. It can compile a code that is able to read the whole nbt spec.

I also included an experimental function that can optimize the generated code using the closure compiler. It is not always faster and, when it is, not by much.

Running examples/compiled.js:

Running 1000000 tests read compiled: 98.29 ms (10173.47k packet/s) parser: 5061.42 ms (197.57k packet/s) read compiled (+closure): 79.97 ms (12504.31k packet/s)

Running examples/compiled_nbt.js:

Running 10000 tests read compiled: 414.75 ms (24.11k packet/s) parser: 2324.88 ms (4.30k packet/s) read compiled (+closure): 408.18 ms (24.50k packet/s)

You can view, comment on, or merge this pull request online at:

https://github.com/ProtoDef-io/node-protodef/pull/102 Commit Summary

  • First compiler version
  • Refactor compiler
  • Add nbt example

File Changes

Patch Links:

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ProtoDef-io/node-protodef/pull/102, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAR437U4WNDPLTRG6SEEOJDRRBKBBANCNFSM4M6GOAPQ .

Karang commented 4 years ago

That's cool. Did you try yet to read minecraft protocol.json with this or are there missing features that prevent it ?

As far as node-protodef is concerned, nothing is missing. But to implement the minecraft protocol, the specific primitives needs to be implemented : https://github.com/PrismarineJS/node-minecraft-protocol/blob/master/src/datatypes/minecraft.js (nbt is already done in my example). But following the example of https://github.com/ProtoDef-io/node-protodef/pull/102/files#diff-9e714e61d25d5e6f12b4fcd39795423fR64 it shouldn't be hard to do.

I included the nbt example here (and copy pasted some code from prismarine-nbt) for testing purpose, but in reality, the non- node-protodef code should be in its respective repository.

So, to support minecraft protocol, we need to first update prismarine-nbt then node-minecraft-protocol. (I can already implement the missing bits and include them here for testing, then we can move it later)

rom1504 commented 4 years ago

ok I get it, would be good to add a .md file similar to https://github.com/ProtoDef-io/node-protodef/blob/master/doc/newDatatypes.md that would explain how to add these types

rom1504 commented 4 years ago

Overall API looks ok, so I think it makes sense to start integrating directly in prismarine-nbt and node-minecraft-protocol (you can git depend on node-protodef for now in PRs)

rom1504 commented 4 years ago

would be nice to adapt the unit tests (https://github.com/ProtoDef-io/node-protodef/tree/master/test/dataTypes) and benchmarks (https://github.com/ProtoDef-io/node-protodef/tree/master/benchmark) to run on both the interpreter and the compiler (separately) it'll make it possible to catch any future regression + to do more precise benchmarking whenever needed

Karang commented 4 years ago

would be nice to adapt the unit tests (https://github.com/ProtoDef-io/node-protodef/tree/master/test/dataTypes) and benchmarks (https://github.com/ProtoDef-io/node-protodef/tree/master/benchmark) to run on both the interpreter and the compiler (separately) it'll make it possible to catch any future regression + to do more precise benchmarking whenever needed

done

rom1504 commented 4 years ago

LGTM, thanks for this huge improvement !