phretaddin / schemapack

Create a schema object to encode/decode your JSON in to a compact byte buffer with no overhead.
MIT License
452 stars 33 forks source link

How to include a op code in the first byte always? #17

Closed Jared-Sprague closed 8 years ago

Jared-Sprague commented 8 years ago

Hey @phretaddin!

Thanks for all your help so far! I have another question. We want to convert all of our WebSocket messages that are currently using json to schemapack, but we have a problem: how to know which message is witch?

We are using 'ws' package on the server and, standard native WebSocket on the client.

Currently this is easy because when we send a json message one of the peroperties is 'op' e.g. From server: ws.send(JSON.stringify({op: 'welcome', message: 'welcome to our game'})); On client:

...
switch (message.op) {
    case 'welcome':
    handle_msg_welcome(message);
    break;
...

This is easy when using json, but when we convert to schemapack and everything is a buffer, and we're not using Socket.io that can include a string key with every message, we loose the ability to easily tell messages apart.

Maybe you've already solved this problem in your own projects? One of the ways I was considering solving this was to make the first byte of every buffer a numeric op code in the schema e.g:

var welcomeSchema = sp.build({
    op: "uint8',
    message: "string",
});
var playerSchema = sp.build({
    op: "uint8',
    health: "varuint",
    jumping: "boolean",
    position: [ "int16" ],
    attributes: { str: 'uint8', agi: 'uint8', int: 'uint8' }
});

Then read the first byte directly before deciding which schema to use to decode:

var op == readFirstByte(buffer);
switch (op) {
    case ops.WELCOME:
        handle_welcome( welcomeSchema.decode(buffer) );
        break;
    case ops.PLAYER:
        handle_player( playerSchema.decode(buffer) );
        break;
...

What do you think of that idea? If that's a good solution, how do I guarantee that 'op' will always be in the first byte of the buffer? I know you mentioned that the object is sorted, maybe naming it with double underscore __op: "uint8" or something will force it to be the first in sort order?

Jared-Sprague commented 8 years ago

Looking at your sort method

Looks like if I add a property of 0 it will sort to the first element of the keys e.g.:

var welcomeSchema = sp.build({
    0: "uint8',
    message: "string",
});

Does that mean that 0 propterty of the schema above can always be read as the first byte of the buffer? is that safe?

phretaddin commented 8 years ago

Yes, that should work. That's also how I do it for my game (just have a key like "__message" or "-t" or whatever). As long as it alphabetizes to the first element, you can just read the first byte to determine the type of the message.

Jared-Sprague commented 8 years ago

Ok thanks!

Jared-Sprague commented 8 years ago

Can you share how you are reading the first byte?

phretaddin commented 8 years ago

You could turn it into a Uint8Array.

var view = new Uint8Array(data);
var packetType = view[0];

You could use a DataView too with getUint8(). It just depends on what datatype you're dealing with with your websocket library. If it's a Uint8Array already, you won't have to do anything other than just doing data[0].

Jared-Sprague commented 8 years ago

That worked great thanks!!