mtth / avsc

Avro for JavaScript :zap:
MIT License
1.27k stars 147 forks source link

Question about decoding messages in the browser #412

Closed Filip785 closed 1 year ago

Filip785 commented 1 year ago

Hi, thanks for the library.

I'm trying to use it to decode message in the browser, through WebSocket. I'm sending message from the websocket server in avro binary format as so:

// ...
const avscSchema = avsc.Type.forSchema({
    type: 'record',
    name: 'TestEvent',
    fields: [
        {
            name: 'event',
            type: 'string'
        },
        {
            name: 'count',
            type: 'int'
        }
    ]
});

app.publish('test_publish', avscSchema.toBuffer({event: 'TEST_EVENT', count: 1}));
// ...

Now, on the frontend I receive binary message like this:

// ...
const ws = new WebSocket('<connection_url>');

ws.addEventListener('message', async (e) => {
    // data is instanceof Blob
    const { data } = e;

    // convert to the JS object
    const arrayBuffer = await data.arrayBuffer();
    const buffer = Buffer.from(arrayBuffer);

    const avscSchema = avsc.Type.forSchema({
        type: 'record',
        name: 'TestEvent',
        fields: [
            {
                name: 'event',
                type: 'string'
            },
            {
                name: 'count',
                type: 'int'
            }
        ]
    });   

    // originalObject = {event: 'TEST_EVENT', count: 1}
    const originalObject = avscSchema.fromBuffer(buffer); 
});
// ...

Now, my question is, how can I know what type of schema is coming in from the message (as there can be multiple message types)? How can I know this beforehand so I can convert to correct object? Is it even possible?

Thanks.

mtth commented 1 year ago

Hi @Filip785. A common way of addressing this is with a schema registry, where each schema has a unique ID which you serialize alongside the data. If you have few events you could also combine them into a single union type which you would then use to serialize the messages (sending the combined schema when you first connect for example, so you don't need a separate registry).