mtth / avsc

Avro for JavaScript :zap:
MIT License
1.27k stars 147 forks source link

Not being able to use records that use the "bytes" type field #460

Closed dbrito closed 2 months ago

dbrito commented 4 months ago

Hello

Has anyone here suffered from the problem of not being able to use records that use the "bytes" type field? I've already tried passing the information in byte format to the registry in many different ways (Uint8Array, Buffer, TextEncoder) but all without success

Code example in CodePen

Below is the test code

const avro = require('avsc');

const type = avro.Type.forSchema({
    type: 'record',
    name: 'Pet',
    fields: [
        { name: 'name', type: 'string' },
        { name: 'payload', type: 'bytes' },
    ]
});

const AVRO_RECORD = {
    name: 'Douglas',
    payload: new TextEncoder().encode('This is a string')
}

try {
    const buf = type.toBuffer(AVRO_RECORD); // Encoded buffer.
    const val = type.fromBuffer(buf); // AVRO_RECORD { name: 'Douglas', payload: .....}
    console.log('VALUE', val)
} catch (e) {
    console.error(e)
}
mtth commented 4 months ago

Hi @dbrito. bytes are mapped to Buffer instances in avsc's current release. Wrapping the text encoder's output in the CodePen works:

import avro from "https://esm.sh/avsc";
import buffer from "https://esm.sh/buffer";

const type = avro.Type.forSchema({
  type: 'record',
  name: 'Pet',
  fields: [         
    {name: 'name', type: 'string'},
    {name: 'payload', type: 'bytes'},
  ]
}); 

const AVRO_RECORD = {
  name: 'Douglas',
  payload: buffer.Buffer.from(new TextEncoder().encode('This is a string'))
}
dbrito commented 4 months ago

Hello @mtth

Thank you very much! This will help immensely in the rest of the project here, I'm implementing privacy-sandbox's attribution-reporting and I was stuck at this stage of generating the avro files for the reports

One more thank you very much