mtth / avsc

Avro for JavaScript :zap:
MIT License
1.27k stars 147 forks source link

invalid union index: 11 #416

Closed rcnespoli closed 1 year ago

rcnespoli commented 1 year ago

Hi, I'm receiving this error when I try to decode. Error message:

{"type":"Error","message":"invalid union index: 11","stack":"Error: invalid union index: 11
    at UnwrappedUnionType._read (/home/node/app/node_modules/avsc/lib/types.js:1318:11)
    at RecordType.readRecord [as _read] (eval at RecordType._createReader (/home/node/app/node_modules/avsc/lib/types.js:2296:10), <anonymous>:5:8)
    at readValue (/home/node/app/node_modules/avsc/lib/types.js:2902:17)
    at RecordType.Type.fromBuffer (/home/node/app/node_modules/avsc/lib/types.js:601:13)
    at KafkaMessageParser.decode (/home/node/app/dist/src/kafka/application/kafka-message-parser.js:30:29)
    at CustomersPartyChangedHandler.execute (/home/node/app/dist/src/customer/application/kafka/handlers/customer-party-changed.handler.js:30:26)
    at KafkaMessageBrokerConsumer.messagesHandle (/home/node/app/dist/src/kafka/application/kafka-message-broker-consumer.js:68:14)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Runner.eachMessage (/home/node/app/dist/src/kafka/application/kafka-message-broker-consumer.js:54:17)
    at async Runner.processEachMessage (/home/node/app/node_modules/kafkajs/src/consumer/runner.js:231:9)"},"msg":"invalid union index: 11"}

Buffered message:

[79,98,106,1,2,22,97,118,114,111,46,115,99,104,101,109,97,156,71,123,34,116,121,112,101,34,58,34,114,101,99,111,114,100,34,44,34,110,97,109,101,34,58,34,114,101,99,111,114,100,34,44,34,102,105,101,108,100,115,34,58,91,123,34,110,97,109,101,34,58,34,109,101,116,97,95,105,100,34,44,34,116,121,112,101,34,58,91,34,108,111,110,103,34,44,34,100,111,117,98,108,101,34,44,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,109,101,116,97,95,101,118,101,110,116,95,116,121,112,101,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,109,101,116,97,95,101,110,116,105,116,121,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,109,101,116,97,95,108,97,121,101,114,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,109,101,116,97,95,109,97,115,116,101,114,95,118,105,101,119,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,109,101,116,97,95,111,114,105,103,105,110,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,109,101,116,97,95,115,111,117,114,99,101,95,115,121,115,116,101,109,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,109,101,116,97,95,109,111,100,105,102,105,101,100,95,98,121,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,109,101,116,97,95,97,99,116,105,118,97,116,105,111,110,95,100,97,116,101,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,109,101,116,97,95,99,114,101,97,116,105,111,110,95,100,97,116,101,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,109,101,116,97,95,100,101,97,99,116,105,118,97,116,105,111,110,95,100,97,116,101,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,109,101,116,97,95,100,101,108,101,116,105,111,110,95,100,97,116,101,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,109,101,116,97,95,108,97,115,116,95,117,112,100,97,116,101,95,100,97,116,101,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,109,101,116,97,95,108,97,115,116,95,115,111,117,114,99,101,95,117,112,100,97,116,101,95,100,97,116,101,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,109,101,116,97,95,115,111,117,114,99,101,95,116,105,109,101,115,116,97,109,112,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,109,101,116,97,95,97,99,116,105,118,97,116,105,111,110,95,116,105,100,34,44,34,116,121,112,101,34,58,91,34,108,111,110,103,34,44,34,100,111,117,98,108,101,34,44,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,109,101,116,97,95,99,114,101,97,116,105,111,110,95,116,105,100,34,44,34,116,121,112,101,34,58,91,34,108,111,110,103,34,44,34,100,111,117,98,108,101,34,44,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,109,101,116,97,95,100,101,97,99,116,105,118,97,116,105,111,110,95,116,105,100,34,44,34,116,121,112,101,34,58,91,34,108,111,110,103,34,44,34,100,111,117,98,108,101,34,44,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,109,101,116,97,95,100,101,108,101,116,105,111,110,95,116,105,100,34,44,34,116,121,112,101,34,58,91,34,108,111,110,103,34,44,34,100,111,117,98,108,101,34,44,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,109,101,116,97,95,108,97,115,116,95,117,112,100,97,116,101,95,116,105,100,34,44,34,116,121,112,101,34,58,91,34,108,111,110,103,34,44,34,100,111,117,98,108,101,34,44,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,109,101,116,97,95,108,97,115,116,95,115,111,117,114,99,101,95,117,112,100,97,116,101,95,116,105,100,34,44,34,116,121,112,101,34,58,91,34,108,111,110,103,34,44,34,100,111,117,98,108,101,34,44,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,114,101,99,111,114,100,95,101,110,103,95,97,99,116,105,118,101,34,44,34,116,121,112,101,34,58,91,34,98,111,111,108,101,97,110,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,111,108,100,95,101,110,103,95,97,99,116,105,118,101,34,44,34,116,121,112,101,34,58,91,34,98,111,111,108,101,97,110,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,114,101,99,111,114,100,95,101,110,103,95,108,97,115,116,95,117,112,100,97,116,101,95,100,97,116,101,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,111,108,100,95,101,110,103,95,108,97,115,116,95,117,112,100,97,116,101,95,100,97,116,101,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,114,101,99,111,114,100,95,101,110,103,95,99,114,101,97,116,105,111,110,95,100,97,116,101,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,111,108,100,95,101,110,103,95,99,114,101,97,116,105,111,110,95,100,97,116,101,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,114,101,99,111,114,100,95,101,110,103,95,100,101,108,101,116,105,111,110,95,100,97,116,101,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,111,108,100,95,101,110,103,95,100,101,108,101,116,105,111,110,95,100,97,116,101,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,114,101,99,111,114,100,95,101,110,103,95,97,99,116,105,118,97,116,105,111,110,95,100,97,116,101,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,111,108,100,95,101,110,103,95,97,99,116,105,118,97,116,105,111,110,95,100,97,116,101,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,114,101,99,111,114,100,95,101,110,103,95,100,101,97,99,116,105,118,97,116,105,111,110,95,100,97,116,101,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,111,108,100,95,101,110,103,95,100,101,97,99,116,105,118,97,116,105,111,110,95,100,97,116,101,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,114,101,99,111,114,100,95,101,110,103,95,110,97,116,117,114,101,34,44,34,116,121,112,101,34,58,91,34,98,111,111,108,101,97,110,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,111,108,100,95,101,110,103,95,110,97,116,117,114,101,34,44,34,116,121,112,101,34,58,91,34,98,111,111,108,101,97,110,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,114,101,99,111,114,100,95,101,110,103,95,109,111,100,105,102,105,101,100,95,98,121,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,111,108,100,95,101,110,103,95,109,111,100,105,102,105,101,100,95,98,121,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,114,101,99,111,114,100,95,112,97,114,116,121,95,109,97,115,116,101,114,95,105,100,34,44,34,116,121,112,101,34,58,91,34,108,111,110,103,34,44,34,100,111,117,98,108,101,34,44,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,111,108,100,95,112,97,114,116,121,95,109,97,115,116,101,114,95,105,100,34,44,34,116,121,112,101,34,58,91,34,108,111,110,103,34,44,34,100,111,117,98,108,101,34,44,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,114,101,99,111,114,100,95,99,109,111,95,112,104,111,110,101,95,110,117,109,98,101,114,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,111,108,100,95,99,109,111,95,112,104,111,110,101,95,110,117,109,98,101,114,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,114,101,99,111,114,100,95,99,109,111,95,112,104,111,110,101,95,117,115,97,103,101,95,116,121,112,101,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,111,108,100,95,99,109,111,95,112,104,111,110,101,95,117,115,97,103,101,95,116,121,112,101,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,114,101,99,111,114,100,95,99,109,111,95,112,104,111,110,101,95,103,114,111,117,112,34,44,34,116,121,112,101,34,58,91,34,105,110,116,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,111,108,100,95,99,109,111,95,112,104,111,110,101,95,103,114,111,117,112,34,44,34,116,121,112,101,34,58,91,34,105,110,116,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,114,101,99,111,114,100,95,99,109,111,95,99,111,117,110,116,114,121,95,99,111,100,101,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,111,108,100,95,99,109,111,95,99,111,117,110,116,114,121,95,99,111,100,101,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,114,101,99,111,114,100,95,99,109,111,95,97,114,101,97,95,99,111,100,101,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,111,108,100,95,99,109,111,95,97,114,101,97,95,99,111,100,101,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,114,101,99,111,114,100,95,99,109,111,95,101,120,99,104,97,110,103,101,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,111,108,100,95,99,109,111,95,101,120,99,104,97,110,103,101,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,114,101,99,111,114,100,95,99,109,111,95,108,111,99,97,108,95,110,117,109,98,101,114,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,111,108,100,95,99,109,111,95,108,111,99,97,108,95,110,117,109,98,101,114,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,114,101,99,111,114,100,95,99,109,111,95,101,120,116,101,110,115,105,111,110,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,111,108,100,95,99,109,111,95,101,120,116,101,110,115,105,111,110,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,114,101,99,111,114,100,95,99,109,111,95,118,97,108,105,100,95,105,110,100,105,99,97,116,111,114,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,111,108,100,95,99,109,111,95,118,97,108,105,100,95,105,110,100,105,99,97,116,111,114,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,114,101,99,111,114,100,95,99,109,111,95,115,116,97,110,100,97,114,100,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,111,108,100,95,99,109,111,95,115,116,97,110,100,97,114,100,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,114,101,99,111,114,100,95,99,109,111,95,108,97,115,116,95,117,112,100,97,116,101,95,100,116,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,111,108,100,95,99,109,111,95,108,97,115,116,95,117,112,100,97,116,101,95,100,116,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,114,101,99,111,114,100,95,99,109,111,95,108,97,115,116,95,117,112,100,97,116,101,100,95,117,115,101,114,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,111,108,100,95,99,109,111,95,108,97,115,116,95,117,112,100,97,116,101,100,95,117,115,101,114,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,114,101,99,111,114,100,95,99,109,111,95,115,116,97,116,117,115,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,111,108,100,95,99,109,111,95,115,116,97,116,117,115,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,104,97,115,95,101,118,101,110,116,34,44,34,116,121,112,101,34,58,91,34,98,111,111,108,101,97,110,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,114,101,99,111,114,100,95,109,97,115,116,101,114,95,105,100,34,44,34,116,121,112,101,34,58,91,34,108,111,110,103,34,44,34,100,111,117,98,108,101,34,44,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,111,108,100,95,109,97,115,116,101,114,95,105,100,34,44,34,116,121,112,101,34,58,91,34,108,111,110,103,34,44,34,100,111,117,98,108,101,34,44,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,114,101,99,111,114,100,95,115,111,117,114,99,101,95,105,100,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,111,108,100,95,115,111,117,114,99,101,95,105,100,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,114,101,99,111,114,100,95,112,97,114,116,121,95,115,111,117,114,99,101,95,105,100,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,111,108,100,95,112,97,114,116,121,95,115,111,117,114,99,101,95,105,100,34,44,34,116,121,112,101,34,58,91,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,114,101,99,111,114,100,95,112,97,114,116,121,95,109,97,115,116,101,114,95,105,100,95,105,34,44,34,116,121,112,101,34,58,91,34,108,111,110,103,34,44,34,100,111,117,98,108,101,34,44,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,111,108,100,95,112,97,114,116,121,95,109,97,115,116,101,114,95,105,100,95,105,34,44,34,116,121,112,101,34,58,91,34,108,111,110,103,34,44,34,100,111,117,98,108,101,34,44,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,109,101,116,97,95,105,100,95,105,34,44,34,116,121,112,101,34,58,91,34,108,111,110,103,34,44,34,100,111,117,98,108,101,34,44,34,115,116,114,105,110,103,34,44,34,110,117,108,108,34,93,125,44,123,34,110,97,109,101,34,58,34,104,97,115,95,101,118,101,110,116,95,105,34,44,34,116,121,112,101,34,58,91,34,98,111,111,108,101,97,110,34,44,34,110,117,108,108,34,93,125,93,125,0,13,34,136,127,150,67,138,104,147,251,7,108,214,6,126,15,2,164,4,0,166,23,0,12,73,78,83,69,82,84,0,14,112,104,111,110,101,95,109,0,12,109,97,115,116,101,114,0,14,109,97,115,116,101,114,115,2,2,0,10,97,100,109,105,110,0,38,50,48,50,50,45,49,49,45,50,56,32,49,52,58,53,49,58,53,56,0,38,50,48,50,50,45,49,49,45,50,56,32,49,52,58,53,49,58,53,56,2,2,0,38,50,48,50,50,45,49,49,45,50,56,32,49,52,58,53,49,58,53,56,2,2,0,212,15,0,212,15,6,6,0,212,15,6,0,1,0,0,2,2,2,2,2,2,2,2,2,2,2,0,0,2,2,0,192,20,6,0,28,40,52,49,54,41,32,53,55,57,45,48,54,56,55,2,0,34,84,101,108,101,112,104,111,110,101,32,68,97,121,116,105,109,101,2,0,2,2,2,2,0,6,52,49,54,2,0,6,53,55,57,2,0,8,48,54,56,55,2,2,2,0,24,86,97,108,105,100,32,78,117,109,98,101,114,2,2,2,2,2,2,2,0,12,65,99,116,105,118,101,2,0,1,0,166,23,6,0,12,53,54,53,51,51,50,2,0,12,53,54,49,48,49,53,2,0,192,20,6,0,168,17,0,1,13,34,136,127,150,67,138,104,147,251,7,108,214,6,126,15]

Unbuffered message:

"Obj\u0001\u0002\u0016avro.schema�G{\"type\":\"record\",\"name\":\"record\",\"fields\":[{\"name\":\"meta_id\",\"type\":[\"long\",\"double\",\"string\",\"null\"]},{\"name\":\"meta_event_type\",\"type\":[\"string\",\"null\"]},{\"name\":\"meta_entity\",\"type\":[\"string\",\"null\"]},{\"name\":\"meta_layer\",\"type\":[\"string\",\"null\"]},{\"name\":\"meta_master_view\",\"type\":[\"string\",\"null\"]},{\"name\":\"meta_origin\",\"type\":[\"string\",\"null\"]},{\"name\":\"meta_source_system\",\"type\":[\"string\",\"null\"]},{\"name\":\"meta_modified_by\",\"type\":[\"string\",\"null\"]},{\"name\":\"meta_activation_date\",\"type\":[\"string\",\"null\"]},{\"name\":\"meta_creation_date\",\"type\":[\"string\",\"null\"]},{\"name\":\"meta_deactivation_date\",\"type\":[\"string\",\"null\"]},{\"name\":\"meta_deletion_date\",\"type\":[\"string\",\"null\"]},{\"name\":\"meta_last_update_date\",\"type\":[\"string\",\"null\"]},{\"name\":\"meta_last_source_update_date\",\"type\":[\"string\",\"null\"]},{\"name\":\"meta_source_timestamp\",\"type\":[\"string\",\"null\"]},{\"name\":\"meta_activation_tid\",\"type\":[\"long\",\"double\",\"string\",\"null\"]},{\"name\":\"meta_creation_tid\",\"type\":[\"long\",\"double\",\"string\",\"null\"]},{\"name\":\"meta_deactivation_tid\",\"type\":[\"long\",\"double\",\"string\",\"null\"]},{\"name\":\"meta_deletion_tid\",\"type\":[\"long\",\"double\",\"string\",\"null\"]},{\"name\":\"meta_last_update_tid\",\"type\":[\"long\",\"double\",\"string\",\"null\"]},{\"name\":\"meta_last_source_update_tid\",\"type\":[\"long\",\"double\",\"string\",\"null\"]},{\"name\":\"record_eng_active\",\"type\":[\"boolean\",\"null\"]},{\"name\":\"old_eng_active\",\"type\":[\"boolean\",\"null\"]},{\"name\":\"record_eng_last_update_date\",\"type\":[\"string\",\"null\"]},{\"name\":\"old_eng_last_update_date\",\"type\":[\"string\",\"null\"]},{\"name\":\"record_eng_creation_date\",\"type\":[\"string\",\"null\"]},{\"name\":\"old_eng_creation_date\",\"type\":[\"string\",\"null\"]},{\"name\":\"record_eng_deletion_date\",\"type\":[\"string\",\"null\"]},{\"name\":\"old_eng_deletion_date\",\"type\":[\"string\",\"null\"]},{\"name\":\"record_eng_activation_date\",\"type\":[\"string\",\"null\"]},{\"name\":\"old_eng_activation_date\",\"type\":[\"string\",\"null\"]},{\"name\":\"record_eng_deactivation_date\",\"type\":[\"string\",\"null\"]},{\"name\":\"old_eng_deactivation_date\",\"type\":[\"string\",\"null\"]},{\"name\":\"record_eng_nature\",\"type\":[\"boolean\",\"null\"]},{\"name\":\"old_eng_nature\",\"type\":[\"boolean\",\"null\"]},{\"name\":\"record_eng_modified_by\",\"type\":[\"string\",\"null\"]},{\"name\":\"old_eng_modified_by\",\"type\":[\"string\",\"null\"]},{\"name\":\"record_party_master_id\",\"type\":[\"long\",\"double\",\"string\",\"null\"]},{\"name\":\"old_party_master_id\",\"type\":[\"long\",\"double\",\"string\",\"null\"]},{\"name\":\"record_cmo_phone_number\",\"type\":[\"string\",\"null\"]},{\"name\":\"old_cmo_phone_number\",\"type\":[\"string\",\"null\"]},{\"name\":\"record_cmo_phone_usage_type\",\"type\":[\"string\",\"null\"]},{\"name\":\"old_cmo_phone_usage_type\",\"type\":[\"string\",\"null\"]},{\"name\":\"record_cmo_phone_group\",\"type\":[\"int\",\"null\"]},{\"name\":\"old_cmo_phone_group\",\"type\":[\"int\",\"null\"]},{\"name\":\"record_cmo_country_code\",\"type\":[\"string\",\"null\"]},{\"name\":\"old_cmo_country_code\",\"type\":[\"string\",\"null\"]},{\"name\":\"record_cmo_area_code\",\"type\":[\"string\",\"null\"]},{\"name\":\"old_cmo_area_code\",\"type\":[\"string\",\"null\"]},{\"name\":\"record_cmo_exchange\",\"type\":[\"string\",\"null\"]},{\"name\":\"old_cmo_exchange\",\"type\":[\"string\",\"null\"]},{\"name\":\"record_cmo_local_number\",\"type\":[\"string\",\"null\"]},{\"name\":\"old_cmo_local_number\",\"type\":[\"string\",\"null\"]},{\"name\":\"record_cmo_extension\",\"type\":[\"string\",\"null\"]},{\"name\":\"old_cmo_extension\",\"type\":[\"string\",\"null\"]},{\"name\":\"record_cmo_valid_indicator\",\"type\":[\"string\",\"null\"]},{\"name\":\"old_cmo_valid_indicator\",\"type\":[\"string\",\"null\"]},{\"name\":\"record_cmo_standard\",\"type\":[\"string\",\"null\"]},{\"name\":\"old_cmo_standard\",\"type\":[\"string\",\"null\"]},{\"name\":\"record_cmo_last_update_dt\",\"type\":[\"string\",\"null\"]},{\"name\":\"old_cmo_last_update_dt\",\"type\":[\"string\",\"null\"]},{\"name\":\"record_cmo_last_updated_user\",\"type\":[\"string\",\"null\"]},{\"name\":\"old_cmo_last_updated_user\",\"type\":[\"string\",\"null\"]},{\"name\":\"record_cmo_status\",\"type\":[\"string\",\"null\"]},{\"name\":\"old_cmo_status\",\"type\":[\"string\",\"null\"]},{\"name\":\"has_event\",\"type\":[\"boolean\",\"null\"]},{\"name\":\"record_master_id\",\"type\":[\"long\",\"double\",\"string\",\"null\"]},{\"name\":\"old_master_id\",\"type\":[\"long\",\"double\",\"string\",\"null\"]},{\"name\":\"record_source_id\",\"type\":[\"string\",\"null\"]},{\"name\":\"old_source_id\",\"type\":[\"string\",\"null\"]},{\"name\":\"record_party_source_id\",\"type\":[\"string\",\"null\"]},{\"name\":\"old_party_source_id\",\"type\":[\"string\",\"null\"]},{\"name\":\"record_party_master_id_i\",\"type\":[\"long\",\"double\",\"string\",\"null\"]},{\"name\":\"old_party_master_id_i\",\"type\":[\"long\",\"double\",\"string\",\"null\"]},{\"name\":\"meta_id_i\",\"type\":[\"long\",\"double\",\"string\",\"null\"]},{\"name\":\"has_event_i\",\"type\":[\"boolean\",\"null\"]}]}\u0000\r\"��C�h��\u0007l�\u0006~\u000f\u0002�\u0004\u0000�\u0017\u0000\fINSERT\u0000\u000ephone_m\u0000\fmaster\u0000\u000emasters\u0002\u0002\u0000
admin\u0000&2022-11-28 14:51:58\u0000&2022-11-28 14:51:58\u0002\u0002\u0000&2022-11-28 14:51:58\u0002\u0002\u0000�\u000f\u0000�\u000f\u0006\u0006\u0000�\u000f\u0006\u0000\u0001\u0000\u0000\u0002\u0002\u0002\u0002\u0002\u0002\u0002\u0002\u0002\u0002\u0002\u0000\u0000\u0002\u0002\u0000�\u0014\u0006\u0000\u001c(416) 579-0687\u0002\u0000\"Telephone Daytime\u0002\u0000\u0002\u0002\u0002\u0002\u0000\u0006416\u0002\u0000\u0006579\u0002\u0000\b0687\u0002\u0002\u0002\u0000\u0018Valid Number\u0002\u0002\u0002\u0002\u0002\u0002\u0002\u0000\fActive\u0002\u0000\u0001\u0000�\u0017\u0006\u0000\f565332\u0002\u0000\f561015\u0002\u0000�\u0014\u0006\u0000�\u0011\u0000\u0001\r\"��C�h��\u0007l�\u0006~\u000f"

Schema:

{
  type: "record",
  name: "record",
  fields: [
    { name: "meta_id", type: ["long", "double", "string", "null"] },
    { name: "meta_event_type", type: ["string", "null"] },
    { name: "meta_entity", type: ["string", "null"] },
    { name: "meta_layer", type: ["string", "null"] },
    { name: "meta_master_view", type: ["string", "null"] },
    { name: "meta_origin", type: ["string", "null"] },
    { name: "meta_source_system", type: ["string", "null"] },
    { name: "meta_modified_by", type: ["string", "null"] },
    { name: "meta_activation_date", type: ["string", "null"] },
    { name: "meta_creation_date", type: ["string", "null"] },
    { name: "meta_deactivation_date", type: ["string", "null"] },
    { name: "meta_deletion_date", type: ["string", "null"] },
    { name: "meta_last_update_date", type: ["string", "null"] },
    { name: "meta_last_source_update_date", type: ["string", "null"] },
    { name: "meta_source_timestamp", type: ["string", "null"] },
    { name: "meta_activation_tid", type: ["long", "double", "string", "null"] },
    { name: "meta_creation_tid", type: ["long", "double", "string", "null"] },
    { name: "meta_deactivation_tid", type: ["long", "double", "string", "null"] },
    { name: "meta_deletion_tid", type: ["long", "double", "string", "null"] },
    { name: "meta_last_update_tid", type: ["long", "double", "string", "null"] },
    { name: "meta_last_source_update_tid", type: ["long", "double", "string", "null"] },
    { name: "record_eng_active", type: ["boolean", "null"] },
    { name: "old_eng_active", type: ["boolean", "null"] },
    { name: "record_eng_last_update_date", type: ["string", "null"] },
    { name: "old_eng_last_update_date", type: ["string", "null"] },
    { name: "record_eng_creation_date", type: ["string", "null"] },
    { name: "old_eng_creation_date", type: ["string", "null"] },
    { name: "record_eng_deletion_date", type: ["string", "null"] },
    { name: "old_eng_deletion_date", type: ["string", "null"] },
    { name: "record_eng_activation_date", type: ["string", "null"] },
    { name: "old_eng_activation_date", type: ["string", "null"] },
    { name: "record_eng_deactivation_date", type: ["string", "null"] },
    { name: "old_eng_deactivation_date", type: ["string", "null"] },
    { name: "record_eng_nature", type: ["boolean", "null"] },
    { name: "old_eng_nature", type: ["boolean", "null"] },
    { name: "record_eng_modified_by", type: ["string", "null"] },
    { name: "old_eng_modified_by", type: ["string", "null"] },
    { name: "record_party_master_id", type: ["long", "double", "string", "null"] },
    { name: "old_party_master_id", type: ["long", "double", "string", "null"] },
    { name: "record_cmo_phone_number", type: ["string", "null"] },
    { name: "old_cmo_phone_number", type: ["string", "null"] },
    { name: "record_cmo_phone_usage_type", type: ["string", "null"] },
    { name: "old_cmo_phone_usage_type", type: ["string", "null"] },
    { name: "record_cmo_phone_group", type: ["int", "null"] },
    { name: "old_cmo_phone_group", type: ["int", "null"] },
    { name: "record_cmo_country_code", type: ["string", "null"] },
    { name: "old_cmo_country_code", type: ["string", "null"] },
    { name: "record_cmo_area_code", type: ["string", "null"] },
    { name: "old_cmo_area_code", type: ["string", "null"] },
    { name: "record_cmo_exchange", type: ["string", "null"] },
    { name: "old_cmo_exchange", type: ["string", "null"] },
    { name: "record_cmo_local_number", type: ["string", "null"] },
    { name: "old_cmo_local_number", type: ["string", "null"] },
    { name: "record_cmo_extension", type: ["string", "null"] },
    { name: "old_cmo_extension", type: ["string", "null"] },
    { name: "record_cmo_valid_indicator", type: ["string", "null"] },
    { name: "old_cmo_valid_indicator", type: ["string", "null"] },
    { name: "record_cmo_standard", type: ["string", "null"] },
    { name: "old_cmo_standard", type: ["string", "null"] },
    { name: "record_cmo_last_update_dt", type: ["string", "null"] },
    { name: "old_cmo_last_update_dt", type: ["string", "null"] },
    { name: "record_cmo_last_updated_user", type: ["string", "null"] },
    { name: "old_cmo_last_updated_user", type: ["string", "null"] },
    { name: "record_cmo_status", type: ["string", "null"] },
    { name: "old_cmo_status", type: ["string", "null"] },
    { name: "has_event", type: ["boolean", "null"] },
    { name: "record_master_id", type: ["long", "double", "string", "null"] },
    { name: "old_master_id", type: ["long", "double", "string", "null"] },
    { name: "record_source_id", type: ["string", "null"] },
    { name: "old_source_id", type: ["string", "null"] },
    { name: "record_party_source_id", type: ["string", "null"] },
    { name: "old_party_source_id", type: ["string", "null"] },
    { name: "record_party_master_id_i", type: ["long", "double", "string", "null"] },
    { name: "old_party_master_id_i", type: ["long", "double", "string", "null"] },
    { name: "meta_id_i", type: ["long", "double", "string", "null"] },
    { name: "has_event_i", type: ["boolean", "null"] },
  ],
}

Code(doesn't working)

const type = Type.forSchema(schema, {
      wrapUnions: false,
      registry: { long: this.avroLongType() },
    });
    const result = type.fromBuffer(data);

Code(working)

  async decodeFromAvro(data: Buffer, cb: (data: unknown) => Promise<void>): Promise<void> {
    this.logger.debug("Processing avro message", { data });
    const blockDecoder = new streams.BlockDecoder({
      parseHook: (schema) => {
        this.logger.debug("Schema", { schema });
        return Type.forSchema(schema, {
          wrapUnions: false,
          registry: { long: this.avroLongType() },
        });
      },
    });
    blockDecoder.on("data", async (decodeData: unknown) => {
      this.logger.debug("Returning avro message", { decodeData });
      await cb(decodeData);
    });
    blockDecoder.end(data);
  }

The code that is working it's hard to handle a batch messages from kafka. Could you help me?

mtth commented 1 year ago

Hi @rcnespoli. Thanks for the detailed report and apologies for the slow response. The message you received encodes an object container file, this is why you need to use a BlockDecoder to read it. Kafka message typically encode a single object, in which case you would be able to use the simpler approach (trimming the prefix from the buffer first).

If you have control over the publisher, I'd recommend switching to this lighter encoding. It would also reduce the size of messages: right now the schema is included in each one. Otherwise I'm afraid you'll need to stick to the block decoder implementation. Consider also checking if the message contains more than one record ('data' event on the decoder).

rcnespoli commented 1 year ago

Hi, no problem. But my question is how can we create the same message using lib? To decode I'm doing with lib and all thing is working fine. But to create some test scenarios I need create some messages fixtures.