Open samhwang opened 2 years ago
@Nevon We're having the same issue when consuming a topic with a schema without top level record.
For example this code:
// this fails!
registry.decode(buffer, {
[SchemaType.AVRO]: {readerSchema}, // We can't use a RawAvroSchema type without a Record
})
But if we use the schema id thats included in the Kafka message it works.
// this works!
registry.decode(buffer)
On the other hand the avsc lib allows this, we can check it's typing:
export type AvroSchema = DefinedType | DefinedType[]; // Note the array here
type DefinedType = PrimitiveType | ComplexType | LogicalType | string;
type PrimitiveType = 'null' | 'boolean' | 'int' | 'long' | 'float' | 'double' | 'bytes' | 'string';
type ComplexType = NamedType | RecordType | EnumType | MapType | ArrayType | FixedType;
type LogicalType = ComplexType & LogicalTypeExtension;
Hey KafkaJS Team, just wanted to started that this project is superb. However, I just came into this weird hiccup, which I think shouldn't be. I'm still new to how Kafka and Avro (and KafkaJS) works, so if I'm wrong, please lead me in the right direction :D.
Issue Description
I'm trying to read an avro file that does not have a top level record, but instead it has an array of records.
This is validated to be correct, both by a locally hosted Kafka Schema Registry (using the docker image![Screen Shot 2021-11-12 at 2 18 05 pm](https://user-images.githubusercontent.com/4188758/141403401-f99da710-0234-409c-b606-d060a6602675.png)
confluentinc/cp-schema-registry:6.2.1
) and on Confluent Cloud (see screenshot attached).Reproduction steps/repo
I have created a sample repo in Typescript to reproduce this issue at: samhwang/test-node-kafka.
Reproduction steps
Install the reproduction repo
Register schema via a http request to schema registry. This works fine.
This script is essentially:
@kafkajs/confluent-schema-registry
. This just breaks at just reading theavsc
file.Possible solution?
validatedSchema
function insrc/utils/readAVSC.ts
expecting a top level record. Is there something in theavsc
that can be used to validate instead of this?