search

kafkajs / confluent-schema-registry

is a library that makes it easier to interact with the Confluent schema registry
https://www.npmjs.com/package/@kafkajs/confluent-schema-registry
MIT License
157 stars 102 forks source link

Unable to register schemas with canonical forms that reduce to a primitive type #157

Open BryceCicada opened 3 years ago

BryceCicada commented 3 years ago

I'd like to use a schema-registry avro schema to encode a kafka message key. Most of the time, the key is pretty simple like a string or a long. Other times, it's more complex and I can use a record type.

When the key isn't a record, but it's just a primitive type, I'm struggling to register the schema. For example:

      const schemaRegistry = new SchemaRegistry({host: "http://my-registry:8081"})
      await schemaRegistry.register({type: SchemaType.AVRO, schema: '"string"'})

gives:

     ConfluentSchemaRegistryArgumentError: Invalid name: undefined
      at AvroHelper.validate (node_modules/@kafkajs/confluent-schema-registry/src/AvroHelper.ts:26:13)
      at SchemaRegistry.register (node_modules/@kafkajs/confluent-schema-registry/src/SchemaRegistry.ts:106:12)
      at Context.<anonymous> (test/service-bus/serde_test.ts:45:28)
      at processImmediate (internal/timers.js:461:21)

I've also tried:

      await schemaRegistry.register({type: SchemaType.AVRO, schema: JSON.stringify({type: "string"})})
      await schemaRegistry.register({type: SchemaType.AVRO, schema: JSON.stringify({name: "foo", type: "string"})})

which have schemas that reduce to the same avro canonical form as the first example, but these give the same error.

I can register the schemas with the registry directly, eg:

$ curl -X POST localhost:8081/subjects/my-topic-key/versions -H "Content-Type: application/json" -d '{"schema": "\"string\""}'
{"id":1}
$ curl -X POST localhost:8081/subjects/my-topic-key/versions -H "Content-Type: application/json" -d '{"schema": "{\"type\": \"string\"}"}'  # NB Same canonical form -> same id from registry
{"id":1} 
$ curl -X POST localhost:8081/subjects/my-topic-key/versions -H "Content-Type: application/json" -d '{"schema": "{\"name\": \"foo\", \"type\": \"string\"}"}' # NB Same canonical form again
{"id":1} 
$ curl localhost:8081/schemas/ids/1
{"schema":"\"string\""}

which makes me think that AvroHelper.validate() might be a bit too strict in requiring a name field on the schema.

BryceCicada commented 3 years ago

Commenting on my own issue here as I understand more...

I suspect the reason for requiring a name on the schema is because of the implicit subject naming that uses the name field from the schema. Would it make sense to relax this constraint in AvroHelper.validate() if the subject name were passed in to register()?

filipstiglic commented 1 year ago

Hello, any update on this?

rogen-code commented 8 months ago

Hit this today and would love to know if anyone has a workaround.