dev services for confluent schema registry

[Dieken]

Description

Following https://quarkus.io/guides/kafka-schema-registry-avro, Quarkus automatically starts docker containers for Kafka and Apicurio registry, but won't for Confluent schema registry.

Confluent has published an official docker image cp-schema-registry but unluckily it's too big.

BTW, when use Confluent schema registry, quarkus doesn't automatically include avro-maven-plugin, I have to explicitly configure it in pom.xml, see https://github.com/quarkusio/quarkus/issues/19276. (Resolved by adding extension quarkus-avro)

Implementation ideas

Confluent schema registry is written in pure Java, maybe Quarkus can just start a Java process for the registry service.

https://github.com/confluentinc/schema-registry

[quarkus-bot[bot]]

/cc @stuartwdouglas

[Ladicek]

CC @cescoffier @ozangunalp

[cescoffier]

I would stay away from running it in Java because of port conflicts and classloading issues. The lack of a slim and supported image is annoying. Doing one ourselves would require a non-meaningless maintenance cost.

[Ladicek]

Also the Confluent Schema Registry code, except of the client libraries, is licensed under Confluent Community License, which doesn't seem to be an open source license. So there's a risk there, in my opinion (which doesn't weigh much, as I'm not a lawyer :-) ).

[lw-mcno]

Maybe for some using the Confluent-compatible endpoints of Apicurio registry would be enough. Here's one working configuration:

  quarkus:
    apicurio-registry:
      devservices:
        port: 50501

    kafka-streams:
      schema-registry-url: 'http://localhost:${quarkus.apicurio-registry.devservices.port}/apis/ccompat/v6'

[msillence]

Maybe for some using the Confluent-compatible endpoints of Apicurio registry would be enough. Here's one working configuration:

  quarkus:
    apicurio-registry:
      devservices:
        port: 50501

    kafka-streams:
      schema-registry-url: 'http://localhost:${quarkus.apicurio-registry.devservices.port}/apis/ccompat/v6'

Please could you add this to the guide?: https://quarkus.io/guides/kafka-schema-registry-avro

[ozangunalp]

Schema registry dev services inject config to work with reactive messaging kafka connector, ie. mp.messaging.connector.smallrye-kafka.schema.registry.url and mp.messaging.connector.smallrye-kafka.apicurio.registry.url.

We can also inject the correct url to the schema-registry-url depending on the quarkus.kafka-streams.schema-registry-key config. Meanwhile, this can be the correct workaround : quarkus.kafka-streams.schema-registry-url=${mp.messaging.connector.smallrye-kafka.schema.registry.url}