search

Apicurio / apicurio-registry

An API/Schema registry - stores APIs and Schemas.
https://www.apicur.io/registry/
Apache License 2.0
554 stars 250 forks source link

Having trouble using: Legacy4ByteIdHandler #4830

Open reidmeyer opened 6 days ago

reidmeyer commented 6 days ago

Description

Registry Version: 2.5.9.Final Persistence type: in-memory

I'm trying to use the Legacy4ByteIdHandler as an id-handler in my kafka connect connector, as described in: https://www.apicur.io/registry/docs/apicurio-registry/2.5.x/getting-started/assembly-configuring-kafka-client-serdes.html.

I add the apicurio plugin to my kafka connect with:

# apicurio-registry-distro-connect-converter
# https://mvnrepository.com/artifact/io.apicurio/apicurio-registry-distro-connect-converter
RUN wget --no-verbose --output-document=apicurio-converter.tar.gz \
        https://repo1.maven.org/maven2/io/apicurio/apicurio-registry-distro-connect-converter/2.6.0.Final/apicurio-registry-distro-connect-converter-2.6.0.Final.tar.gz \
    && mkdir -p /opt/kafka/plugins/apicurio-converter/ \
    && tar -x -f apicurio-converter.tar.gz -C /opt/kafka/plugins/apicurio-converter/ \
    && rm -f apicurio-converter.tar.gz

and most things work, but there is a classpath issue or something around the Legacy4ByteIdHandler. First I get a cannot find class exception on the Legacy4ByteIdHandler.

Next, I try to add the classpath of the serde common manually (where the byte handler is) with: ENV CLASSPATH=/opt/kafka/plugins/apicurio-converter/apicurio-registry-serde-common-2.6.0.Final.jar:$CLASSPATH and i get a new error about missing class ArtifactReference, so I also add to the classpath: ENV CLASSPATH=/opt/kafka/plugins/apicurio-converter/apicurio-registry-schema-resolver-2.6.0.Final.jar:$CLASSPATH

Then I'm stuck on error: java.lang.ClassCastException: class io.apicurio.registry.serde.Legacy4ByteIdHandler cannot be cast to class io.apicurio.registry.utils.serde.strategy.IdHandler (io.apicurio.registry.serde.Legacy4ByteIdHandler is in unnamed module of loader 'app'; io.apicurio.registry.utils.serde.strategy.IdHandler is in unnamed module of loader org.apache.kafka.connect.runtime.isolation.PluginClassLoader @4b4dd216)

Perhaps I have a misunderstanding of the way the classpath works; hoping someone can assist.

PS: I technically have 4-byte effectively working by using the as-confluent flag, but I'd like to be able to use the Legacy4ByteIdHandler itself.

Environment

Kubernetes v1.26.15

Using kafka connect, aiven postgres sink.

Steps to Reproduce

  1. Have kafka connect
  2. Create a connector with config that uses io.apicurio.registry.utils.converter.AvroConverter.
  3. Try to use value.converter.apicurio.registry.id-handler: io.apicurio.registry.serde.Legacy4ByteIdHandler

Expected vs Actual Behaviour

I expect kafka connect to be able to find the Legacy4ByteIdHandler.