KafkaProtobufSerializer does not URL-encode subject names during schema registration [Java]

[capitalg]

I have a protobuf file that imports "google/protobuf/timestamp.proto". When my kafka producer, using KafkaProtobufSerializer, tries to register the protobuf file, it fails with an exception. From the schema registry logs, I see that KafkaProtobufSerializer is sending a POST request that looks like this:

"POST /subjects/google/protobuf/timestamp.proto/versions HTTP/1.1" 404 49 11

and that fails with javax.ws.rs.NotFoundException: HTTP 404 Not Found.

A successful schema registration request sent via curl against the Rest API of the registry looks like this:

"POST /subjects/google%2Fprotobuf%2Ftimestamp.proto/versions HTTP/1.1" 200 8 23

and that succeeds.

This leads me to believe that KafkaProtobufSerializer is not accurately URL-encoding the default ReferenceSubjectNameStrategy used for imports with /s. I also cannot find a way to change the ReferenceSubjectNameStrategy for my producer as a potential way of mitigating this issue. My kafka producer is configured as follows:

 ...
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, KafkaProtobufSerializer.class);
...
producer = new org.apache.kafka.clients.producer.KafkaProducer<>(props);

It seems that .NET had a similar issue which was eventually resolved: https://github.com/confluentinc/confluent-kafka-dotnet/issues/1258

I see comments there saying that Java does not have this issue, but I am running into the same issue with Java.

[rayokota]

@capitalg , which version of CP are you using?

[capitalg]

The schema registry server is using CP version 5.5.0. The schema registry Java client is version 6.1.1.

[capitalg]

I disabled auto.register.schemas in the Producer and registered google.protobuf.Timestamp manually via the Rest API, but now the KafkaProtobufSerializer fails the GET request for the same reason (lack of URL-encoding).

Here is the GET request sent by KafkaProtobufSerializer: "GET /subjects/google/protobuf/timestamp.proto/versions/latest HTTP/1.1" 404 49 11

It also fails with the same error:

[2021-05-17 18:15:20,619] ERROR Request Failed with exception  (io.confluent.rest.exceptions.DebuggableExceptionMapper:62)
javax.ws.rs.NotFoundException: HTTP 404 Not Found

Here is a successful one I sent via curl: "GET /subjects/google%2Fprotobuf%2Ftimestamp.proto/versions/1 HTTP/1.1" 200 556 3

[rayokota]

Can you try with CP version 5.5.4 or later?

[capitalg]

Just tested with 6.1.1 for schema registry server, and it works. It seemed like a malformed request and therefore a client issue, but I guess somehow the registry server was rejecting the request.

Thanks for the help!