confluentinc / confluent-kafka-dotnet

Confluent's Apache Kafka .NET client
https://github.com/confluentinc/confluent-kafka-dotnet/wiki
Apache License 2.0
35 stars 854 forks source link

c# .net local key/value deserialization error for client consumer #1498

Open vishwajeets7 opened 3 years ago

vishwajeets7 commented 3 years ago

Description

.net code 3.1 Confluent.kafka 1.5.x Confluent.schemaregistry 1.5.x confluent.schemaregistryserde.avro

using the sample code example from confluent-kafka-dotnet repository to read the kafka topic published through connector.

DB2 to kafka source connector is producing the kafka topic as well the schema registry is used to store the schema definition, sample extracted below: "fields":[{"name":"ID", "type":"int","logicalType":"INTEGER", "dbColumnName":"ID"},"default":0}, ....

with avro deserializer the error thrown is "Local value deserializer error" (same for deserializing key) unable to proceed beyond this point.

When the deserializer is not used i could receive the data with unicode characters appended to the message values with ConsumerBuilder<string, string>,

Example : \u0004PM \u0002N

(note: when the message is received with out deserializer the field order is not consistent, the columns get sorted randomly...believe it is sort by alphabet, not sure though)

please advise.

mhowlett commented 3 years ago

look at the inner exception for more information.

vishwajeets7 commented 3 years ago

Shows deserialization error with stack trace info.. can't place the stack info details here as it is on other machine.

when key deserializer is used, inner exception show " schema 303 not found....RestNotFoundException" with value deserializer is shows "schema 304 not found...RestNotFoundException"

further to add i could as well see the schema at schema url/subjects//versions/latest

vishwajeets7 commented 3 years ago

Do we have any updates on this issue?? this is a widely known issue for .net clients.

anchitj commented 3 months ago

@vishwajeets7 IS this still an issue? If so, please add more details on the exception.