Schema validation

[mrkannan3]

Description

producer is java and consumer is .NET

I would like to write a common consumer code with schema validation. The idea is to use one base-class (kafkaMessage) (through DI or partial class; we will add actual fields for each consumer) so the consumer code is reusable and no needs to change for other topics (every topics have their own fields).

I am now getting the below error. I would like to know if schema validation can skip the validation for name & namespace but do the validation for fields.

Getting mismatch validation error Avro.AvroException: Schema mismatch. Reader: {"type":"record","name":"### KafkaMessage","namespace":"### KafkaService","fields": [{"name":"empname","type":"string"},{"name":"empadd","type":"string"} {"name":"dob","type":"string"}]} writer: {"type":"record","name":"### Employee","namespace":"### com.example","fields": [{"name":"empname","type":"string"},{"name":"empadd","type":"string"} {"name":"dob","type":"string"}]}

How to reproduce

If producer's class name and package name are different than consumer's class and namespace then this error is thrown.

Please note that I am now writing both producer and consumer in .NET to test out the solution but eventually producer is going to be java.

Checklist

Please provide the following information:

• [x] Confluent.Kafka nuget version: 1.0 beta2
• [ ] Apache Kafka version:
• [ ] Client configuration:
• [ ] Operating system:
• [ ] Provide logs (with "debug" : "..." as necessary in configuration)
• [ ] Provide broker log excerpts
• [ ] Critical issue

[lzerma]

Did you solve this @mrkannan3 ?

[TiagoBrenck]

Does anyone have a solution for this? I am also using .NET Core consumer, and Java producer. The Java producer has this schema

{\"type\":\"record\",\"name\":\"common\"...}

A class called common is quite annoying so I tried to create a .avsc with a better class name, but the validation rejects it due to name mismatch.

It would be awesome to have a config to ignore namespace and name validation.

[TiagoBrenck]

@mrkannan3 @lzerma just found the solution.

After digging deep in the Avro source code, I found this: https://github.com/apache/avro/blob/master/lang/csharp/src/apache/main/Schema/RecordSchema.cs#L311

So there is a property called aliases. If you set it in your .avsc the validation works. Example of my .avsc to fix common class name:

{
"type": "record",
  "name": "MyCustomClassNameThatDoesntMatch",
  "aliases": [ "common" ],
  "namespace": "foo.barr",
....
...
...
}