julianpeeters / avrohugger

Generate Scala case class definitions from Avro schemas
Apache License 2.0
201 stars 120 forks source link

SpecificRecord Support for multi record union type #116

Open xuehaowen opened 5 years ago

xuehaowen commented 5 years ago

I need to deserialize avro message which implemented SpecificRecord. The schema I used is like: union { record1, record2, record3} event;

But I find SpecificRecord does not support OptionShapelessCoproduct, OptionEitherShapelessCoproduct, or OptionalShapelessCoproduct.

val avroScalaCustomTypes = SpecificRecord.defaultTypes.copy(union = OptionEitherShapelessCoproduct) val generator = new Generator(SpecificRecord, restrictedFieldNumber = true, avroScalaCustomTypes = Some(avroScalaCustomTypes))

java.lang.RuntimeException: Unions beyond nullable fields are not supported at scala.sys.package$.error(package.scala:27) at avrohugger.format.specific.converters.JavaConverter$.convertToJava(JavaConverter.scala:35) at avrohugger.format.specific.methods.GetGenerator$.avrohugger$format$specific$methods$GetGenerator$$asGetCase$1(GetGenerator.scala:23)

Can we add support for specific record

Dr-Nikson commented 5 years ago

👍🏼 +1. Same issue.

Would it make sense to support this kind of unions? Because it doesn't work event for two types inside union:

{
  "name": "myField",
  "type": ["MyEvent1", "MyEvent2"]
}
nicolaemarasoiu commented 4 years ago
[error] stack trace is suppressed; run last Compile / avroScalaGenerate for the full output
[error] (Compile / managedSources) spray.json.DeserializationException: Object expected in field 'name'
[error] (Compile / avroScalaGenerate) Unions, beyond nullable fields, are not supported. 
[error] Found a union of more than one type

Top level in this avsc file is not a single type like usual, but 4 types are defined inside an array: https://github.com/julianpeeters/avrohugger/issues/24

If the types are independent, it is not an issue to split the file in multiple avsc files.

But if there are dependencies, i do not know currently any way in Avro hugger to ensure that the avsc files are parsed in their dependency order: https://github.com/julianpeeters/avrohugger/issues/137