julianpeeters / avrohugger

Generate Scala case class definitions from Avro schemas
Apache License 2.0
202 stars 120 forks source link

Expected a single top-level record, found a union of more than one type #24

Open thoughtpoet opened 9 years ago

thoughtpoet commented 9 years ago

Having an enum with schemas seems to not be supported. The example below is arbitrary, and could be worked around, but it's a recurring pattern.

[ { "namespace": "bom.aaa.types", "type": "record", "name": "EmailFactor", "fields": [ { "name": "email", "type": "string" } ] }, { "namespace": "bom.aaa.types", "type": "record", "name": "PhoneFactor", "fields": [ { "name": "phone", "type": "string" } ] }, { "namespace": "bom.aaa.types", "type": "record", "name": "TwoFactorAuthentication", "fields": [ { "name": "factor", "type": ["bom.aaa.types.EmailFactor", "bom.aaa.types.PhoneFactor"] }, { "name": "tokenId", "type": "string" } ] } ]

In this case, the following error results: fpatton:aaa fpatton$ sbt avro:generate [info] Loading global plugins from /Users/fpatton/.sbt/0.13/plugins [info] Loading project definition from /Users/fpatton/src/arena/git/aaa/project [info] Set current project to aaa (in build file:/Users/fpatton/src/arena/git/aaa/) [info] Compiling AVSC /Users/fpatton/src/arena/git/aaa/src/main/avro/bom/aaa/types/GeneralContext.avsc [info] Compiling AVSC /Users/fpatton/src/arena/git/aaa/src/main/avro/bom/aaa/types/UserLogEntity.avsc [info] Compiling AVSC /Users/fpatton/src/arena/git/aaa/src/main/avro/bom/aaa/types/User.avsc [info] Compiling AVSC /Users/fpatton/src/arena/git/aaa/src/main/avro/bom/aaa/types/TwoFactorAuthentication.avsc java.lang.RuntimeException: Expected a single top-level record, found a union of more than one type: List({"type":"record","name":"EmailFactor","namespace":"bom.aaa.types","fields":[{"name":"email","type":"string"}]}, {"type":"record","name":"PhoneFactor","namespace":"bom.aaa.types","fields":[{"name":"phone","type":"string"}]}, {"type":"record","name":"TwoFactorAuthentication","namespace":"bom.aaa.types","fields":[{"name":"factor","type":[{"type":"record","name":"EmailFactor","fields":[{"name":"email","type":"string"}]},{"type":"record","name":"PhoneFactor","fields":[{"name":"phone","type":"string"}]}]},{"name":"tokenId","type":"string"}]}) at scala.sys.package$.error(package.scala:27) at avrohugger.input.parsers.FileInputParser$$anonfun$getSchemas$1.apply(FileInputParser.scala:37) at avrohugger.input.parsers.FileInputParser$$anonfun$getSchemas$1.apply(FileInputParser.scala:32) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) at scala.collection.immutable.List.foreach(List.scala:318) at scala.collection.TraversableLike$class.map(TraversableLike.scala:244) at scala.collection.AbstractTraversable.map(Traversable.scala:105) at avrohugger.input.parsers.FileInputParser.getSchemas(FileInputParser.scala:32) at avrohugger.Generator.fileToFile(Generator.scala:53) at sbtavrohugger.FileWriter$$anonfun$generateCaseClasses$2.apply(FileWriter.scala:24) at sbtavrohugger.FileWriter$$anonfun$generateCaseClasses$2.apply(FileWriter.scala:22) at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:60) at scala.collection.mutable.MutableList.foreach(MutableList.scala:30) at sbtavrohugger.FileWriter$.generateCaseClasses(FileWriter.scala:22) at sbtavrohugger.formats.standard.GeneratorTask$$anonfun$caseClassGeneratorTask$1$$anonfun$1.apply(GeneratorTask.scala:36) at sbtavrohugger.formats.standard.GeneratorTask$$anonfun$caseClassGeneratorTask$1$$anonfun$1.apply(GeneratorTask.scala:34) at sbt.FileFunction$$anonfun$cached$1.apply(Tracked.scala:235) at sbt.FileFunction$$anonfun$cached$1.apply(Tracked.scala:235) at sbt.FileFunction$$anonfun$cached$2$$anonfun$apply$3$$anonfun$apply$4.apply(Tracked.scala:249) at sbt.FileFunction$$anonfun$cached$2$$anonfun$apply$3$$anonfun$apply$4.apply(Tracked.scala:245) at sbt.Difference.apply(Tracked.scala:224) at sbt.Difference.apply(Tracked.scala:206) at sbt.FileFunction$$anonfun$cached$2$$anonfun$apply$3.apply(Tracked.scala:245) at sbt.FileFunction$$anonfun$cached$2$$anonfun$apply$3.apply(Tracked.scala:244) at sbt.Difference.apply(Tracked.scala:224) at sbt.Difference.apply(Tracked.scala:200) at sbt.FileFunction$$anonfun$cached$2.apply(Tracked.scala:244) at sbt.FileFunction$$anonfun$cached$2.apply(Tracked.scala:242) at sbtavrohugger.formats.standard.GeneratorTask$$anonfun$caseClassGeneratorTask$1.apply(GeneratorTask.scala:38) at sbtavrohugger.formats.standard.GeneratorTask$$anonfun$caseClassGeneratorTask$1.apply(GeneratorTask.scala:31) at scala.Function6$$anonfun$tupled$1.apply(Function6.scala:35) at scala.Function6$$anonfun$tupled$1.apply(Function6.scala:34) at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47) at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40) at sbt.std.Transform$$anon$4.work(System.scala:63) at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226) at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226) at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17) at sbt.Execute.work(Execute.scala:235) at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226) at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226) at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159) at sbt.CompletionService$$anon$2.call(CompletionService.scala:28) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) error Expected a single top-level record, found a union of more than one type: List({"type":"record","name":"EmailFactor","namespace":"bom.aaa.types","fields":[{"name":"email","type":"string"}]}, {"type":"record","name":"PhoneFactor","namespace":"bom.aaa.types","fields":[{"name":"phone","type":"string"}]}, {"type":"record","name":"TwoFactorAuthentication","namespace":"bom.aaa.types","fields":[{"name":"factor","type":[{"type":"record","name":"EmailFactor","fields":[{"name":"email","type":"string"}]},{"type":"record","name":"PhoneFactor","fields":[{"name":"phone","type":"string"}]}]},{"name":"tokenId","type":"string"}]}) [error] Total time: 0 s, completed Nov 17, 2015 4:31:33 PM

julianpeeters commented 9 years ago

I don't see an enum in your schema, so please correct me if I'm missing the mark here. Instead I see a union of records (the type of the TwoFactorAuthentication record is [EmailFactor, PhoneFactor]), and unions of this type are not are not supported. "Optional" unions are supported (e.g.[null, EmailFactor]) because they map cleanly on to Scala's Option type, but AFAIK there is as of now no clean way to support true unions in Scala. It looks like Dotty will bring true unions to Scala, and although that is sometime off, the tentative plan is to add support then.

So, apologies that this is not supported, and I will make the error message a little clearer.

thoughtpoet commented 9 years ago

Thanks, Julian. Yes, I meant unions. I've been using them quite heavily to model constraints where fields go from optional to required to absent based on other fields. Thanks for the great repos, they will be very useful. --fred On Nov 17, 2015 7:13 PM, "Julian Peeters" notifications@github.com wrote:

I don't see an enum in your schema, so please correct me if I'm missing the mark here. Instead I see a union of records (the type of the TwoFactorAuthentication record is [EmailFactor, PhoneFactor]), and unions of this type are not are not supported. "Optional" unions are supported (e.g.[null, EmailFactor]) because they map cleanly on to Scala's Option type, but AFAIK there is as of now no clean way to support true unions in Scala. It looks like Dotty will bring true unions to Scala, and although that is sometime off, the tentative plan is to add support then.

So, apologies that this is not supported, and I will make the error message a little clearer.

— Reply to this email directly or view it on GitHub https://github.com/julianpeeters/avrohugger/issues/24#issuecomment-157589271 .

nicolaemarasoiu commented 4 years ago

If the types are independent, it is not an issue to split the file in multiple avsc files.

But if there are dependencies, i do not know currently any way in Avro hugger to ensure that the avsc files are parsed in their dependency order: #137