Be less strict about nullable <--> non-nullable type conversions.

AbsaOSS / ABRiS

Avro SerDe for Apache Spark structured APIs.

Apache License 2.0

229 stars 75 forks source link

Closed cerveada closed 2 years ago

cerveada commented 4 years ago

It makes sense to allow users to convert non-nullable type to nullable one.
spark-avro even allows to convert nullable to non-nulable and fails only when null value is present. https://issues.apache.org/jira/browse/SPARK-27838

JonDegn commented 3 years ago

I agree. This is definitely a pain point for us.

cerveada commented 2 years ago

This is solved in Abris 5 by using spark-avro for conversions. When using Spark Avro 3+ it should work. Check the documentation for more info.