How to ignore malformed-records while using from_avro in abris

[willianmrs]

Hello, we use abris to deserialize Avro from a confluent schema-registry. But sometimes we get some records with errors that we want to ignore, how can we do that?

I was looking into withExceptionHandler, but it needs a default record with a matching schema, which is something we do not exactly have... It is possible just to ignore those records?

[cerveada]

I am not sure if it's possible, since Abris implements Spark's UnaryExpression that always expects output.

DeserializationExceptionHandler actually gives you the reader schema. Could you create the default record dynamically based on that? You can later filter out these default rows.

[willianmrs]

PermissiveRecordExceptionHandler implement. I'm closing the issue, thanks everyone for the help.