AbsaOSS / spark-commons

Apache License 2.0
7 stars 0 forks source link

Add methods to the `trait ErrorHandling` that will return the type/schema of the generated `ErrorColumn` and their aggregation. #91

Closed benedeki closed 1 year ago

benedeki commented 1 year ago

Background

There's a lack of information for the caller about the actual implementation of the trait ErrorHandling. The information might be useful for whatever reason - call optimization etc.

Feature

Add methods to the trait ErrorHandling that will return the DataType of the returned ErrorColumn and of the aggregation (that should be probably Option[DataType] in case the aggregation actually does not add a column.

Advanced: Try to find a solution that class ErrorColumn would be able to return the attached Column type too (while keeping it still a value-class).