AbsaOSS / spark-commons

Apache License 2.0
7 stars 0 forks source link

Introducing Map for error columns and their values #85

Closed benedeki closed 1 year ago

benedeki commented 1 year ago

After the presentation, discussions with others and more thoughts, I came to the idea to combine the errCol (which would be better as sequence anyway) and rawValues into one column of map data type. That resulted in a need to do some more exercise in the supporting types, but I believe to for the better.

Also some general code improvements were added as I stumbled upon stuff during the refactoring.

benedeki commented 1 year ago

Moving the the main PR, general direction approved in direct discussions.