CSVReader: Removed tablesaw dep and replaced with apache commons for csv reading

trustyai-explainability / trustyai-service

TrustyAI service providing cloud-native fairness metrics and explainability

Apache License 2.0

1 stars 3 forks source link

CSVReader: Removed tablesaw dep and replaced with apache commons for csv reading #5

Closed RobGeada closed 1 year ago

RobGeada commented 1 year ago

Note: this fails the README workflow, will investigate asap

ruivieira commented 1 year ago

@RobGeada I've added a (MinIO related) fix to this PR, otherwise this would not work: it now reads the data as a String from all the MinIO transferred bytes and corrects the header not being detected.

The column type was also checked for every row, in here we assume that the type is the same within the column and only check it once, reducing overhead.