search

gisaia / ARLAS-proc

Workaround about data ingestion with computing frameworks
Apache License 2.0
4 stars 0 forks source link

Simplify DataModel objects #124

Closed laurent-thiebaud-gisaia closed 5 years ago

laurent-thiebaud-gisaia commented 5 years ago

DataModel may only contain mandatory columns

Fix #116

I added a check of Datamodel columns in DataFrameFormatter, I know we're not supposed to do validation in a Formatter. We could do the validation in a dedicated transformer, but I don't see the gain to make it standalone by now. Or we could rename the Transformer with something like DataFramePreparer but I can't find anything smart. If you have a great idea...

laurent-thiebaud-gisaia commented 5 years ago

Fixed tests names and rebased