Closed laurent-thiebaud-gisaia closed 5 years ago
By the way, some configuration may also be put in customer specific project with the wrappers, like tempo configurations
We stated that
DataFrameValidator should be renamed to DataFrameCleaner
I prefer to rename it to DataFrameFormatter
as it doesn't clean DF, it formats it to a spark-compliant format.
As a consequence, the wrapper asArlasFormattedData
is made of DataFrameFormatter
, WithArlasTimestamp
and WithArlasPartition
. It may sound strange?
In
TransformableDataFrame
, wrappers:asArlasCleanedData should be renamed to asArlasFormattedData ; by the wy the transformer
DataFrameValidator
should be renamed to DataFrameCleanerasArlasMotions
should contain all transformers to get the motion columns ; it should be put in the customer specific projectasArlasCourses
should be put in the customer specific projectasArlasMovingState
;asArlasVisibleSequencesThroughTempo
,asArlasBasicData
should be removedMaybe
asArlasResampledMotions
isn't necessary anymore?