Also added the column common functions implicit class
The extractors will allow something that I find interesting, and right now its hard to do in spark, to diverge the logic by the type of the column, an example will clarify more:
def transformDateOrTimestamp(colName: String)(df:DataFrame): IntegerColumn =
df(colName) match {
case DateColumn(dc) => dc.day_of_month
case TimestampColumn(tc) => tc.day_of_month
case _ => IntegerColumn(0)
}
This combines the power of the pattern matching and custom extractors to describe easily.
The implicit class allows to add common logic to the typed columns, like the method "as" and when the Boolean type is added, type safety comparations of columns 😄
Also added the column common functions implicit class
The extractors will allow something that I find interesting, and right now its hard to do in spark, to diverge the logic by the type of the column, an example will clarify more:
This combines the power of the pattern matching and custom extractors to describe easily.
The implicit class allows to add common logic to the typed columns, like the method "as" and when the Boolean type is added, type safety comparations of columns 😄