search

modelop / hadrian

Implementations of the Portable Format for Analytics (PFA)
Apache License 2.0
130 stars 49 forks source link

Aurelius: avro_from_df should handle case of missing values #53

Open steadyfish opened 6 years ago

steadyfish commented 6 years ago

Currently, avro_from_df function just takes data type of the column and converts it to corresponding PFA type. It'd help if this function handles the case where a column has some missing values.

Minimal reproducible example:

a = data.frame(col1 = c(1, 2), col2 = c(1.234565, NA))
avro_from_df(a)

I'd expect col2 to be of type avro_union(avro_double, avro_null) and not avro_double.