search

vegas-viz / Vegas

The missing MatPlotLib for Scala + Spark
MIT License
730 stars 98 forks source link

Nulls in dataset result in NPE #76

Closed aishfenton closed 8 years ago

aishfenton commented 8 years ago

When loading Movies dataset through Spark, a NPE exception is thrown.

Need to handle null values more gracefully. Since Vegas takes data as a Map[String,Any] we could check if DF cell is null, and then just set value to null (encoder already handles turning this into a Json Null at appropriate time).

More generally withDataFrame should set native values where possible. Currently it turns everything into a Map[String, String].

aishfenton commented 8 years ago

A good dataset to try this on is movies.json, which contains many missing values, including some numeric types.