TresAmigosSD / SMV

Spark Modularized View
Apache License 2.0
42 stars 22 forks source link

Create SmvInMemModule and InMemIoStrategy #1490

Open ninjapapa opened 5 years ago

ninjapapa commented 5 years ago

We need a SmvGenericModule sub class which handles very light weight data output (basically Python object, without need to persist on disk). For that type of module, need an InMemIoStrategy which "persist" data as cache within the strategy object.