Clean up/re-architecting of dg-spark

FINRAOS / DataGenerator

DataGenerator is a Java library for systematically producing large volumes of data. DataGenerator frames data production as a modeling problem, with a user providing a model of dependencies among variables and the library traversing the model to produce relevant data sets.

http://finraos.github.io/DataGenerator

Apache License 2.0

161 stars 170 forks source link

Clean up/re-architecting of dg-spark #278

Closed mpeter28 closed 8 years ago

mpeter28 commented 8 years ago

Derived from the code in Brijeshrpatel9:master

dg-spark has been folded into dg-core
SparkDistributor has been trimmed down and changed in functionality
Frontier now uses the Strategy pattern completely
CustomTagExtension extends Serializable so individual tags will not need to
DataConsumer picked up an extra method called 'transformAndReturn', which is 'consume' sans writers

mpeter28 commented 8 years ago

Travis is failing the build because of the HDFS Distributor example, which this pull request doesn't touch. Anyone know what's up with HDFS Distributor example?