juanrh / sscheck

ScalaCheck for Spark
Apache License 2.0
63 stars 9 forks source link

New generators from avro files #1 #40

Closed mtelloz closed 8 years ago

mtelloz commented 8 years ago

Implement a ScalaCheck generator from a file in the local file system that contains the test cases. We use Avro as the serialization format for the test cases in order to store the test cases together with a its schema, and because this serialization format offers a rich data model.

Milestone implemented using avro reflection and generating an automatic case-class on the project. A twitter schema is being used as example. The avro file was geerated using the Scalavro library which let us serialize data BUT without the capability of attaching the scheme in the binary file, that's why the generator is not generic in this milestone