search

Netflix / PigPen

Map-Reduce for Clojure
Apache License 2.0
565 stars 55 forks source link

Make functional tests a library #92

Closed mbossenbroek closed 9 years ago

mbossenbroek commented 9 years ago

Convert functional tests so that they can be run from any platform. The tests are defined slightly differently, such that a TestHarness instance provides the functionality of creating mock data, reading and writing files, and executing commands.

Implementors of new platforms would create something similar to pigpen.pig.functional-test. I'll be adding a version that generates a pig script, runs it using pig, and reads the output.

@pkozikow - Let me know if you think this will work for cascading.

cc @daveray @mapstrchakra

pkozikow commented 9 years ago

It looks like it should work. The only problem I see is that some tests using text-based loaders such as pig/load-tsv directly will break since cascading requires a custom version of load-text.