Break into profiles - Githubissues

jayunit100 / bigpetstore

[OBSOLETE] Moved to Apache BigTop !!! See you there.

http://jayunit100.github.io/bigpetstore/slides

9 stars 3 forks source link

Break into profiles #13

Closed jayunit100 closed 10 years ago

jayunit100 commented 10 years ago

Problem: Pig/Hive Crunch deps are fundamentally incompatabile, or at least not expected / gauranteed to work together.

Solution : Run each job as a separate maven profile.

New problem : All the job results are created in DIFFERENT JVMs. So, we have to output a plain text file, from each - and ... in a final build step, compare that all outputs are equal.

jayunit100 commented 10 years ago

upgrade to exact bigtop deps (hadoop 2.0 ... etc... )