search

jayunit100 / bigpetstore

[OBSOLETE] Moved to Apache BigTop !!! See you there.
http://jayunit100.github.io/bigpetstore/slides
9 stars 3 forks source link

Refactor all ETL's to write to Disk in tests #15

Closed jayunit100 closed 10 years ago

jayunit100 commented 10 years ago

The ETL class is Pig. It will ultimately clean data for hive to query. It should just write to disk in all cases.

jayunit100 commented 10 years ago

Ive just commited a diagram of the flow, to make it linear. now no redundancy between hive/pig. no need for overdoing the comparison stuff. this will make it easier to code for us. We will have hive/pig work together, rather than compete :)

jayunit100 commented 10 years ago

http://bit.ly/1bnzYj1 <-- theres the image of the new architecture