Shopify / camus

Kafka->HDFS pipeline from LInkedIn. It is a mapreduce job that does distributed data loads out of Kafka.
7 stars 4 forks source link

Camus staging #94

Closed dterror-zz closed 6 years ago

dterror-zz commented 6 years ago

Quick and Dirty Camus Staging ™️

why multiple targets: In a distant past, we've had to run two Camus in parallel, that's because we had to migrate from reading from one cluster to another and the offsets were all different, so we added support to running multiple Camus in the same 'environment'.

c.c @datwright