More stats reporting. - Githubissues

Shopify / camus

Kafka->HDFS pipeline from LInkedIn. It is a mapreduce job that does distributed data loads out of Kafka.

7 stars 4 forks source link

More stats reporting. #99

Closed olessia closed 6 years ago

olessia commented 6 years ago

Report the case when the timestamp was not found/parsed. This commit may temporarily increase the logging, but once we validate what's happening we can take it and the other spammy logging out.
Submit more data to statsd:
- job start
- run-time stats
- pull-max-time-reached: submit 0 when the partition was fully pulled (this should help with alerting)

dterror-zz commented 6 years ago

datwright commented 6 years ago

ack sorry! I pushed master to the staging branch to run camus staging, I didn't realize there was a PR on this branch, sorry @olessia! I guess that auto-closes the PR.

You'll have to re-push your changes to staging