ParserDailyVolumeTooLow

[measurementlab]

Alertmanager URL: http://status.mlab-oti.measurementlab.net:9093

• firing http://status.mlab-oti.measurementlab.net:9090/graph?g0.expr=sum%28increase%28etl_test_count%7Bservice%21~%22.%2Abatch.%2A%22%7D%5B1d%5D%29%29+BY+%28service%29+%3C+%280.7+%2A+quantile%280.5%2C+label_replace%28sum%28increase%28etl_test_count%7Bservice%21~%22.%2Abatch.%2A%22%7D%5B1d%5D+OFFSET+1d%29%29+BY+%28service%29%2C+%22delay%22%2C+%221d%22%2C+%22%22%2C+%22.%2A%22%29+or+label_replace%28sum%28increase%28etl_test_count%7Bservice%21~%22.%2Abatch.%2A%22%7D%5B1d%5D+OFFSET+3d%29%29+BY+%28service%29%2C+%22delay%22%2C+%223d%22%2C+%22%22%2C+%22.%2A%22%29+or+label_replace%28sum%28increase%28etl_test_count%7Bservice%21~%22.%2Abatch.%2A%22%7D%5B1d%5D+OFFSET+5d%29%29+BY+%28service%29%2C+%22delay%22%2C+%225d%22%2C+%22%22%2C+%22.%2A%22%29+or+label_replace%28sum%28increase%28etl_test_count%7Bservice%21~%22.%2Abatch.%2A%22%7D%5B1d%5D+OFFSET+1w%29%29+BY+%28service%29%2C+%22delay%22%2C+%227d%22%2C+%22%22%2C+%22.%2A%22%29+or+label_replace%28label_replace%28vector%280%29%2C+%22delay%22%2C+%22c1%22%2C+%22%22%2C+%22.%2A%22%29%2C+%22service%22%2C+%22etl-ndt-parser%22%2C+%22%22%2C+%22.%2A%22%29+or+label_replace%28label_replace%28vector%280%29%2C+%22delay%22%2C+%22c2%22%2C+%22%22%2C+%22.%2A%22%29%2C+%22service%22%2C+%22etl-ndt-parser%22%2C+%22%22%2C+%22.%2A%22%29+or+label_replace%28label_replace%28vector%280%29%2C+%22delay%22%2C+%22c1%22%2C+%22%22%2C+%22.%2A%22%29%2C+%22service%22%2C+%22etl-sidestream-parser%22%2C+%22%22%2C+%22.%2A%22%29+or+label_replace%28label_replace%28vector%280%29%2C+%22delay%22%2C+%22c2%22%2C+%22%22%2C+%22.%2A%22%29%2C+%22service%22%2C+%22etl-sidestream-parser%22%2C+%22%22%2C+%22.%2A%22%29+or+label_replace%28label_replace%28vector%280%29%2C+%22delay%22%2C+%22c1%22%2C+%22%22%2C+%22.%2A%22%29%2C+%22service%22%2C+%22etl-traceroute-parser%22%2C+%22%22%2C+%22.%2A%22%29+or+label_replace%28label_replace%28vector%280%29%2C+%22delay%22%2C+%22c2%22%2C+%22%22%2C+%22.%2A%22%29%2C+%22service%22%2C+%22etl-traceroute-parser%22%2C+%22%22%2C+%22.%2A%22%29%29+BY+%28service%29%29&g0.tab=1

  Labels:

  • alertname = ParserDailyVolumeTooLow
  • service = etl-traceroute-parser
  • severity = page

  Annotations:

  • hints = Are machines online? Is data being collected? Is the parser working?
  • summary = Today's test volume is less than 70% of nominal daily volume.
  • url = http://status.mlab-oti.measurementlab.net:3000/dashboard/file/Alert_ParserDailyVolumeTooLow.json

TODO: add graph url from annotations.

[pboothe]

Something was wrong with our PT parser. Either the parser itself or the monitoring. @yachang is working on it, and in the meantime has rolled things back. The alert is no longer firing, so this bug is getting closed. Follow up work is occurring at https://github.com/m-lab/etl/issues/407