kostmo / circleci-failure-tracker

A log analyzer for CircleCI. Note that this project is now hosted at pytorch/dr-ci
https://github.com/pytorch/dr-ci
5 stars 2 forks source link

Jobs sometimes inaccurately classified as non-flaky due to duplicate failure notifications #84

Open kostmo opened 4 years ago

kostmo commented 4 years ago

Part of the logic for determining non-flakiness checks whether a job has failed more than once at the same commit (i.e., if it fails consistently). However, sometimes CircleCI sends multiple failure notifications for the same Build ID. We should deduplicate these; only count multiple failures if they came from distinct Build IDs at a given commit and job.

One attempt has been made so far to fix this, by introducing a de-duplicating database view (see 70269e949). However, the query performed poorly, probably because of the way data was split across multiple tables.

The next step is to consolidate the data into one table so that a unified index can be created, then measure the performance of the query again.

cc @malfet @orionr