One of the diffing service pods started failing and returning a variety of errors, causing downstream errors in the DB’s auto-analysis job. @Mr0grog was offline at a camp and was unable to address it for a full day.
The problem appears to have been caused by a broken process pool, but I’m still looking into the original cause and possible remediation in code. The incident itself has been resolved, though.
One of the diffing service pods started failing and returning a variety of errors, causing downstream errors in the DB’s auto-analysis job. @Mr0grog was offline at a camp and was unable to address it for a full day.
The problem appears to have been caused by a broken process pool, but I’m still looking into the original cause and possible remediation in code. The incident itself has been resolved, though.
/cc @danielballan