pantsbuild / pants

The Pants Build System
https://www.pantsbuild.org
Apache License 2.0
3.19k stars 613 forks source link

retry on downloading files server errors #21093

Open cburroughs opened 2 weeks ago

cburroughs commented 2 weeks ago

Is your feature request related to a problem? Please describe.

We have recently seen a CI job fail along the lines of:

00:14:03.03 [ERROR] 1 Exception encountered:
Engine traceback:
  in `lint` goal
IntrinsicError: Server error (502) downloading file helm-unittest-linux-amd64-0.3.3.tgz from https://github.com/helm-unittest/helm-unittest/releases/download/v0.3.3/helm-unittest-linux-amd64-0.3.3.tgz

This is almost certainly a transient error.

Describe the solution you'd like

Some amount of retry/backoff for this class of errors.

Describe alternatives you've considered

Retry the entire CI job on any error. Besides using a lot more compute, I would prefer to avoid treats invocations of pants as flaky things that need retry-wrappers.

Additional context

src/rust/engine/src/downloads.rs has a variety of references to retries already, but I don't read Rust well enough to be clear on the current state.