search

fleetdm / fleet

Open-source platform for IT, security, and infrastructure teams. (Linux, macOS, Chrome, Windows, cloud, data center)
https://fleetdm.com
Other
2.92k stars 409 forks source link

Handle non-Ok response codes in download.go #21388

Open iansltx opened 3 weeks ago

iansltx commented 3 weeks ago

Problem

Sometimes we don't catch invalid downloads until extraction, or further down the process, when we could have retried earlier based on an unexpected HTTP response code.

Potential solutions

Either log (warn-level) or error out (allowing for a retry) when we see a non-200 response code in pkg/download/download.go. What we do depends on whether we're confident that all calls to this function should return 200s (I'm giving it a solid "probably" because the only usages here are in nvd and oval).

What is the expected workflow as a result of your proposal?

Flaky test runs start recording the HTTP response code from e.g. Canonical rather than a "magic number" failure when bz2 files download, assuming Canonical is returning 4xx/5xx on the responses that we're seeing fail now.

iansltx commented 3 weeks ago

Example of a tlakey test run (saw this yesterday as well): https://github.com/fleetdm/fleet/actions/runs/10465033080/job/28979594785

lucasmrod commented 3 days ago

@iansltx Is this request still accurate after the recent changes to the download package and the decision to fix the Ubuntu Kinetic URL in a different way?

iansltx commented 3 days ago

Yeah, we've solved this already.