search

anchore / vunnel

Tool for collecting vulnerability data from various sources (used to build the grype database)
Apache License 2.0
69 stars 25 forks source link

fix: lower retry threshold and continue on 403 for amazon linux feed #527

Closed spiffcs closed 5 months ago

spiffcs commented 5 months ago

Summary

The amazon linux rss feeds for vulnerabilities currently has multiple 403'ed entries for the following feeds:

This PR is a draft hack that get's the amazon linux provider to completion while warning on links that are unable to be reached.

For more information about this failure see: https://github.com/anchore/vunnel/actions/runs/8477396093

Note: for that run a single entry failed causing the whole process to exit. On this branch, users will see multiple failures of entries, but given the new exception clause the provider will continue to try other entries in the rss feed.

I do not think we should merge or consider this PR until we get an answer from aws security on the current state of the feeds and if any updates are expected surrounding the 403'ed records.

It's also worth noting that given the current state of the Amazon Linux Feeds we're missing an entire namespace:

{'amazon:distro:amazonlinux:2022'}

This is missing because every entry in the following feed is 403'ed: https://alas.aws.amazon.com/AL2022/alas.rss

willmurphyscode commented 5 months ago

We need to be careful that vunnel can't exit zero and emit a completely broken result set.

Today, vunnel exits non-zero when it fails to build a new result set for a provider, which causes grype-db to re-use the previous day's cached results. After this change, if due to a misconfiguration or something, a large percentage of the listings raise 403, vunnel could write a result set with big gaps in it but still exit 0, causing the result set with big gaps to be used in grype-db, instead of the result set from the previous day.