search

anchore / grype

A vulnerability scanner for container images and filesystems
Apache License 2.0
8.47k stars 549 forks source link

Publish Grype image with embedded database #837

Open acartag7 opened 2 years ago

acartag7 commented 2 years ago

What happened: During our ci pipelines, we have noticed that when we run grype to scan a container image it sometimes fails when there is a high CPU usage on our pod or on the node that the pod is running. The failure comes from a timeout when it tries to download the vulnerability db.

Is there any requirement on how much resources does grype need to have reserved for this task? The limits we have are these: resources: limits: cpu: '1' memory: 2Gi requests: cpu: 500m memory: 1Gi

What you expected to happen: The database gets downloaded normally and the execution runs fine.

How to reproduce it (as minimally and precisely as possible): In any Kubernetes pod, just add some CPU load with something similar to : stress-ng --cpu 8 --vm 10 --vm-bytes 2G --timeout 60000s

After this try to run a scan: time grype ubuntu:latest -vv

After some minutes you will see the following line:

[0030] ERROR failed to fetch latest version: Get "https://toolbox-data.anchore.io/grype/releases/latest/VERSION": net/http: TLS handshake timeout

This takes around real 6m20.532s user 0m3.045s sys 1m48.307s

In our case, we download it from our artifactory registry.

Anything else we need to know?: When there is no load, the execution is fine and takes less time: real 0m50.315s user 0m11.424s sys 0m6.151s

Environment:

cpendery commented 2 years ago

@spiffcs I think one possible solution is having an additional Grype image (maybe tagged grype:ci) bundled with the database inside for CI that is republished whenever a new database snapshot gets produced and has GRYPE_DB_AUTO_UPDATE=false. I think the repository_dispatch GitHub actions workflow would allow you to kick of a republish of that image remotely once a new database snapshot is made. Linked a very rough example below

nigelgbanks commented 2 years ago

I should throw this in as well that grype does not handle multiple reads from the database concurrently well either.

With GRYPE_DB_AUTO_UPDATE=false multiple invocations of grype at the same time seem to block each other out so that is something to be aware of. If you intend as I did to share the database. 

 2022-08-04T11:05:50.3803827Z [0000]  WARN matcher failed for pkg=Pkg(type=apk, name=freetype, version=2.11.1-r2, upstreams=1): failed to find vulnerabilities for apk upstream source package: provider failed to fetch namespace='alpine:distro:alpine:3.15' pkg='freetype': database is locked (5) (SQLITE_BUSY)

@cpendery's approach of packaging the database in the image would resolve my issues as well.

cpendery commented 2 years ago

@nigelgbanks I think https://github.com/anchore/grype/issues/859 and https://github.com/anchore/grype/pull/854 relate to the multiple invocations. That fix should be in v0.45.0

kzantow commented 2 years ago

@nigelgbanks the error you referenced with database is locked (5) (SQLITE_BUSY) should be fixed with the latest release (yesterday, as @cpendery noted).

spiffcs commented 2 years ago

@awca22 From the community meeting (8/4): We're going to move forward and start publishing a grype:ci with the prepackaged db.

Look for that coming out in the next release cycle.

We're also going to do some investigation to see if we can bring down/isolate the CPU usage for the download/verification of the DB

Thanks @awca22 again for filing the issue, and thanks @cpendery for posting the solution/suggestion!

mirekphd commented 1 year ago

What happened to the excellent idea of the integrated app and db container? I seem to be unable to find the planned ci tag in your Docker Hub repo (anchore/grype).

The database file has over 1 GB now: 1040506880 Jul 28 16:53 vulnerability.db

Pulling it before each scan (the default setting of check-for-app-update/GRYPE_CHECK_FOR_APP_UPDATE) seems prudent but a tiny bit inefficient - once a day would be fine if it cannot be updated incrementally.

kzantow commented 1 year ago

@mirekphd there should be a fairly simple way to build your own image that does this, without the need for published images with stale databases, would that work for you? Here's an example Dockerfile:

FROM alpine:latest
RUN apk update && apk --no-cache add curl && \
  curl -sSfL https://raw.githubusercontent.com/anchore/grype/main/install.sh | sh -s -- -b / && \
  /grype db update
ENV GRYPE_CHECK_FOR_APP_UPDATE=false GRYPE_DB_AUTO_UPDATE=false
ENTRYPOINT [ "/grype" ]

This disables both the app update check and the db update check when running the image.

Also note: Grype has a staleness check, this can be disabled by adding another ENV entry: GRYPE_DB_VALIDATE_AGE=false but of course this is not ideal, since vulnerability data is constantly evolving. This wouldn't be a problem if you built an image daily.

This would substitute downloading the database with downloading the image. Maybe that's better for you?

tomerse-sg commented 3 months ago

Hi, I noticed the same issue when i run grype from docker. I do need to download the db, but it takes very long and uses more than 4gb memory.

using grype 0.74.4