search

aquasecurity / trivy

Find vulnerabilities, misconfigurations, secrets, SBOM in containers, Kubernetes, code repositories, clouds and more
https://aquasecurity.github.io/trivy
Apache License 2.0
23.74k stars 2.33k forks source link

Consider different storage options #18

Closed jbergstroem closed 1 year ago

jbergstroem commented 5 years ago

From bolt docs:

Please note that Bolt obtains a file lock on the data file so multiple processes cannot open the same database at the same time.

Lets say you have a CI which would do something like docker run -v /tmp/cache:.cache/trivy mydockerimage trivy ... This would imply that two scans would try to access the same cachefile and likely bail.

For this reason, exploring other storage options that would allow concurrent access could make sense. Perhaps another pattern would be copying the cache for each run and separate updates to the cache?

knqyf263 commented 5 years ago

@jbergstroem Thank you for the feedback. I'm considering about using Redis. How do you think?

jbergstroem commented 5 years ago

@jbergstroem Thank you for the feedback. I'm considering about using Redis. How do you think?

Had a think about it. I like the idea of a pluggable storage mechanism since storage at this point is a bit of a "flavor of the month" question. I would personally prefer something that speaks sql – but redis sounds perfectly fine as well – hence the pluggable idea.

christian-weiss commented 5 years ago

As long as you have no other backends. What about having one docker container being executed every hour to update a docker volume. And then mount/bind that docker volume to all other docker trivy containers as read-only and with --skip-update - that way you do not have concurrent writes and fastest possible trivy scan results.

This works well, if all docker containers are executed on the same host.On a small farm it is maybe an option to have the "update container" to be executed / sticky on each "worker node". On a large farm this is maybe not an option.

jbergstroem commented 5 years ago

As long as you have no other backends. What about having one docker container being executed every hour to update a docker volume. And then mount/bind that docker volume to all other docker trivy containers as read-only and with --skip-update - that way you do not have concurrent writes and fastest possible trivy scan results.

This works well, if all docker containers are executed on the same host.On a small farm it is maybe an option to have the "update container" to be executed / sticky on each "worker node". On a large farm this is maybe not an option.

I'm doing something like this at the moment; daily databases and mounting them to the cache dir. The size of these dailies are pretty significant though. I feel a bit bad for potentially pushing them to docker hub but can do if there's an interest..

knqyf263 commented 1 year ago

We have the Redis option now. https://aquasecurity.github.io/trivy/v0.38/docs/vulnerability/examples/cache/#cache-backend