aquasecurity / trivy-operator

Kubernetes-native security toolkit
https://aquasecurity.github.io/trivy-operator/latest
Apache License 2.0
1.21k stars 201 forks source link

Java DB from local volume #1700

Open chen-keinan opened 9 months ago

chen-keinan commented 9 months ago

Discussed in https://github.com/aquasecurity/trivy-operator/discussions/1699

Originally posted by **gerbil** December 15, 2023 We got an issue with java db (400+mb) downloads from China. For main DB it's possible to use local volume and store it in local folder of trivy server (/home/scanner/.cache/trivy/db), but seems like the same is not possible for Java DB (/home/scanner/.cache/trivy/java-db) Any suggestions how we could try to mitigate it? We tried to skip java db update, but seems like it's trying to fetch it anyway: ``` clientServerSkipUpdate = true skipJavaDBUpdate = true offlineScan = true ``` Log from java scanning job: ``` The first run cannot skip downloading Java DB 2023-12-15T12:27:16.172Z ←[31mFATAL←[0m image scan error: scan error: scan failed: failed analysis: analyze error: pipeline error: failed to analyze layer (sha256:22bf74b2421f27ab1892c376590f6c9a520f7cee7f5eb57cb00c98367edfe82f) : post analysis error: post analysis error: Unable to initialize the Java DB: Java DB update failed: '--skip-java-db-update' cannot be specified on the first run ```
gerbil commented 9 months ago

Any ideas for workaround?

jpbteixeira commented 5 months ago

Did anyone sorted out this? Using trivy v.0.50.1 and trivy-operator v.0.19.3 still not working. Error: "The first run cannot skip downloading Java DB"

values.yaml offlineScan: true clientServerSkipUpdate: true skipJavaDBUpdate: true

I have manually added both db and java-db along with their manifest.json files to trivy-server cache "/home/scanner/.cache/trivy"

chen-keinan commented 5 months ago

@jpbteixeira can you please share more info on your env. ? air-gapped ? trivy is running in client/server mode?

jpbteixeira commented 5 months ago

trivy-operator deployed using helm chart version 0.21.2 trivy version: 0.50.1 trivy-operator 0.19.3 trivy.mode: ClientServer trivy.command: image

Approach 1: Cluster only have access to private registry - artifactory Both trivy-db and trivy-java-db have been uploaded to artifactory. Trivy-db works fine, trivy-java-db don't since there is no way we can provide credentials to authenticate

Approach 2: Using Air-Gapped Environment Values.yaml

trivy-db and trivy-java-db have been added to TRIVY_CACHE_DIR: /home/scanner/.cache/trivy on trivy-server image

Files under java-db: metadata.json and trivy-java-db

ERROR: Unable to initialize the Java DB: Java DB update failed: Java DB update error: '--skip-java-db-update' cannot be specified on the first run

Let me know if any more details are needed, please @chen-keinan

iambenzo commented 4 months ago

I'm having issues with this too.

The Operator is running with builtInTrivyServer: true (helm value) and I understand from the docs that, whilst the scanner pods get the trivy-db from the deployed server, they don't get the trivy-java-db in the same way.

I was hoping to get around the requirement of pulling the trivy-java-db each time (for the purposes of a PoC) by setting up a PVC, or baking the java-db into the scanner's container - which looks to have Trivy's cache at /root/.cache/trivy.

image

Having placed the java-db in that location, I'm still getting this error:

ERROR: Unable to initialize the Java DB: Java DB update failed: Java DB update error: '--skip-java-db-update' cannot be specified on the first run

Edit - I noticed that the cache location was being overridden by an environment variable to /home/scanner/.cache/trivy. I have also tried placing the java db in this location, without success.

According to the Trivy Docs, this should work.

Has anyone found a workaround for this?

The Trivy Server deployment has no issues in obtaining the trivy-db from my private OCI repository but the Scanners don't seem to authenticate.

Is there value in updating the mechanism for obtaining the trivy-java-db to match the mechanism for obtaining the trivy-db? Or is this more of a "Trivy" issue, than a "Trivy Operator" issue?

chen-keinan commented 4 months ago

@iambenzo can you describe scan-job pod and confirm that --skip-java-db-update is passed on args ?

kubectl describe pod <scan-job-xxx> -n trivy-system
iambenzo commented 4 months ago

Hey @chen-keinan

Here is the Args value from the scan job description:

    Args:
      -c
      trivy image --slow '<REDACTED>' --scanners vuln,secret --image-config-scanners secret    --skip-java-db-update --cache-dir /tmp/trivy/.cache --quiet --list-all-pkgs --format json --server 'http://trivy-service.trivy-system:4954' > /tmp/scan/result_trivy-server.json &&  bzip2 -c /tmp/scan/result_trivy-server.json | base64

The <REDACTED> part is the link to my private container registry.

chen-keinan commented 3 months ago

@iambenzo thanks for the input. I'll have a look and update if enhancement is required

jemag commented 3 days ago

has anyone been successful in implementing it or is this simply not possible at the moment?