aquasecurity / trivy

Find vulnerabilities, misconfigurations, secrets, SBOM in containers, Kubernetes, code repositories, clouds and more
https://aquasecurity.github.io/trivy
Apache License 2.0
23.36k stars 2.31k forks source link

fix(helm): properly handle multiple archived dependencies #7781

Open nikpivkin opened 1 day ago

nikpivkin commented 1 day ago

Discussed in https://github.com/aquasecurity/trivy/discussions/7778

Originally posted by **prezha** October 23, 2024 ### Description i think i found a bug in trivy that is causing a `file does not exist` issue while scanning tar-ed helm charts with with newer (ie, 0.55.0+) trivy versions - tl;dr: - helm parser calls ParseFS() [recursively](https://github.com/aquasecurity/trivy/blob/9514148767865baddd73a49245385574927f7a74/pkg/iac/scanners/helm/parser/parser.go#L112) that in turn [uses](https://github.com/aquasecurity/trivy/blob/9514148767865baddd73a49245385574927f7a74/pkg/iac/scanners/helm/parser/parser.go#L85-L85) fs.WalkDir() which is also recursive itself - once an archive is processed, it gets [removed](https://github.com/aquasecurity/trivy/blob/9514148767865baddd73a49245385574927f7a74/pkg/iac/scanners/helm/parser/parser_tar.go#L101) from the memoryfs.FS - now, while returning from recursive calls, the "parent" caller might try to process archive that was previously known to fs.WalkDir() to exist, but the "child" might have already processed and removed it - so, we'd need to detect and skip those the bug was probably introduced in commit [e95152f](https://github.com/aquasecurity/trivy/commit/e95152f796308c25aaaa6cf75b2e7a81b3ab4388) as part of the https://github.com/aquasecurity/trivy/pull/7289, where a check was [removed](https://github.com/aquasecurity/trivy/commit/e95152f796308c25aaaa6cf75b2e7a81b3ab4388#diff-a8b46bcbae1373c8cfdcc22b2cd2d476644574ea2ff4cc71e24ef2c484773922L132-L135), which further means that it was probably introduced in trivy v0.55.0 but it was working in v0.54.1 btw, i have a pr proposal that might fix this issue, if you'd like to review, just let me know ### Desired Behavior trivy does not error while processing tar-ed helm charts ### Actual Behavior trivy errors with `file does not exist` while processing tar-ed helm charts ### Reproduction Steps ```bash 1. add eg, https://github.com/bitnami/charts/tree/main/bitnami/common v2.26.0 2. add eg, https://github.com/open-telemetry/opentelemetry-helm-charts/releases/tag/opentelemetry-collector-0.108.0 3. run 'trivy fs --scanners misconfig --debug --quiet=false .' ``` ### Target Filesystem ### Scanner Misconfiguration ### Output Format None ### Mode Standalone ### Debug Output ```bash $ trivy fs --scanners misconfig --debug --quiet=false . 2024-10-22T23:49:44+01:00 DEBUG No plugins loaded 2024-10-22T23:49:44+01:00 DEBUG Default config file "file_path=trivy.yaml" not found, using built in values 2024-10-22T23:49:44+01:00 DEBUG Cache dir dir="/home/prezha/.cache/trivy" 2024-10-22T23:49:44+01:00 DEBUG Cache dir dir="/home/prezha/.cache/trivy" 2024-10-22T23:49:44+01:00 DEBUG Parsed severities severities=[UNKNOWN LOW MEDIUM HIGH CRITICAL] 2024-10-22T23:49:44+01:00 DEBUG Ignore statuses statuses=[] 2024-10-22T23:49:44+01:00 INFO [misconfig] Misconfiguration scanning is enabled 2024-10-22T23:49:44+01:00 DEBUG [misconfig] Failed to open the check metadata err="open /home/prezha/.cache/trivy/policy/metadata.json: no such file or directory" 2024-10-22T23:49:44+01:00 INFO [misconfig] Need to update the built-in checks 2024-10-22T23:49:44+01:00 INFO [misconfig] Downloading the built-in checks... 2024-10-22T23:49:44+01:00 DEBUG [misconfig] Loading check bundle repository="ghcr.io/aquasecurity/trivy-checks:1" 156.02 KiB / 156.02 KiB [---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s 0s 2024-10-22T23:49:45+01:00 DEBUG [misconfig] Digest of the built-in checks digest="sha256:ae151c4eecf35c507d8f866121ddfbf46540b041bc7bca7cdd8d9f70ceb6f12c" 2024-10-22T23:49:45+01:00 DEBUG [misconfig] Checks successfully loaded from disk 2024-10-22T23:49:45+01:00 DEBUG Enabling misconfiguration scanners scanners=[azure-arm cloudformation dockerfile helm kubernetes terraform terraformplan-json terraformplan-snapshot] 2024-10-22T23:49:45+01:00 DEBUG Initializing scan cache... type="memory" 2024-10-22T23:49:45+01:00 DEBUG Skipping path path=".git" 2024-10-22T23:49:45+01:00 DEBUG Scanning files for misconfigurations... scanner="Helm" 2024-10-22T23:49:46+01:00 DEBUG [rego] Overriding filesystem for checks 2024-10-22T23:49:46+01:00 DEBUG [rego] Embedded libraries are loaded count=13 2024-10-22T23:49:46+01:00 DEBUG [rego] Embedded checks are loaded count=508 2024-10-22T23:49:46+01:00 DEBUG [rego] Checks from disk are loaded count=521 2024-10-22T23:49:46+01:00 DEBUG [rego] Overriding filesystem for data 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/common/Chart.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/common/values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/common/templates/_affinities.tpl" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/common/templates/_capabilities.tpl" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/common/templates/_compatibility.tpl" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/common/templates/_errors.tpl" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/common/templates/_images.tpl" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/common/templates/_ingress.tpl" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/common/templates/_labels.tpl" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/common/templates/_names.tpl" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/common/templates/_resources.tpl" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/common/templates/_secrets.tpl" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/common/templates/_storage.tpl" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/common/templates/_tplvalues.tpl" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/common/templates/_utils.tpl" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/common/templates/_warnings.tpl" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/common/templates/validations/_cassandra.tpl" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/common/templates/validations/_mariadb.tpl" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/common/templates/validations/_mongodb.tpl" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/common/templates/validations/_mysql.tpl" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/common/templates/validations/_postgresql.tpl" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/common/templates/validations/_redis.tpl" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/common/templates/validations/_validations.tpl" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/common/.helmignore" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/common/README.md" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/Chart.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/values.schema.json" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/templates/NOTES.txt" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/templates/_config.tpl" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/templates/_helpers.tpl" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/templates/_pod.tpl" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/templates/clusterrole.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/templates/clusterrolebinding.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/templates/configmap-agent.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/templates/configmap-statefulset.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/templates/configmap.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/templates/daemonset.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/templates/deployment.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/templates/hpa.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/templates/ingress.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/templates/networkpolicy.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/templates/pdb.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/templates/podmonitor.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/templates/prometheusrule.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/templates/service.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/templates/serviceaccount.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/templates/servicemonitor.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/templates/statefulset.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/.helmignore" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/CONTRIBUTING.md" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/README.md" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/UPGRADING.md" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/ci/GOMEMLIMIT-values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/ci/clusterrole-values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/ci/config-override-values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/ci/daemonset-values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/ci/deployment-values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/ci/disabling-protocols-values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/ci/hpa-deployment-values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/ci/hpa-statefulset-values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/ci/multiple-ingress-values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/ci/networkpolicy-override-values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/ci/networkpolicy-values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/ci/preset-clustermetrics-values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/ci/preset-hostmetrics-values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/ci/preset-k8sevents-values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/ci/preset-kubeletmetrics-values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/ci/preset-kubernetesattributes-values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/ci/preset-logscollection-values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/ci/probes-values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/ci/statefulset-values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/README.md" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/alternate-config/rendered/clusterrole.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/alternate-config/rendered/clusterrolebinding.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/alternate-config/rendered/configmap.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/alternate-config/rendered/deployment.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/alternate-config/rendered/service.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/alternate-config/rendered/serviceaccount.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/alternate-config/values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/daemonset-and-deployment/daemonset-values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/daemonset-and-deployment/deployment-values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/daemonset-and-deployment/rendered/configmap-agent.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/daemonset-and-deployment/rendered/configmap.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/daemonset-and-deployment/rendered/daemonset.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/daemonset-and-deployment/rendered/deployment.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/daemonset-and-deployment/rendered/service.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/daemonset-and-deployment/rendered/serviceaccount.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/daemonset-collector-logs/rendered/configmap-agent.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/daemonset-collector-logs/rendered/daemonset.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/daemonset-collector-logs/rendered/serviceaccount.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/daemonset-collector-logs/values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/daemonset-hostmetrics/rendered/configmap-agent.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/daemonset-hostmetrics/rendered/daemonset.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/daemonset-hostmetrics/rendered/serviceaccount.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/daemonset-hostmetrics/values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/daemonset-lifecycle-hooks/rendered/configmap-agent.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/daemonset-lifecycle-hooks/rendered/daemonset.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/daemonset-lifecycle-hooks/rendered/serviceaccount.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/daemonset-lifecycle-hooks/values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/daemonset-only/rendered/configmap-agent.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/daemonset-only/rendered/daemonset.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/daemonset-only/rendered/serviceaccount.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/daemonset-only/values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/deployment-only/rendered/configmap.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/deployment-only/rendered/deployment.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/deployment-only/rendered/service.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/deployment-only/rendered/serviceaccount.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/deployment-only/values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/deployment-otlp-traces/rendered/configmap.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/deployment-otlp-traces/rendered/deployment.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/deployment-otlp-traces/rendered/service.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/deployment-otlp-traces/rendered/serviceaccount.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/deployment-otlp-traces/values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/deployment-use-existing-configMap/deployment-values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/deployment-use-existing-configMap/rendered/deployment.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/deployment-use-existing-configMap/rendered/service.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/deployment-use-existing-configMap/rendered/serviceaccount.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/kubernetesAttributes/rendered/clusterrole.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/kubernetesAttributes/rendered/clusterrolebinding.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/kubernetesAttributes/rendered/configmap.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/kubernetesAttributes/rendered/deployment.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/kubernetesAttributes/rendered/service.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/kubernetesAttributes/rendered/serviceaccount.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/kubernetesAttributes/values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/statefulset-only/rendered/configmap-statefulset.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/statefulset-only/rendered/service.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/statefulset-only/rendered/serviceaccount.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/statefulset-only/rendered/statefulset.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/statefulset-only/values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/statefulset-with-pvc/rendered/configmap-statefulset.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/statefulset-with-pvc/rendered/service.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/statefulset-with-pvc/rendered/serviceaccount.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/statefulset-with-pvc/rendered/statefulset.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/statefulset-with-pvc/values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/using-GOMEMLIMIT/rendered/configmap.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/using-GOMEMLIMIT/rendered/deployment.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/using-GOMEMLIMIT/rendered/service.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/using-GOMEMLIMIT/rendered/serviceaccount.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/using-GOMEMLIMIT/values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/using-custom-config/rendered/deployment.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/using-custom-config/rendered/service.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/using-custom-config/rendered/serviceaccount.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/using-custom-config/values.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/using-shared-processes/rendered/configmap.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/using-shared-processes/rendered/deployment.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/using-shared-processes/rendered/service.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/using-shared-processes/rendered/serviceaccount.yaml" 2024-10-22T23:49:46+01:00 DEBUG [helm parser] Unpacking tar entry file_path="charts/opentelemetry-collector/examples/using-shared-processes/values.yaml" 2024-10-22T23:49:46+01:00 FATAL Fatal error - fs scan error: github.com/aquasecurity/trivy/pkg/commands/artifact.Run /home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:387 - scan error: github.com/aquasecurity/trivy/pkg/commands/artifact.(*runner).scanArtifact /home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:261 - scan failed: github.com/aquasecurity/trivy/pkg/commands/artifact.(*runner).scan /home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:622 - failed analysis: github.com/aquasecurity/trivy/pkg/scanner.Scanner.ScanArtifact /home/runner/work/trivy/trivy/pkg/scanner/scan.go:158 - post analysis error: github.com/aquasecurity/trivy/pkg/fanal/artifact/local.Artifact.Inspect /home/runner/work/trivy/trivy/pkg/fanal/artifact/local/fs.go:121 - post analysis error: github.com/aquasecurity/trivy/pkg/fanal/analyzer.AnalyzerGroup.PostAnalyze /home/runner/work/trivy/trivy/pkg/fanal/analyzer/analyzer.go:510 - helm scan error: github.com/aquasecurity/trivy/pkg/fanal/analyzer/config.(*Analyzer).PostAnalyze /home/runner/work/trivy/trivy/pkg/fanal/analyzer/config/config.go:46 - scan config error: github.com/aquasecurity/trivy/pkg/misconf.(*Scanner).Scan /home/runner/work/trivy/trivy/pkg/misconf/scanner.go:153 - walk dir error: failed to add tar "charts/opentelemetry-collector-0.108.0.tgz" to FS: failed to open tar: open charts/opentelemetry-collector-0.108.0.tgz: file does not exist ``` ### Operating System openSUSE Tumbleweed, Ubuntu 22.04, macOS Sonoma 14.7 ### Version ```bash $ trivy version Version: 0.56.2 Check Bundle: Digest: sha256:ae151c4eecf35c507d8f866121ddfbf46540b041bc7bca7cdd8d9f70ceb6f12c DownloadedAt: 2024-10-22 22:49:45.533804486 +0000 UTC ``` ### Checklist - [X] Run `trivy clean --all` - [X] Read [the troubleshooting](https://aquasecurity.github.io/trivy/latest/docs/references/troubleshooting/)
prezha commented 1 day ago

thanks @nikpivkin for creating the issue

fwiw, my thinking was to add

            // when returning from recursive calls, archives that were previously processed were also removed from fs,
            // ref: https://github.com/aquasecurity/trivy/blob/9514148767865baddd73a49245385574927f7a74/pkg/iac/scanners/helm/parser/parser_tar.go#L101
            // but they might be already picked by fs.WalkDir(), so we need to skip them
            // note that checking entry.Info() might not work:
            // "If the file has been removed or renamed since the directory read, Info *may* return an error satisfying errors.Is(err, ErrNotExist)."
            // ref: https://pkg.go.dev/io/fs#DirEntry
            if matches, err := fs.Glob(p.workingFS, path); err == nil && matches == nil {
                return nil
            }

just after this line https://github.com/aquasecurity/trivy/blob/9514148767865baddd73a49245385574927f7a74/pkg/iac/scanners/helm/parser/parser.go#L98

nikpivkin commented 1 day ago

@prezha I think you can use fs.Stat to check if the file exists. In any case, Helm knows how to handle dependencies in the archive, so we may not extract them to avoid re-scanning all files.

prezha commented 1 day ago

@nikpivkin you're right, fs.Stat would be simpler/better than fs.Glob to check if archive file exists and skip it if not