search

aquasecurity / harbor-scanner-trivy

Use Trivy as a plug-in vulnerability scanner in the Harbor registry
https://goharbor.io
Apache License 2.0
215 stars 73 forks source link

trivy should skip to generate sbom for container images that have layers[0].mediaType being not valid #476

Open zyyw opened 3 months ago

zyyw commented 3 months ago

Reproduce steps:

  1. deploy Harbor with this offline build: https://storage.googleapis.com/harbor-builds/harbor-offline-installer-latest.tgz
  2. create replication endpoint DockerHub and create a pull-based replication rule (name should be library/redis, tag matching 7.2.4)
  3. go to the redis repo, and select the index manifest replicated to Harbor, and then click GENERATE SBOM. You will see some of the container images having SBOM generated successfully while the others failed with error message, like below: 
    2024-05-14T09:58:25Z [DEBUG] [/pkg/scan/job.go:401]: registration:
2024-05-14T09:58:25Z [INFO] [/pkg/scan/job.go:412]: {
"uuid": "5d4e8fc9-10d4-11ef-898e-0242ac140009",
"name": "Trivy",
"description": "The Trivy scanner adapter",
"url": "http://trivy-adapter:8080",
"disabled": false,
"is_default": true,
"health": "healthy",
"auth": "",
"access_credential": "[HIDDEN]",
"skip_certVerify": false,
"use_internal_addr": true,
"adapter": "Trivy",
"vendor": "Aqua Security",
"version": "v0.50.4",
"create_time": "2024-05-13T02:56:15.631936Z",
"update_time": "2024-05-13T02:56:15.631937Z"
}
2024-05-14T09:58:25Z [DEBUG] [/pkg/scan/job.go:401]: scanRequest:
2024-05-14T09:58:25Z [INFO] [/pkg/scan/job.go:412]: {
"registry": {
"url": "http://core:8080",
"authorization": "[HIDDEN]"
},
"artifact": {
"namespace_id": 12,
"repository": "library/redis",
"tag": "",
"digest": "sha256:63e00e276c28cc8d5d4d670aacc53d8ae6b6e08a70ee59666ce3aa4aba06007f",
"mime_type": "application/vnd.oci.image.manifest.v1+json",
"size": 2277725
},
"enabled_capabilities": null
}
2024-05-14T09:58:25Z [INFO] [/pkg/scan/job.go:174]: Report mime types: [application/vnd.security.sbom.report+json; version=1.0]
2024-05-14T09:58:25Z [INFO] [/pkg/scan/job.go:231]: Get report for mime type: application/vnd.security.sbom.report+json; version=1.0
2024-05-14T09:58:27Z [DEBUG] [/pkg/scan/job.go:244]: check scan report for mime application/vnd.security.sbom.report+json; version=1.0 at 2024/05/14 09:58:27
2024-05-14T09:58:27Z [INFO] [/pkg/scan/job.go:257]: Report with mime type application/vnd.security.sbom.report+json; version=1.0 is not ready yet, retry after 5 seconds
2024-05-14T09:58:32Z [DEBUG] [/pkg/scan/job.go:244]: check scan report for mime application/vnd.security.sbom.report+json; version=1.0 at 2024/05/14 09:58:32
2024-05-14T09:58:32Z [INFO] [/pkg/scan/job.go:257]: Report with mime type application/vnd.security.sbom.report+json; version=1.0 is not ready yet, retry after 5 seconds
2024-05-14T09:58:37Z [DEBUG] [/pkg/scan/job.go:244]: check scan report for mime application/vnd.security.sbom.report+json; version=1.0 at 2024/05/14 09:58:37
2024-05-14T09:58:37Z [INFO] [/pkg/scan/job.go:257]: Report with mime type application/vnd.security.sbom.report+json; version=1.0 is not ready yet, retry after 5 seconds
2024-05-14T09:58:42Z [DEBUG] [/pkg/scan/job.go:244]: check scan report for mime application/vnd.security.sbom.report+json; version=1.0 at 2024/05/14 09:58:42
2024-05-14T09:58:42Z [ERROR] [/pkg/scan/job.go:296]: scan job: fetch scan report, mimetype application/vnd.security.sbom.report+json; version=1.0: running trivy wrapper: running trivy: exit status 1: 2024-05-14T09:58:40.385Z    DEBUG  ["cyclonedx" "spdx" "spdx-json" "github"] automatically enables '--list-all-pkgs'.
2024-05-14T09:58:40.386Z    DEBUG  Severities: ["UNKNOWN" "LOW" "MEDIUM" "HIGH" "CRITICAL"]
2024-05-14T09:58:40.386Z    DEBUG  Ignore statuses {"statuses": null}
2024-05-14T09:58:40.386Z    INFO   "--format spdx" and "--format spdx-json" disable security scanning
2024-05-14T09:58:40.393Z    DEBUG  cache dir:  /home/scanner/.cache/trivy
2024-05-14T09:58:40.393Z    DEBUG  Enabling misconfiguration scanners: [azure-arm cloudformation dockerfile helm kubernetes terraform terraformplan-json terraformplan-snapshot]
2024-05-14T09:58:40.439Z    DEBUG  The nuget packages directory couldn't be found. License search disabled
2024-05-14T09:58:40.454Z    DEBUG  Image ID: sha256:5283e486205a0217526302f535acc18969d064e27e5db7b0dffeddcd899a4a82
2024-05-14T09:58:40.454Z    DEBUG  Diff IDs: [sha256:70b1e9c5bc0dad76caceccd108901ab94d5ee93ed915c475ed9ecee25889aae3 sha256:30994670b6824f47fbf36939b35db192f90d4aeb873c48a02a75e2f1a7f539c4]
2024-05-14T09:58:40.454Z    DEBUG  Base Layers: []
2024-05-14T09:58:40.455Z    DEBUG  Missing image ID in cache: sha256:5283e486205a0217526302f535acc18969d064e27e5db7b0dffeddcd899a4a82
2024-05-14T09:58:40.455Z    DEBUG  Missing diff ID in cache: sha256:70b1e9c5bc0dad76caceccd108901ab94d5ee93ed915c475ed9ecee25889aae3
2024-05-14T09:58:40.455Z    DEBUG  Missing diff ID in cache: sha256:30994670b6824f47fbf36939b35db192f90d4aeb873c48a02a75e2f1a7f539c4
2024-05-14T09:58:40.476Z    FATAL  image scan error:
github.com/aquasecurity/trivy/pkg/commands/artifact.Run
    /home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:425
- scan error:
github.com/aquasecurity/trivy/pkg/commands/artifact.(*runner).scanArtifact
    /home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:269
- scan failed:
github.com/aquasecurity/trivy/pkg/commands/artifact.scan
    /home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:710
- failed analysis:
github.com/aquasecurity/trivy/pkg/scanner.Scanner.ScanArtifact
    /home/runner/work/trivy/trivy/pkg/scanner/scan.go:148
- analyze error:
github.com/aquasecurity/trivy/pkg/fanal/artifact/image.Artifact.Inspect
    /home/runner/work/trivy/trivy/pkg/fanal/artifact/image/image.go:126
- pipeline error:
github.com/aquasecurity/trivy/pkg/fanal/artifact/image.Artifact.inspect
    /home/runner/work/trivy/trivy/pkg/fanal/artifact/image/image.go:229
- failed to analyze layer (sha256:30994670b6824f47fbf36939b35db192f90d4aeb873c48a02a75e2f1a7f539c4):
github.com/aquasecurity/trivy/pkg/fanal/artifact/image.Artifact.inspect.func1
    /home/runner/work/trivy/trivy/pkg/fanal/artifact/image/image.go:216
- walk error:
github.com/aquasecurity/trivy/pkg/fanal/artifact/image.Artifact.inspectLayer
    /home/runner/work/trivy/trivy/pkg/fanal/artifact/image/image.go:290
- failed to extract the archive:
github.com/aquasecurity/trivy/pkg/fanal/walker.LayerTar.Walk
    /home/runner/work/trivy/trivy/pkg/fanal/walker/tar.go:44
- archive/tar: invalid tar header
: general response handler: unexpected status code: 500, expected: 200

For those container images that have SBOM generated successfully, it is because they have application/vnd.oci.image.layer.v1.tar+gzip in layers[0].mediaType.

Screenshot 2024-05-14 at 6 15 51 PM

For those container images that failed, it is because they do NOT have application/vnd.oci.image.layer.v1.tar+gzip in layers[0].mediaType.

Screenshot 2024-05-14 at 6 14 08 PM

Trivy has an assumption that the layers[0].mediaType is tar+gzip related, but it is not always true.

knqyf263 commented 3 months ago

What kind of error does Harbor expect when scanning an artifact Trivy doesn't support?

zyyw commented 3 months ago

This is the response code for POST /scan request:

Maybe it should return 400 when scanning an artifact Trivy doesn't support. @stonezdj @wy65701436 what's your opinion?