search

docker / sbom-cli-plugin

Plugin for Docker CLI to support SBOM creation using Syft
Apache License 2.0
154 stars 15 forks source link

TestAllFormatsExpressible/format:syft-table fails when building sbom-cli-plugin #32

Open davidhay1969 opened 1 year ago

davidhay1969 commented 1 year ago

What happened:

The TestAllFormatsExpressible test fails due, I believe, to syft related issue: -

cd /root/go/src/github.com/docker/sbom-cli-plugin/test/cli

go test -v ./... --run TestAllFormatsExpressible

=== RUN   TestAllFormatsExpressible
    utils_test.go:56: obtaining fixture image for image-pkg-coverage
=== RUN   TestAllFormatsExpressible/format:syft-3-json
=== RUN   TestAllFormatsExpressible/format:cyclonedx-1-xml
=== RUN   TestAllFormatsExpressible/format:cyclonedx-1-json
=== RUN   TestAllFormatsExpressible/format:github-0-json
=== RUN   TestAllFormatsExpressible/format:spdx-2-tag-value
=== RUN   TestAllFormatsExpressible/format:spdx-2-json
=== RUN   TestAllFormatsExpressible/format:syft-table
    all_formats_expressible_test.go:28: there may not be any report output (len=747)
    all_formats_expressible_test.go:31: STDOUT:
         NAME              VERSION    TYPE
        Pygments          2.6.1      python
        apt               1.8.2      deb
        bundler           2.1.4      gem
        dash              0.5.8-2.4  deb
        dive              0.9.2-1    rpm
        libc-utils        0.7.2-r0   apk
        musl-utils        1.1.24-r2  apk
        netbase           5.4        deb
        nikic/fast-route  v1.3.0     php-composer
        npm               6.14.6     npm
        psr/container     2.0.2      php-composer
        psr/http-factory  1.0.1      php-composer
        requests          2.22.0     python
        someotherpkg      3.19.0     python
        somerequests      3.22.0     python
        unbundler         3.1.4      gem

    all_formats_expressible_test.go:32: STDERR:
         [0000]  WARN cataloger 'java-cataloger' failed to parse entries at location=Location<id=5 RealPath="/java/example-jenkins-plugin.hpi" VirtualPath="/java/example-jenkins-plugin.hpi" Layer="sha256:a908332de3f8ebb4f7c95fb6869b745325f8fe96bb4d1c472b263fba6b173529">: unable to read files from java archive: unable to open zip archive (/tmp/syft-archive-contents-1598349393/archive-example-jenkins-plugin.hpi): cannot find beginning of zip archive="/tmp/syft-archive-contents-1598349393/archive-example-jenkins-plugin.hpi" : zip: not a valid zip file
        [0000]  WARN cataloger 'java-cataloger' failed to parse entries at location=Location<id=4 RealPath="/java/example-java-app-maven-0.1.0.jar" VirtualPath="/java/example-java-app-maven-0.1.0.jar" Layer="sha256:a908332de3f8ebb4f7c95fb6869b745325f8fe96bb4d1c472b263fba6b173529">: unable to read files from java archive: unable to open zip archive (/tmp/syft-archive-contents-1355980804/archive-example-java-app-maven-0.1.0.jar): cannot find beginning of zip archive="/tmp/syft-archive-contents-1355980804/archive-example-java-app-maven-0.1.0.jar" : zip: not a valid zip file

    all_formats_expressible_test.go:33: COMMAND: /root/go/src/github.com/docker/sbom-cli-plugin/snapshot/sbom-cli-plugin_linux_amd64/docker-sbom sbom stereoscope-fixture-image-pkg-coverage:c531ccd41ba451da7aa4700ba89e889b9109c2841710bb0c80af91d10705b6d6 --format syft-table
=== RUN   TestAllFormatsExpressible/format:syft-text
--- FAIL: TestAllFormatsExpressible (1.49s)
    --- PASS: TestAllFormatsExpressible/format:syft-3-json (0.15s)
    --- PASS: TestAllFormatsExpressible/format:cyclonedx-1-xml (0.14s)
    --- PASS: TestAllFormatsExpressible/format:cyclonedx-1-json (0.17s)
    --- PASS: TestAllFormatsExpressible/format:github-0-json (0.16s)
    --- PASS: TestAllFormatsExpressible/format:spdx-2-tag-value (0.17s)
    --- PASS: TestAllFormatsExpressible/format:spdx-2-json (0.16s)
    --- FAIL: TestAllFormatsExpressible/format:syft-table (0.13s)
    --- PASS: TestAllFormatsExpressible/format:syft-text (0.15s)
FAIL
FAIL    github.com/docker/sbom-cli-plugin/test/cli  1.867s
FAIL

I see the same if I run the bundled docker-sbom binary, which includes syft v0.46.3 : -

/root/go/src/github.com/docker/sbom-cli-plugin/snapshot/sbom-cli-plugin_linux_amd64/docker-sbom sbom stereoscope-fixture-image-pkg-coverage:c531ccd41ba451da7aa4700ba89e889b9109c2841710bb0c80af91d10705b6d6 --format syft-table

Syft v0.46.3
 ✔ Loaded image
 ✔ Parsed image
 ✔ Cataloged packages      [16 packages]
[0000]  WARN cataloger 'java-cataloger' failed to parse entries at location=Location<id=4 RealPath="/java/example-java-app-maven-0.1.0.jar" VirtualPath="/java/example-java-app-maven-0.1.0.jar" Layer="sha256:a908332de3f8ebb4f7c95fb6869b745325f8fe96bb4d1c472b263fba6b173529">: unable to read files from java archive: unable to open zip archive (/tmp/syft-archive-contents-3960168755/archive-example-java-app-maven-0.1.0.jar): cannot find beginning of zip archive="/tmp/syft-archive-contents-3960168755/archive-example-java-app-maven-0.1.0.jar" : zip: not a valid zip file
[0000]  WARN cataloger 'java-cataloger' failed to parse entries at location=Location<id=5 RealPath="/java/example-jenkins-plugin.hpi" VirtualPath="/java/example-jenkins-plugin.hpi" Layer="sha256:a908332de3f8ebb4f7c95fb6869b745325f8fe96bb4d1c472b263fba6b173529">: unable to read files from java archive: unable to open zip archive (/tmp/syft-archive-contents-2985024269/archive-example-jenkins-plugin.hpi): cannot find beginning of zip archive="/tmp/syft-archive-contents-2985024269/archive-example-jenkins-plugin.hpi" : zip: not a valid zip file
NAME              VERSION    TYPE
Pygments          2.6.1      python
apt               1.8.2      deb
bundler           2.1.4      gem
dash              0.5.8-2.4  deb
dive              0.9.2-1    rpm
libc-utils        0.7.2-r0   apk
musl-utils        1.1.24-r2  apk
netbase           5.4        deb
nikic/fast-route  v1.3.0     php-composer
npm               6.14.6     npm
psr/container     2.0.2      php-composer
psr/http-factory  1.0.1      php-composer
requests          2.22.0     python
someotherpkg      3.19.0     python
somerequests      3.22.0     python
unbundler         3.1.4      gem

ls -al /root/go/src/github.com/docker/sbom-cli-plugin/snapshot/sbom-cli-plugin_linux_amd64/docker-sbom

-rwxr-xr-x 1 root root 21733376 Jun 23  2022 /root/go/src/github.com/docker/sbom-cli-plugin/snapshot/sbom-cli-plugin_linux_amd64/docker-sbom

/root/go/src/github.com/docker/sbom-cli-plugin/snapshot/sbom-cli-plugin_linux_amd64/docker-sbom sbom --version

sbom-cli-plugin 0.6.1-SNAPSHOT-b17d47d, build b17d47dc0b20061e7924e835716caef3c6cc6a46

Debug shows a little more: -

/root/go/src/github.com/docker/sbom-cli-plugin/snapshot/sbom-cli-plugin_linux_amd64/docker-sbom sbom --debug stereoscope-fixture-image-pkg-coverage:c531ccd41ba451da7aa4700ba89e889b9109c2841710bb0c80af91d10705b6d6 --format syft-table

[0000] DEBUG application config:
package:
  cataloger:
    enabled: true
    scope: squashed
  search-unindexed-archives: false
  search-indexed-archives: true
exclude: []
platform: ""
output: ""
format: syft-table
quiet: false
log:
  structured: false
  level: ""
  file: ""
debug: true

[0000]  INFO syft version: v0.46.3
[0000] DEBUG   ├── compiler: gc
[0000] DEBUG   ├── gitCommit: b17d47dc0b20061e7924e835716caef3c6cc6a46
[0000] DEBUG   ├── gitDescription: v0.6.1-2-gb17d47d-dirty
[0000] DEBUG   ├── goVersion: go1.19.4
[0000] DEBUG   ├── platform: linux/amd64
[0000] DEBUG   ├── syftVersion: v0.46.3
[0000] DEBUG   └── version: 0.6.1-SNAPSHOT-b17d47d
[0000] DEBUG image metadata: digest=sha256:22391dca0d1a510d5fcc9f4295848ce72bff55994ef808cbdfeeabfdc1d43843 mediaType=application/vnd.docker.distribution.manifest.v2+json tags=[stereoscope-fixture-image-pkg-coverage:c531ccd41ba451da7aa4700ba89e889b9109c2841710bb0c80af91d10705b6d6 stereoscope-fixture-image-pkg-coverage:latest] from-lib=stereoscope
[0000] DEBUG layer metadata: index=0 digest=sha256:a908332de3f8ebb4f7c95fb6869b745325f8fe96bb4d1c472b263fba6b173529 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip from-lib=stereoscope
[0000] DEBUG layer metadata: index=1 digest=sha256:cb90c02c204e8f97351fc204f67e5f432f733179629cc59215648e8c35520276 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip from-lib=stereoscope
[0000] DEBUG layer metadata: index=2 digest=sha256:aee5ab65d15f551ee339b0e75c1a732d2a59cc6e0bfd301139b454d8069b2b00 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip from-lib=stereoscope
[0000]  INFO could not identify distro
[0000]  INFO cataloging image
[0000] DEBUG cataloging with "ruby-gemspec-cataloger"
[0000] DEBUG discovered 2 packages
[0000] DEBUG cataloging with "python-package-cataloger"
[0000] DEBUG discovered 4 packages
[0000] DEBUG cataloging with "php-composer-installed-cataloger"
[0000] DEBUG discovered 3 packages
[0000] DEBUG cataloging with "javascript-package-cataloger"
[0000] DEBUG discovered 1 packages
[0000] DEBUG cataloging with "dpkgdb-cataloger"
[0000] DEBUG discovered 3 packages
[0000] DEBUG cataloging with "rpmdb-cataloger"
[0000] DEBUG discovered 1 packages
[0000] DEBUG cataloging with "java-cataloger"
[0000]  WARN cataloger 'java-cataloger' failed to parse entries at location=Location<id=4 RealPath="/java/example-java-app-maven-0.1.0.jar" VirtualPath="/java/example-java-app-maven-0.1.0.jar" Layer="sha256:a908332de3f8ebb4f7c95fb6869b745325f8fe96bb4d1c472b263fba6b173529">: unable to read files from java archive: unable to open zip archive (/tmp/syft-archive-contents-3683551109/archive-example-java-app-maven-0.1.0.jar): cannot find beginning of zip archive="/tmp/syft-archive-contents-3683551109/archive-example-java-app-maven-0.1.0.jar" : zip: not a valid zip file
[0000]  WARN cataloger 'java-cataloger' failed to parse entries at location=Location<id=5 RealPath="/java/example-jenkins-plugin.hpi" VirtualPath="/java/example-jenkins-plugin.hpi" Layer="sha256:a908332de3f8ebb4f7c95fb6869b745325f8fe96bb4d1c472b263fba6b173529">: unable to read files from java archive: unable to open zip archive (/tmp/syft-archive-contents-701429832/archive-example-jenkins-plugin.hpi): cannot find beginning of zip archive="/tmp/syft-archive-contents-701429832/archive-example-jenkins-plugin.hpi" : zip: not a valid zip file
[0000] DEBUG discovered 0 packages
[0000] DEBUG cataloging with "apkdb-cataloger"
[0000] DEBUG discovered 2 packages
[0000] DEBUG cataloging with "go-module-binary-cataloger"
[0000] DEBUG discovered 0 packages
[0000] DEBUG cataloging with "dotnet-deps-cataloger"
[0000] DEBUG discovered 0 packages
NAME              VERSION    TYPE
Pygments          2.6.1      python
apt               1.8.2      deb
bundler           2.1.4      gem
dash              0.5.8-2.4  deb
dive              0.9.2-1    rpm
libc-utils        0.7.2-r0   apk
musl-utils        1.1.24-r2  apk
netbase           5.4        deb
nikic/fast-route  v1.3.0     php-composer
npm               6.14.6     npm
psr/container     2.0.2      php-composer
psr/http-factory  1.0.1      php-composer
requests          2.22.0     python
someotherpkg      3.19.0     python
somerequests      3.22.0     python
unbundler         3.1.4      gem

I can reproduce this by installing the same version of syft : -

wget https://github.com/anchore/syft/releases/download/v0.46.3/syft_0.46.3_linux_amd64.deb

dpkg --install syft_0.46.3_linux_amd64.deb

syft --version

syft 0.46.3

syft stereoscope-fixture-image-pkg-coverage:c531ccd41ba451da7aa4700ba89e889b9109c2841710bb0c80af91d10705b6d6

 ✔ Loaded image
 ✔ Parsed image
 ✔ Cataloged packages      [16 packages]
[0000]  WARN cataloger 'java-cataloger' failed to parse entries at location=Location<id=5 RealPath="/java/example-jenkins-plugin.hpi" VirtualPath="/java/example-jenkins-plugin.hpi" Layer="sha256:a908332de3f8ebb4f7c95fb6869b745325f8fe96bb4d1c472b263fba6b173529">: unable to read files from java archive: unable to open zip archive (/tmp/syft-archive-contents-1650874893/archive-example-jenkins-plugin.hpi): cannot find beginning of zip archive="/tmp/syft-archive-contents-1650874893/archive-example-jenkins-plugin.hpi" : zip: not a valid zip file
[0000]  WARN cataloger 'java-cataloger' failed to parse entries at location=Location<id=4 RealPath="/java/example-java-app-maven-0.1.0.jar" VirtualPath="/java/example-java-app-maven-0.1.0.jar" Layer="sha256:a908332de3f8ebb4f7c95fb6869b745325f8fe96bb4d1c472b263fba6b173529">: unable to read files from java archive: unable to open zip archive (/tmp/syft-archive-contents-582683570/archive-example-java-app-maven-0.1.0.jar): cannot find beginning of zip archive="/tmp/syft-archive-contents-582683570/archive-example-java-app-maven-0.1.0.jar" : zip: not a valid zip file
NAME              VERSION    TYPE
Pygments          2.6.1      python
apt               1.8.2      deb
bundler           2.1.4      gem
dash              0.5.8-2.4  deb
dive              0.9.2-1    rpm
libc-utils        0.7.2-r0   apk
musl-utils        1.1.24-r2  apk
netbase           5.4        deb
nikic/fast-route  v1.3.0     php-composer
npm               6.14.6     npm
psr/container     2.0.2      php-composer
psr/http-factory  1.0.1      php-composer
requests          2.22.0     python
someotherpkg      3.19.0     python
somerequests      3.22.0     python
unbundler         3.1.4      gem

If I instead download/install the latest version of syft : -

dpkg --remove syft

wget https://github.com/anchore/syft/releases/download/v0.64.0/syft_0.64.0_linux_amd64.deb

dpkg --install syft_0.64.0_linux_amd64.deb

syft --version

syft 0.64.0

I don't see the same issue: -

syft stereoscope-fixture-image-pkg-coverage:c531ccd41ba451da7aa4700ba89e889b9109c2841710bb0c80af91d10705b6d6

 ✔ Loaded image
 ✔ Parsed image
 ✔ Cataloged packages      [16 packages]
NAME              VERSION    TYPE
Pygments          2.6.1      python
apt               1.8.2      deb
bundler           2.1.4      gem
dash              0.5.8-2.4  deb
dive              0.9.2-1    rpm
libc-utils        0.7.2-r0   apk
musl-utils        1.1.24-r2  apk
netbase           5.4        deb
nikic/fast-route  v1.3.0     php-composer
npm               6.14.6     npm
psr/container     2.0.2      php-composer
psr/http-factory  1.0.1      php-composer
requests          2.22.0     python
someotherpkg      3.19.0     python
somerequests      3.22.0     python
unbundler         3.1.4      gem

What you expected to happen:

The TestAllFormatsExpressible test should pass

How to reproduce it (as minimally and precisely as possible):

See above

Anything else we need to know?:

This only appears to fail thusly on Ubuntu Linux; testing syft v0.46.3 on macOS doesn't exhibit the same issue: -

wget https://github.com/anchore/syft/releases/download/v0.46.3/syft_0.46.3_darwin_arm64.tar.gz

tar xvzf syft_0.46.3_darwin_arm64.tar.gz

./syft --version

syft 0.46.3

syft stereoscope-fixture-image-pkg-coverage:latest

 ✔ Loaded image
 ✔ Parsed image
 ✔ Cataloged packages      [20 packages]

NAME                    VERSION       TYPE
Pygments                2.6.1         python
apt                     1.8.2         deb
bundler                 2.1.4         gem
dash                    0.5.8-2.4     deb
dive                    0.9.2-1       rpm
example-java-app-maven  0.1.0         java-archive
example-jenkins-plugin  1.0-SNAPSHOT  jenkins-plugin
joda-time               2.9.2         java-archive
libc-utils              0.7.2-r0      apk
musl-utils              1.1.24-r2     apk
netbase                 5.4           deb
nikic/fast-route        v1.3.0        php-composer
npm                     6.14.6        npm
psr/container           2.0.2         php-composer
psr/http-factory        1.0.1         php-composer
requests                2.22.0        python
someotherpkg            3.19.0        python
somerequests            3.22.0        python
unbundler               3.1.4         gem

Failing Ubuntu

lsb_release -a

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.5 LTS
Release:    20.04
Codename:   focal

Working macOS

sw_vers

ProductName:        macOS
ProductVersion:     13.1
BuildVersion:       22C65

Environment:

docker version

Client:
 Version:           20.10.12
 API version:       1.41
 Go version:        go1.16.2
 Git commit:        20.10.12-0ubuntu2~20.04.1
 Built:             Wed Apr  6 02:14:38 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          20.10.12
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.2
  Git commit:       20.10.12-0ubuntu2~20.04.1
  Built:            Thu Feb 10 15:03:35 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.5.9-0ubuntu1~20.04.6
  GitCommit:
 runc:
  Version:          1.1.0-0ubuntu1~20.04.2
  GitCommit:
 docker-init:
  Version:          0.19.0
  GitCommit:

N/A

davidhay1969 commented 1 year ago

I tried a hack of "upgrading" syft in go.mod, changing from: -

        github.com/anchore/syft v0.46.3

to: -

        github.com/anchore/syft v0.64.0

but that broke go mod tidy

go: finding module for package github.com/anchore/syft/syft/logger
github.com/docker/sbom-cli-plugin/internal/log imports
    github.com/anchore/syft/syft/logger: module github.com/anchore/syft@latest found (v0.64.0), but does not contain package github.com/anchore/syft/syft/logger

and go get

github.com/docker/sbom-cli-plugin imports
    github.com/docker/sbom-cli-plugin/cmd imports
    github.com/docker/sbom-cli-plugin/internal/log imports
    github.com/anchore/syft/syft/logger: cannot find module providing package github.com/anchore/syft/syft/logger