Attesting with Syft fails

hdiederichs commented 1 year ago

What happened: When trying to attest an image as described in the README.md it fails with exit status 1 and no further information about what went wrong.

What you expected to happen: SBOM should be created. If it is not possible to create an SBOM, the error message should give a hint, what is missing.

Steps to reproduce the issue: I have installed cosign and syft (for versions see below). Perform the following commands to reproduce:

cd ~
cosign generate-key-pair
MY_PRIVATE_KEY=~/cosign.key
COSIGN_PASSWORD=hunter2
syft attest --key $MY_PRIVATE_KEY -o spdx-json docker.io/alpine:latest > alpine_latest_sbom_attestation.json

Output of the last command with -vv:

[0000]  INFO syft version: 0.76.1
[0000] DEBUG application config:
configPath: /home/azurediamond/config.json
verbosity: 2
quiet: false
output:
- spdx-json
output-template-path: ""
file: ""
check-for-app-update: true
dev:
  profile-cpu: false
  profile-mem: false
log:
  structured: false
  level: debug
  file: ""
catalogers: []
package:
  cataloger:
    enabled: true
    scope: Squashed
  search-unindexed-archives: false
  search-indexed-archives: true
golang:
  search-local-mod-cache-licenses: false
  local-mod-cache-dir: ""
attest: {}
file-metadata:
  cataloger:
    enabled: false
    scope: Squashed
  digests:
  - sha256
file-classification:
  cataloger:
    enabled: false
    scope: Squashed
file-contents:
  cataloger:
    enabled: false
    scope: Squashed
  skip-files-above-size: 1048576
  globs: []
secrets:
  cataloger:
    enabled: false
    scope: AllLayers
  additional-patterns: {}
  exclude-pattern-names: []
  reveal-values: false
  skip-files-above-size: 1048576
registry:
  insecure-skip-tls-verify: false
  insecure-use-http: false
  auth: []
exclude: []
platform: ""
name: foo
parallelism: 1
default-image-pull-source: ""

[0000] DEBUG checking if a new version of syft is available
[0000] DEBUG no new syft update available
[0000] DEBUG image: source=DockerDaemon location=docker.io/alpine:latest from-lib=stereoscope
[0000] DEBUG image metadata: digest=sha256:9ed4aefc74f6792b5a804d1d146fe4b4a2299147b0f50eaf2b08435d7b38c27e mediaType=application/vnd.docker.distribution.manifest.v2+json tags=[alpine:latest] from-lib=stereoscope
[0000] DEBUG layer metadata: index=0 digest=sha256:f1417ff83b319fbdae6dd9cd6d8c9c88002dcd75ecf6ec201c8c6894681cf2b5 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip from-lib=stereoscope
[0000]  INFO identified distro: Alpine Linux v3.17
[0000]  INFO cataloging image
[0000] DEBUG cataloging packages catalogers=16 parallelism=1
[0000] DEBUG discovered 0 packages cataloger=alpmdb-cataloger
[0000] DEBUG discovered 0 packages cataloger=ruby-gemspec-cataloger
[0000] DEBUG discovered 0 packages cataloger=python-package-cataloger
[0000] DEBUG discovered 0 packages cataloger=php-composer-installed-cataloger
[0000] DEBUG discovered 0 packages cataloger=javascript-package-cataloger
[0000] DEBUG discovered 0 packages cataloger=dpkgdb-cataloger
[0000] DEBUG discovered 0 packages cataloger=rpm-db-cataloger
[0000] DEBUG discovered 0 packages cataloger=java-cataloger
[0000] DEBUG discovered 0 packages cataloger=graalvm-native-image-cataloger
[0000] DEBUG discovered 15 packages cataloger=apkdb-cataloger
[0000] DEBUG found path duplicate of /lib/ld-musl-x86_64.so.1
[0000] DEBUG found path duplicate of /usr/share/apk/keys/alpine-devel@lists.alpinelinux.org-58199dcc.rsa.pub
[0000] DEBUG found path duplicate of /usr/share/apk/keys/alpine-devel@lists.alpinelinux.org-616ae350.rsa.pub
[0000] DEBUG found path duplicate of /usr/share/apk/keys/alpine-devel@lists.alpinelinux.org-524d27bb.rsa.pub
[0000] DEBUG found path duplicate of /usr/share/apk/keys/alpine-devel@lists.alpinelinux.org-616a9724.rsa.pub
[0000] DEBUG found path duplicate of /usr/share/apk/keys/alpine-devel@lists.alpinelinux.org-524d27bb.rsa.pub
[0000] DEBUG found path duplicate of /usr/share/apk/keys/alpine-devel@lists.alpinelinux.org-616adfeb.rsa.pub
[0000] DEBUG found path duplicate of /usr/share/apk/keys/alpine-devel@lists.alpinelinux.org-5e69ca50.rsa.pub
[0000] DEBUG found path duplicate of /usr/share/apk/keys/alpine-devel@lists.alpinelinux.org-58cbb476.rsa.pub
[0000] DEBUG found path duplicate of /usr/share/apk/keys/alpine-devel@lists.alpinelinux.org-616abc23.rsa.pub
[0000] DEBUG found path duplicate of /usr/share/apk/keys/alpine-devel@lists.alpinelinux.org-60ac2099.rsa.pub
[0000] DEBUG found path duplicate of /usr/share/apk/keys/alpine-devel@lists.alpinelinux.org-616db30d.rsa.pub
[0000] DEBUG found path duplicate of /usr/share/apk/keys/alpine-devel@lists.alpinelinux.org-58e4f17d.rsa.pub
[0000] DEBUG found path duplicate of /usr/share/apk/keys/alpine-devel@lists.alpinelinux.org-616ac3bc.rsa.pub
[0000] DEBUG found path duplicate of /usr/share/apk/keys/alpine-devel@lists.alpinelinux.org-4a6a0840.rsa.pub
[0000] DEBUG found path duplicate of /usr/share/apk/keys/alpine-devel@lists.alpinelinux.org-5243ef4b.rsa.pub
[0000] DEBUG found path duplicate of /usr/share/apk/keys/alpine-devel@lists.alpinelinux.org-61666e3f.rsa.pub
[0000] DEBUG found path duplicate of /usr/share/apk/keys/alpine-devel@lists.alpinelinux.org-4a6a0840.rsa.pub
[0000] DEBUG found path duplicate of /usr/share/apk/keys/alpine-devel@lists.alpinelinux.org-5261cecb.rsa.pub
[0000] DEBUG found path duplicate of /usr/share/apk/keys/alpine-devel@lists.alpinelinux.org-6165ee59.rsa.pub
[0000] DEBUG found path duplicate of /etc/ssl/certs/ca-certificates.crt
[0000] DEBUG found path duplicate of /etc/ssl/certs/ca-certificates.crt
[0000] DEBUG found path duplicate of /etc/ssl/misc/tsget.pl
[0000] DEBUG found path duplicate of /lib/libcrypto.so.3
[0000] DEBUG found path duplicate of /lib/libssl.so.3
[0000] DEBUG found path duplicate of /lib/libz.so.1.2.13
[0000] DEBUG discovered 0 packages cataloger=go-module-binary-cataloger
[0000] DEBUG discovered 0 packages cataloger=dotnet-deps-cataloger
[0000] DEBUG discovered 0 packages cataloger=portage-cataloger
[0000] DEBUG discovered 0 packages cataloger=nix-store-cataloger
[0000] DEBUG discovered 0 packages cataloger=sbom-cataloger
[0000] DEBUG discovered 1 packages cataloger=binary-cataloger
[0000] DEBUG skipping non-package relationship: {From:0xc000103ba0 To:Pkg(name="busybox-binsh" version="1.35.0-r29" type="apk" id="256fc96b4a8c4da8") Type:contains Data:<nil>}
[0000] DEBUG skipping non-package relationship: {From:0xc000103ba0 To:Pkg(name="alpine-baselayout-data" version="3.4.0-r0" type="apk" id="291d1267b40d636f") Type:contains Data:<nil>}
[0000] DEBUG skipping non-package relationship: {From:0xc000103ba0 To:Pkg(name="libssl3" version="3.0.8-r3" type="apk" id="2a95f0251fba7a33") Type:contains Data:<nil>}
[0000] DEBUG skipping non-package relationship: {From:0xc000103ba0 To:Pkg(name="alpine-keys" version="2.4-r1" type="apk" id="2b5e23d349b556cf") Type:contains Data:<nil>}
[0000] DEBUG skipping non-package relationship: {From:0xc000103ba0 To:Pkg(name="busybox" version="1.35.0-r29" type="apk" id="623d53216342d45e") Type:contains Data:<nil>}
[0000] DEBUG skipping non-package relationship: {From:0xc000103ba0 To:Pkg(name="libc-utils" version="0.7.2-r3" type="apk" id="8126b232e2d3c608") Type:contains Data:<nil>}
[0000] DEBUG skipping non-package relationship: {From:0xc000103ba0 To:Pkg(name="busybox" version="1.35.0" type="binary" id="84d19405d71391ee") Type:contains Data:<nil>}
[0000] DEBUG skipping non-package relationship: {From:0xc000103ba0 To:Pkg(name="alpine-baselayout" version="3.4.0-r0" type="apk" id="92b19c7750fb559d") Type:contains Data:<nil>}
[0000] DEBUG skipping non-package relationship: {From:0xc000103ba0 To:Pkg(name="zlib" version="1.2.13-r0" type="apk" id="94014313cfcd2b71") Type:contains Data:<nil>}
[0000] DEBUG skipping non-package relationship: {From:0xc000103ba0 To:Pkg(name="ssl_client" version="1.35.0-r29" type="apk" id="b15247aafcd4a647") Type:contains Data:<nil>}
[0000] DEBUG skipping non-package relationship: {From:0xc000103ba0 To:Pkg(name="ca-certificates-bundle" version="20220614-r4" type="apk" id="b805d823ae624f04") Type:contains Data:<nil>}
[0000] DEBUG skipping non-package relationship: {From:0xc000103ba0 To:Pkg(name="libcrypto3" version="3.0.8-r3" type="apk" id="d3084c788891fb28") Type:contains Data:<nil>}
[0000] DEBUG skipping non-package relationship: {From:0xc000103ba0 To:Pkg(name="musl" version="1.2.3-r4" type="apk" id="d9700f02cf26e8b8") Type:contains Data:<nil>}
[0000] DEBUG skipping non-package relationship: {From:0xc000103ba0 To:Pkg(name="apk-tools" version="2.12.10-r1" type="apk" id="e5f757b0df1f62bc") Type:contains Data:<nil>}
[0000] DEBUG skipping non-package relationship: {From:0xc000103ba0 To:Pkg(name="scanelf" version="1.3.5-r1" type="apk" id="e903138d19e85b80") Type:contains Data:<nil>}
[0000] DEBUG skipping non-package relationship: {From:0xc000103ba0 To:Pkg(name="musl-utils" version="1.2.3-r4" type="apk" id="f71ecf5267e6c37b") Type:contains Data:<nil>}
2023/04/06 10:06:48 error during command execution: 1 error occurred:
        * unable to attest SBOM: exit status 1

Anything else we need to know?: Ubuntu distro is running inside WSL 2.

Environment:

Output of syft version:

Application:        syft
Version:            0.76.1
JsonSchemaVersion:  7.1.1
BuildDate:          2023-04-05T18:13:51Z
GitCommit:          7845381331e873f65fd5013f44b7d85168ced5f5
GitDescription:     v0.76.1
Platform:           linux/amd64
GoVersion:          go1.19.7
Compiler:           gc

Output of cosign version:

GitVersion:    v2.0.0
GitCommit:     d6b9001f8e6ed745fb845849d623274c897d55f2
GitTreeState:  clean
BuildDate:     2023-02-23T19:26:35Z
GoVersion:     go1.20.1
Compiler:      gc
Platform:      linux/amd64

OS (e.g: cat /etc/os-release or similar):

PRETTY_NAME="Ubuntu 22.04.2 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.2 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

hdiederichs commented 1 year ago

Maybe these steps to reproduce help:

cat > Dockerfile << 'EOF'
FROM gcr.io/projectsigstore/cosign:v2.0.1 AS cosign
FROM docker.io/anchore/syft:v0.77.0 AS syft
FROM docker.io/alpine:3.17.3

COPY --from=cosign /ko-app/cosign /usr/local/bin/cosign
COPY --from=syft /syft /usr/local/bin/syft

ENV REGISTRY="docker.io"
ENV REPOSITORY="anchore"
ENV IMAGE_TAG="latest"
ENV USERNAME="AzureDiamond"
ENV REGISTRY_PASSWORD="hunter2"
ENV IMAGE_NAME="syft"
ENV COSIGN_PASSWORD="hunter2too"

RUN cosign generate-key-pair

CMD syft login -u $USERNAME -p $REGISTRY_PASSWORD -vv $REGISTRY && syft attest -o cyclonedx-json --key cosign.key -vv $REGISTRY/$REPOSITORY/$IMAGE_NAME:$IMAGE_TAG > $IMAGE_NAME-$IMAGE_TAG.json
EOF
docker buildx build --load -t test/syft .
docker run -e "USERNAME=yourname" -e "REGISTRY_PASSWORD=s3cret1234!" test/syft

At least I run into the same problem when using this approach.

tgerla commented 1 year ago

Hi @hdiederichs, thanks for the report. Our attestation mechanism requires that you have OCI write access to the image that you are working with. I think that is the main problem that you're running into. It does look like we are either missing an error message or maybe it didn't get copied into this issue. We will try to reproduce this and see if we can surface the correct error message.

hdiederichs commented 1 year ago

I didn't omit any output from the debug logs mentioned above.

Currently, my workaround is as follows:

createSbomAndSign() {
  # some other stuff
  syft -o cyclonedx-json -v "$REGISTRY"/"$REPOSITORY"/"$1"@"$SHA_SUM" > sboms/"$1"_"$IMAGE_TAG".json
  cosign attach sbom --sbom sboms/"$1"_"$IMAGE_TAG".json "$REGISTRY"/"$REPOSITORY"/"$1"@"$SHA_SUM"
}

This seemingly does the same thing and doesn't fail.

holongate commented 1 year ago

I got a related error: syft attest --key cosign.key -o spdx-json ubuntu:22.04 --file sbom-attest.json

Creating attestation    [running cosign]
     ░░         This may include the email address associated with the account with which you authenticate your contractual Agreement.
     ░░         This information will be used for signing this artifact and will be stored in public transparency logs and cannot be removed later, and is subject to the Immutable Record notice at https://lfprojects.org/policies/hosted-project-tools-immutable-records/.
     ░░ 
     ░░ By typing 'y', you attest that (1) you are not submitting the personal data of any other person; and (2) you understand and agree to the statement and the Agreement terms at the URLs listed above.
     ░░ Are you sure you would like to continue? [y/N] Error: signing ubuntu:22.04: should upload to tlog: user declined the prompt
     ░░ main.go:74: error during command execution: signing ubuntu:22.04: should upload to tlog: user declined the prompt
2023/06/06 22:22:52 error during command execution: 1 error occurred:
        * unable to attest SBOM: exit status 1

But never got a chance to type anything as the prompt is never waiting for a the user input and the output file is empty.

I might add that the logging formatting is sexy but the output is totally garbled, and unhelpful most of the time.

turing85 commented 1 year ago

In the most recent version, it seems that local attestation is completely gone. Is this correct?

fpoirotte commented 1 year ago

Newer versions of cosign try to upload attestations to a "transparency log" be default. That's the reason for the prompt and subsequent failure noted by @holongate.

--tlog-upload=false needs to be added to the cosign attest command generated in https://github.com/anchore/syft/blob/main/cmd/syft/cli/commands/attest.go#L216 for attestation to work again.

Downgrading to an earlier version of cosign would probably work too, although I don't know when exactly it was introduced (I tested this with cosign 2.1.1 where the issue related to --tlog-upload=false was already present; I did not have the chance to test earlier versions).

anchore / syft

Attesting with Syft fails #1718