search

anchore / syft

CLI tool and library for generating a Software Bill of Materials from container images and filesystems
Apache License 2.0
6.21k stars 572 forks source link

`make snapshot` fails locally on main branch #1923

Open luhring opened 1 year ago

luhring commented 1 year ago

What happened:

On a fresh clone of the main branch, running make snapshot (or anything that depends on it, like make compare-mac) fails

What you expected to happen:

Not fail 😃 (this makes it more difficult to contribute)

Steps to reproduce the issue:

$ git clone https://github.com/anchore/syft.git
Cloning into 'syft'...
remote: Enumerating objects: 20928, done.
remote: Counting objects: 100% (5031/5031), done.
remote: Compressing objects: 100% (1392/1392), done.
remote: Total 20928 (delta 4107), reused 3947 (delta 3576), pack-reused 15897
Receiving objects: 100% (20928/20928), 15.75 MiB | 14.74 MiB/s, done.
Resolving deltas: 100% (13945/13945), done.
$ cd syft
$ make bootstrap
mkdir -p ./.tmp
go mod download
curl -sSfL https://raw.githubusercontent.com/anchore/quill/main/install.sh | sh -s -- -b ./.tmp/ v0.2.0
[info] using release tag='v0.2.0' version='0.2.0' os='darwin' arch='arm64'
[info] installed ./.tmp//quill
GO111MODULE=off GOBIN=/Users/dan/tmp/syft/.tmp go get -u golang.org/x/perf/cmd/benchstat
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b ./.tmp/ v1.53.3
golangci/golangci-lint info checking GitHub for tag 'v1.53.3'
golangci/golangci-lint info found version: 1.53.3 for v1.53.3/darwin/arm64
golangci/golangci-lint info installed ./.tmp//golangci-lint
curl -sSfL https://raw.githubusercontent.com/wagoodman/go-bouncer/master/bouncer.sh | sh -s -- -b ./.tmp/ v0.4.0
wagoodman/go-bouncer info checking GitHub for tag 'v0.4.0'
wagoodman/go-bouncer info found version: 0.4.0 for v0.4.0/darwin/arm64
wagoodman/go-bouncer info installed ./.tmp//bouncer
curl -sSfL https://raw.githubusercontent.com/anchore/chronicle/main/install.sh | sh -s -- -b ./.tmp/ v0.6.0
anchore/chronicle info checking GitHub for tag 'v0.6.0'
anchore/chronicle info found version: 0.6.0 for v0.6.0/Darwin/all
anchore/chronicle info installed ./.tmp//chronicle
.github/scripts/goreleaser-install.sh -d -b ./.tmp/ v1.19.1
goreleaser/goreleaser info checking GitHub for tag 'v1.19.1'
goreleaser/goreleaser debug http_download https://github.com/goreleaser/goreleaser/releases/v1.19.1
goreleaser/goreleaser info found version: 1.19.1 for v1.19.1/Darwin/arm64
goreleaser/goreleaser debug downloading files into /var/folders/pz/h07rtjmd4mv4wpnr5st5q1480000gn/T/tmp.2fnLinTP
goreleaser/goreleaser debug http_download https://github.com/goreleaser/goreleaser/releases/download/v1.19.1/goreleaser_Darwin_arm64.tar.gz
goreleaser/goreleaser debug http_download https://github.com/goreleaser/goreleaser/releases/download/v1.19.1/checksums.txt
goreleaser/goreleaser info installed ./.tmp//goreleaser
goreleaser/goreleaser err this script is deprecated, please do not use it anymore. check https://github.com/goreleaser/godownloader/issues/207
# the only difference between goimports and gosimports is that gosimports removes extra whitespace between import blocks (see https://github.com/golang/go/issues/20818)
GOBIN="/Users/dan/tmp/syft/.tmp" go install github.com/rinchsan/gosimports/cmd/gosimports@v0.3.8
GOBIN="/Users/dan/tmp/syft/.tmp" go install github.com/neilpa/yajsv@v1.4.1
GOBIN="/Users/dan/tmp/syft/.tmp" go install github.com/sigstore/cosign/v2/cmd/cosign@v2.1.1
GOBIN="/Users/dan/tmp/syft/.tmp" go install github.com/charmbracelet/glow@v1.5.1
Bootstrapping dependencies
$ make compare-mac
Building snapshot artifacts
# create a config with the dist dir overridden
echo "dist: ./snapshot" > ./.tmp/goreleaser.yaml
cat .goreleaser.yaml >> ./.tmp/goreleaser.yaml
# build release snapshots
./.tmp/goreleaser release --clean --skip-publish --skip-sign --snapshot --config ./.tmp/goreleaser.yaml
  • starting release...
  • loading config file                              file=./.tmp/goreleaser.yaml
  • loading environment variables
  • getting and validating git state
    • building...                                    commit=376c42893b38a68e9703470d9e625bf98612a1d4 latest tag=v0.84.1
    • pipe skipped                                   reason=disabled during snapshot mode
  • parsing tag
  • setting defaults
      • DEPRECATED: `brews.tap` should not be used anymore, check https://goreleaser.com/deprecations#brewstap for more info
  • snapshotting
    • building snapshot...                           version=0.84.1-SNAPSHOT-376c4289
  • checking distribution directory
  • loading go mod information
  • build prerequisites
  • writing effective config file
    • writing                                        config=snapshot/config.yaml
  • building binaries
    • building                                       binary=snapshot/windows-build_windows_amd64_v1/syft.exe
    • building                                       binary=snapshot/darwin-build_darwin_amd64_v1/syft
    • building                                       binary=snapshot/darwin-build_darwin_arm64/syft
    • building                                       binary=snapshot/linux-build_linux_ppc64le/syft
    • building                                       binary=snapshot/linux-build_linux_s390x/syft
    • building                                       binary=snapshot/linux-build_linux_amd64_v1/syft
    • building                                       binary=snapshot/linux-build_linux_arm64/syft
    • running hook                                   hook=.tmp/quill sign-and-notarize "/Users/dan/tmp/syft/snapshot/darwin-build_darwin_amd64_v1/syft" --dry-run=true --ad-hoc=true -vv
    • running hook                                   hook=.tmp/quill sign-and-notarize "/Users/dan/tmp/syft/snapshot/darwin-build_darwin_arm64/syft" --dry-run=true --ad-hoc=true -vv
    • took: 5s
  • archives
    • creating                                       archive=snapshot/syft_0.84.1-SNAPSHOT-376c4289_linux_ppc64le.tar.gz
    • creating                                       archive=snapshot/syft_0.84.1-SNAPSHOT-376c4289_linux_arm64.tar.gz
    • creating                                       archive=snapshot/syft_0.84.1-SNAPSHOT-376c4289_linux_amd64.tar.gz
    • creating                                       archive=snapshot/syft_0.84.1-SNAPSHOT-376c4289_linux_s390x.tar.gz
    • creating                                       archive=snapshot/syft_0.84.1-SNAPSHOT-376c4289_darwin_arm64.tar.gz
    • creating                                       archive=snapshot/syft_0.84.1-SNAPSHOT-376c4289_darwin_amd64.tar.gz
    • creating                                       archive=snapshot/syft_0.84.1-SNAPSHOT-376c4289_windows_amd64.zip
    • took: 4s
  • linux packages
    • creating                                       package=syft format=rpm arch=arm64 file=snapshot/syft_0.84.1-SNAPSHOT-376c4289_linux_arm64.rpm
    • creating                                       package=syft format=deb arch=arm64 file=snapshot/syft_0.84.1-SNAPSHOT-376c4289_linux_arm64.deb
    • creating                                       package=syft format=deb arch=ppc64le file=snapshot/syft_0.84.1-SNAPSHOT-376c4289_linux_ppc64le.deb
    • creating                                       package=syft format=rpm arch=amd64v1 file=snapshot/syft_0.84.1-SNAPSHOT-376c4289_linux_amd64.rpm
    • creating                                       package=syft format=rpm arch=s390x file=snapshot/syft_0.84.1-SNAPSHOT-376c4289_linux_s390x.rpm
    • creating                                       package=syft format=deb arch=s390x file=snapshot/syft_0.84.1-SNAPSHOT-376c4289_linux_s390x.deb
    • creating                                       package=syft format=rpm arch=ppc64le file=snapshot/syft_0.84.1-SNAPSHOT-376c4289_linux_ppc64le.rpm
    • creating                                       package=syft format=deb arch=amd64v1 file=snapshot/syft_0.84.1-SNAPSHOT-376c4289_linux_amd64.deb
    • took: 1s
  • calculating checksums
  • homebrew tap formula
    • guessing install to be "bin.install \"syft\""
    • guessing install to be "bin.install \"syft\""
    • guessing install to be "bin.install \"syft\""
    • guessing install to be "bin.install \"syft\""
    • writing                                        formula=snapshot/homebrew/syft.rb
  • docker images
    • building docker image                          image=anchore/syft:v0.84.1-ppc64le
    • building docker image                          image=anchore/syft:v0.84.1-arm64v8
    • building docker image                          image=anchore/syft:debug-arm64v8
    • building docker image                          image=anchore/syft:debug-ppc64le
    • building docker image                          image=anchore/syft:latest
    • building docker image                          image=anchore/syft:debug
    • building docker image                          image=anchore/syft:v0.84.1-s390x
    • building docker image                          image=anchore/syft:debug-s390x
  ⨯ release failed after 10s                 error=docker build failed: failed to build anchore/syft:v0.84.1-s390x: exit status 1: ERROR: use `docker --context=default buildx` to switch to context "default"

Learn more at https://goreleaser.com/errors/docker-build

make: *** [snapshot] Error 1

Anything else we need to know?:

Environment:

macOS 13.4.1 (22F82) (Apple M1 Max)

$ docker version
Client:
 Cloud integration: v1.0.35
 Version:           24.0.2
 API version:       1.43
 Go version:        go1.20.4
 Git commit:        cb74dfc
 Built:             Thu May 25 21:51:16 2023
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.21.1 (114176)
 Engine:
  Version:          24.0.2
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.4
  Git commit:       659604f
  Built:            Thu May 25 21:50:59 2023
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.21
  GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
 runc:
  Version:          1.1.7
  GitCommit:        v1.1.7-0-g860f061
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
$ go version
go version go1.20.5 darwin/arm64
luhring commented 1 year ago

This can be reproduced with just make snapshot, so I'll update the title. (Found this when looking into a strange acceptance test failure in CI for #1897)

tgerla commented 1 year ago

Starting to look at this--I couldn't reproduce the failure until I updated Docker to the latest version:

Worked:

tgerla@Timothys-MacBook-Pro-2 syft % docker version
Client:
 Cloud integration: v1.0.31
 Version:           23.0.5
 API version:       1.42
 Go version:        go1.19.8
 Git commit:        bc4487a
 Built:             Wed Apr 26 16:12:52 2023
 OS/Arch:           darwin/arm64
 Context:           default

Server: Docker Desktop 4.19.0 (106363)
 Engine:
  Version:          23.0.5
  API version:      1.42 (minimum version 1.12)
  Go version:       go1.19.8
  Git commit:       94d3ad6
  Built:            Wed Apr 26 16:17:14 2023
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.20
  GitCommit:        2806fc1057397dbaeefbea0e4e17bddfbd388f38
 runc:
  Version:          1.1.5
  GitCommit:        v1.1.5-0-gf19387a
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Doesn't work:

tgerla@Timothys-MacBook-Pro-2 syft % docker version
Client:
 Cloud integration: v1.0.35
 Version:           24.0.2
 API version:       1.43
 Go version:        go1.20.4
 Git commit:        cb74dfc
 Built:             Thu May 25 21:51:16 2023
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.21.1 (114176)
 Engine:
  Version:          24.0.2
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.4
  Git commit:       659604f
  Built:            Thu May 25 21:50:59 2023
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.21
  GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
 runc:
  Version:          1.1.7
  GitCommit:        v1.1.7-0-g860f061
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
tgerla commented 1 year ago

@luhring, I think maybe I found at least a workaround to this. I ran:

docker context use default

...and then re-ran make snapshot and it completed successfully. FWIW, my contexts look like this:

tgerla@Timothys-MacBook-Pro-2 syft % docker context list
NAME                TYPE                DESCRIPTION                               DOCKER ENDPOINT                                KUBERNETES ENDPOINT   ORCHESTRATOR
default *           moby                Current DOCKER_HOST based configuration   unix:///var/run/docker.sock
desktop-linux       moby                Docker Desktop                            unix:///Users/tgerla/.docker/run/docker.sock

I don't know much about goreleaser so I'm not sure if there is a fix we can make in the build tooling here. Thoughts appreciated.

luhring commented 1 year ago

Thanks @tgerla! That workaround works for me. And TIL about Docker "contexts"...

I also noticed that because Syft defines the make target as the snapshot directory, make will think the snapshot has been completed successfully even when this failure happens. I'm not sure of a way to fix this, since goreleaser is doing so many things in one pass.

I'll let you know if I find a better fix for this one!