anchore / syft

CLI tool and library for generating a Software Bill of Materials from container images and filesystems
Apache License 2.0
5.73k stars 526 forks source link

Syft tries to create the cache directory at a location that has no permission #2984

Closed Joerki closed 4 days ago

Joerki commented 1 week ago

What happened: Hi @kzantow,

the new cache implementation is partially not working. Without changing my previous configuration (#2798). If, in a Docker container syft tries to create the cache subdirectory at an unusual place (root dir, $HOME is missing), this operation failes and syft crashes later on.

When I add -e HOME=/tmp to the docker command line it does not crash.

[0000] WARN unable to get filesystem cache at /.cache/syft: unable to create directory at '/.cache/syft//.cache/syft': mkdir /.cache: permission denied

What you expected to happen: Syft should check writing permissions of the directory where cache shall be created and reports the problem with explicit directory when this operation fails.

Steps to reproduce the issue: Set HOME to a directory without write permissions. Scan an Docker image with Go binaries (like mongo) Enable settings to fetch packages to identify licenses (see settings below).

I created a Docker image with syft binary to use it in differend environments. In the case that HOME is "/", syft crashes. If I change HOME to "/" in my usual, local environment (WSL), syft is reporting the warning, but does not crash.

Anything else we need to know?:

[0000]  WARN unable to get filesystem cache at /.cache/syft: unable to create directory at '/.cache/syft//.cache/syft': mkdir /.cache: permission denied
[0000]  WARN unable to get filesystem cache at /.cache/syft: unable to create directory at '/.cache/syft//.cache/syft': mkdir /.cache: permission denied
[0000]  INFO syft version: 1.7.0
[0000] DEBUG config:
  log:
      quiet: false
      level: debug
      file: ""
  dev:
      profile: none
  config: .syft.yaml
  output:
      - syft-table
  format:
      pretty: null
      template:
          path: ""
          legacy: false
      json:
          legacy: false
          pretty: true
      spdx-json:
          pretty: false
      cyclonedx-json:
          pretty: false
      cyclonedx-xml:
          pretty: false
  check-for-app-update: false
  default-catalogers: []
  select-catalogers: []
  package:
      search-unindexed-archives: false
      search-indexed-archives: true
      exclude-binary-overlap-by-ownership: true
  file:
      metadata:
          selection: owned-by-package
          digests:
              - sha1
              - sha256
      content:
          skip-files-above-size: 256000
          globs: []
      executable:
          globs: []
  scope: all-layers
  parallelism: 1
  relationships:
      package-file-ownership: true
      package-file-ownership-overlap: true
  golang:
      search-local-mod-cache-licenses: true
      local-mod-cache-dir: /go/pkg/mod
      search-remote-licenses: true
      proxy: https://proxy.golang.org,direct
      no-proxy: ""
      main-module-version:
          from-ld-flags: true
          from-contents: true
          from-build-settings: true
  java:
      use-network: false
      maven-url: ""
      max-parent-recursive-depth: 0
  javascript:
      search-remote-licenses: false
      npm-base-url: ""
  linux-kernel:
      catalog-modules: true
  python:
      guess-unpinned-requirements: false
  registry:
      insecure-skip-tls-verify: false
      insecure-use-http: false
      auth: []
      ca-cert: ""
  from: []
  platform: ""
  source:
      name: ""
      version: ""
      base-path: ""
      file:
          digests:
              - SHA-256
      image:
          default-pull-source: ""
  exclude: []
  cache:
      dir: /.cache/syft
      ttl: 7d
failed to run tasks: 1 error occurred:
        * failed to run task: runtime error: invalid memory address or nil pointer dereference at:
goroutine 14 [running]:
runtime/debug.Stack()
        /opt/hostedtoolcache/go/1.22.4/x64/src/runtime/debug/stack.go:24 +0x5e
github.com/anchore/syft/internal/task.runTaskSafely.func1()
        /home/runner/work/syft/syft/internal/task/executor.go:67 +0x3d
panic({0x174b720?, 0x2d5a080?})
        /opt/hostedtoolcache/go/1.22.4/x64/src/runtime/panic.go:770 +0x132
github.com/anchore/syft/syft/pkg/cataloger/golang.(*goLicenseResolver).findLicensesInFS.func1({0x1f5fc08, 0x1}, {0x0?, 0x0?}, {0xc0001036c0?, 0x38?})
        /home/runner/work/syft/syft/syft/pkg/cataloger/golang/licenses.go:142 +0x97
io/fs.WalkDir({0x1f6e9a0, 0xc005c40c20}, {0x1f5fc08, 0x1}, 0xc0024dc948)
        /opt/hostedtoolcache/go/1.22.4/x64/src/io/fs/walk.go:120 +0xcc
github.com/anchore/syft/syft/pkg/cataloger/golang.(*goLicenseResolver).findLicensesInFS(0x1f6e740?, {0xc00430ad00?, 0xc003992ba0?}, {0x1f6e9a0?, 0xc005c40c20?})
        /home/runner/work/syft/syft/syft/pkg/cataloger/golang/licenses.go:141 +0x9c
github.com/anchore/syft/syft/pkg/cataloger/golang.(*goLicenseResolver).getLicensesFromLocal(0xc0023e94a0, {0xc003c6b85c?, 0xc000126720?}, {0xc003c6b876?, 0x30?})
        /home/runner/work/syft/syft/syft/pkg/cataloger/golang/licenses.go:123 +0xe5
github.com/anchore/syft/syft/pkg/cataloger/golang.(*goLicenseResolver).getLicenses(0xc0023e94a0, {0x1f88300, 0xc000126720}, {0xc003c6b85c, 0x19}, {0xc003c6b876, 0x6})
        /home/runner/work/syft/syft/syft/pkg/cataloger/golang/licenses.go:93 +0x13b
github.com/anchore/syft/syft/pkg/cataloger/golang.(*goBinaryCataloger).newGoBinaryPackage(_, {_, _}, _, {_, _}, {_, _}, {0xc002fdd566, 0x6}, ...)
        /home/runner/work/syft/syft/syft/pkg/cataloger/golang/package.go:18 +0xa6
github.com/anchore/syft/syft/pkg/cataloger/golang.(*goBinaryCataloger).buildGoPkgInfo(0xc0023e94a0, {0x1f88300, 0xc000126720}, {{{{0xc001419128, 0x11}, {0xc000b214a0, 0x47}}, {0xc0059c0cf0, 0x11}, {0x1102, ...}}, ...}, ...)
        /home/runner/work/syft/syft/syft/pkg/cataloger/golang/parse_go_binary.go:117 +0x409
github.com/anchore/syft/syft/pkg/cataloger/golang.(*goBinaryCataloger).parseGoBinary(0xc0023e94a0, {0x3e8?, 0x3e8?}, {0x1f88300, 0xc000126720}, 0x17?, {{{{{...}, {...}}, {0xc0059c0cf0, 0x11}, ...}, ...}, ...})
        /home/runner/work/syft/syft/syft/pkg/cataloger/golang/parse_go_binary.go:75 +0x2a5
github.com/anchore/syft/syft/pkg/cataloger/generic.invokeParser({0x1f7f060, 0xc0003eeaf0}, {0x1f88300, 0xc000126720}, {{{{0xc001419128, 0x11}, {0xc000b214a0, 0x47}}, {0xc0059c0cf0, 0x11}, ...}, ...}, ...)
        /home/runner/work/syft/syft/syft/pkg/cataloger/generic/cataloger.go:197 +0x43e
github.com/anchore/syft/syft/pkg/cataloger/generic.(*Cataloger).Catalog(0xc000124f00, {0x1f7f060, 0xc0003eeaf0}, {0x1f88300, 0xc000126720})
        /home/runner/work/syft/syft/syft/pkg/cataloger/generic/cataloger.go:167 +0x4f8
github.com/anchore/syft/internal/task.NewPackageTask.func1({0x1f7f060, 0xc0003eeaf0}, {0x1f88300, 0xc000126720}, {0x1f7f990, 0xc0024281e0})
        /home/runner/work/syft/syft/internal/task/package_task_factory.go:103 +0x267
github.com/anchore/syft/internal/task.task.Execute(...)
        /home/runner/work/syft/syft/internal/task/task.go:64
github.com/anchore/syft/internal/task.runTaskSafely({0x1f7f060?, 0xc0003eeaf0?}, {0x1f77ea8?, 0xc000126080?}, {0x1f88300?, 0xc000126720?}, {0x1f7f990?, 0xc0024281e0?})
        /home/runner/work/syft/syft/internal/task/executor.go:71 +0xa7
github.com/anchore/syft/internal/task.(*Executor).Execute.func1()
        /home/runner/work/syft/syft/internal/task/executor.go:49 +0x131
created by github.com/anchore/syft/internal/task.(*Executor).Execute in goroutine 37
        /home/runner/work/syft/syft/internal/task/executor.go:40 +0xa5

Environment:

Joerki commented 6 days ago

It is also crashing on windows

[0000]  INFO syft version: 1.7.0
[0000] DEBUG config:
  log:
      quiet: false
      level: debug
      file: ""
  dev:
      profile: none
  config: .syft.yaml
  output:
      - syft-json=mongo.json
  format:
      pretty: null
      template:
          path: ""
          legacy: false
      json:
          legacy: false
          pretty: true
      spdx-json:
          pretty: false
      cyclonedx-json:
          pretty: false
      cyclonedx-xml:
          pretty: false
  check-for-app-update: false
  default-catalogers: []
  select-catalogers: []
  package:
      search-unindexed-archives: false
      search-indexed-archives: true
      exclude-binary-overlap-by-ownership: true
  file:
      metadata:
          selection: owned-by-package
          digests:
              - sha1
              - sha256
      content:
          skip-files-above-size: 256000
          globs: []
      executable:
          globs: []
  scope: all-layers
  parallelism: 1
  relationships:
      package-file-ownership: true
      package-file-ownership-overlap: true
  golang:
      search-local-mod-cache-licenses: true
      local-mod-cache-dir: C:\Users\w010701\go\pkg\mod
      search-remote-licenses: true
      proxy: https://proxy.golang.org,direct
      no-proxy: ""
      main-module-version:
          from-ld-flags: true
          from-contents: true
          from-build-settings: true
  java:
      use-network: false
      maven-url: ""
      max-parent-recursive-depth: 0
  javascript:
      search-remote-licenses: false
      npm-base-url: ""
  linux-kernel:
      catalog-modules: true
  python:
      guess-unpinned-requirements: false
  registry:
      insecure-skip-tls-verify: false
      insecure-use-http: false
      auth: []
      ca-cert: ""
  from: []
  platform: ""
  source:
      name: ""
      version: ""
      base-path: ""
      file:
          digests:
              - SHA-256
      image:
          default-pull-source: ""
  exclude: []
  cache:
      dir: C:\Users\abcdef\AppData\Local\cache\syft
      ttl: 7d

Crashing on Debian WSL with this configuration:

joerg@DE12914:~$ syft scan amlac02.azurecr.io/rel/mongo:1.1.1 -o syft-json=mongo.json -vv
[0000]  INFO syft version: 1.7.0
[0000] DEBUG config:
  log:
      quiet: false
      level: debug
      file: ""
  dev:
      profile: none
  config: .syft.yaml
  output:
      - syft-json=mongo.json
  format:
      pretty: null
      template:
          path: ""
          legacy: false
      json:
          legacy: false
          pretty: true
      spdx-json:
          pretty: false
      cyclonedx-json:
          pretty: false
      cyclonedx-xml:
          pretty: false
  check-for-app-update: false
  default-catalogers: []
  select-catalogers: []
  package:
      search-unindexed-archives: false
      search-indexed-archives: true
      exclude-binary-overlap-by-ownership: true
  file:
      metadata:
          selection: owned-by-package
          digests:
              - sha1
              - sha256
      content:
          skip-files-above-size: 256000
          globs: []
      executable:
          globs: []
  scope: all-layers
  parallelism: 1
  relationships:
      package-file-ownership: true
      package-file-ownership-overlap: true
  golang:
      search-local-mod-cache-licenses: true
      local-mod-cache-dir: /home/joerg/go/pkg/mod
      search-remote-licenses: true
      proxy: https://proxy.golang.org,direct
      no-proxy: ""
      main-module-version:
          from-ld-flags: true
          from-contents: true
          from-build-settings: true
  java:
      use-network: false
      maven-url: ""
      max-parent-recursive-depth: 0
  javascript:
      search-remote-licenses: false
      npm-base-url: ""
  linux-kernel:
      catalog-modules: true
  python:
      guess-unpinned-requirements: false
  registry:
      insecure-skip-tls-verify: false
      insecure-use-http: false
      auth: []
      ca-cert: ""
  from: []
  platform: ""
  source:
      name: ""
      version: ""
      base-path: ""
      file:
          digests:
              - SHA-256
      image:
          default-pull-source: ""
  exclude: []
  cache:
      dir: /home/joerg/.cache/syft
      ttl: 7d
Joerki commented 6 days ago

Hi @kzantow,

thanks for your changes. I'll try to build and check.

I also worked on pure Linux machine (before I saw your commit) to check and doing change of settings, and found out:

When I have search-local-mod-cache-licenses setting to true, I get a crash.

golang: search-local-mod-cache-licenses: true

In case it's false, program works as usual.

kzantow commented 4 days ago

Hi @Joerki -- I think I've addressed the issues you've raised here, where invalid directories would result in panics rather than being handled properly. Apologies for that, this seems like a case that probably should have been accounted for from the get-go. Please let us know if the latest release continues to give you issues!