search

anchore / syft

CLI tool and library for generating a Software Bill of Materials from container images and filesystems
Apache License 2.0
7.02k stars 651 forks source link

out of memory exception when scanning images (here: fedora-bootc family) #3800

Open rriemann opened 1 month ago

rriemann commented 1 month ago

What happened:

I run this command with syft 1.22.0 on a CoreOS ARM machine with 8 cores and 16GB.

TMPDIR=$(pwd)/syft_tmp ~/.local/bin/syft scan --select-catalogers -javascript -vv --parallelism 6 -o cyclonedx-json=gl-sbom-report.cdx.json podman:registry.gitlab.com/eu-os/workspace-images/eu-os-base-demo/eu-os-demo:acba8f13-41

Log output 

[0000]  INFO syft version: 1.22.0
[0000] DEBUG config:
  log:
      quiet: false
      level: debug
      file: ""
  dev:
      profile: none
  config: ""
  output:
      - cyclonedx-json=gl-sbom-report.cdx.json
  format:
      pretty: null
      template:
          path: ""
          legacy: false
      json:
          legacy: false
          pretty: false
      spdx-json:
          pretty: false
      cyclonedx-json:
          pretty: false
      cyclonedx-xml:
          pretty: false
  check-for-app-update: true
  default-catalogers: []
  select-catalogers:
      - -javascript
  package:
      search-unindexed-archives: false
      search-indexed-archives: true
      exclude-binary-overlap-by-ownership: true
  license:
      include-unknown-license-content: false
      license-coverage: 75
  file:
      metadata:
          selection: owned-by-package
          digests:
              - sha1
              - sha256
      content:
          skip-files-above-size: 256000
          globs: []
      executable:
          globs: []
  scope: squashed
  parallelism: 6
  relationships:
      package-file-ownership: true
      package-file-ownership-overlap: true
  compliance:
      missing-name: drop
      missing-version: stub
  enrich: []
  dotnet:
      dep-packages-must-have-dll: false
      dep-packages-must-claim-dll: true
      relax-dll-claims-when-bundling-detected: true
  golang:
      search-local-mod-cache-licenses: null
      local-mod-cache-dir: /var/home/rriemann/go/pkg/mod
      search-local-vendor-licenses: null
      local-vendor-dir: ""
      search-remote-licenses: null
      proxy: https://proxy.golang.org,direct
      no-proxy: ""
      main-module-version:
          from-ld-flags: true
          from-contents: true
          from-build-settings: true
  java:
      use-network: null
      use-maven-local-repository: null
      maven-local-repository-dir: /var/home/rriemann/.m2/repository
      maven-url: https://repo1.maven.org/maven2
      max-parent-recursive-depth: 0
      resolve-transitive-dependencies: false
  javascript:
      search-remote-licenses: null
      npm-base-url: ""
      include-dev-dependencies: null
  linux-kernel:
      catalog-modules: true
  python:
      guess-unpinned-requirements: false
  registry:
      insecure-skip-tls-verify: false
      insecure-use-http: false
      auth: []
      ca-cert: ""
  from: []
  platform: ""
  source:
      name: ""
      version: ""
      base-path: ""
      file:
          digests:
              - SHA-256
      image:
          default-pull-source: ""
          max-layer-size: ""
  exclude: []
  unknowns:
      remove-when-packages-defined: true
      executables-without-packages: true
      unexpanded-archives: true
  cache:
      dir: /var/home/rriemann/.cache/syft
      ttl: 7d
[0000] DEBUG checking if a new version of syft is available
[0000] DEBUG no new syft update available
[0172] DEBUG image metadata: digest=sha256:ccd66eeb906c20dde572038d60def95e590f85bff8af6f24363bd54fa6e17804 mediaType=application/vnd.docker.distribution.manifest.v2+json tags=[registry.gitlab.com/eu-os/workspace-images/eu-os-base-demo/eu-os-demo:latest]
[0172] DEBUG layer metadata: index=0 digest=sha256:87e2a2d8ad5b02dac5176ee4f401cc208d844ac5c62fc6e920ddcc6e720d82ca mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0173] DEBUG layer metadata: index=1 digest=sha256:14f86658c86aac2a4571ce1d0086bf42a6ad80f615940a90c92914f91f5053ab mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0173] DEBUG layer metadata: index=2 digest=sha256:5803e500ad380b9e7b4ce40e62fb762e52068aaeedd2867bcd99a3c17d50d456 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0174] DEBUG layer metadata: index=3 digest=sha256:682c6ceb0e263d1dae84f472f05a737a5f4ee0fab6d0646b97b465d1fdbbf564 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0174] DEBUG layer metadata: index=4 digest=sha256:84a13ea341341914709c75db4abf477de53ae1aa4bd99593499dba3a21720655 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0174] DEBUG layer metadata: index=5 digest=sha256:60bd82537848b3568190940a1dd0d5e22c82cda54bf281faf0f9c06765184aab mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0174] DEBUG layer metadata: index=6 digest=sha256:76296e4a7d5b6e0370455e7c650d6d27eedbc8ccb4410c76178e0f08e561868a mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0175] DEBUG layer metadata: index=7 digest=sha256:37c1214947937689381263359e024f51c297450ceb579c6806d69a1309fa4bc3 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0175] DEBUG layer metadata: index=8 digest=sha256:aa1950ad3f1a2ee76cb2f8bf8f7df6f5fbc77b703b3e6c0fc945c1fbc51cf4a0 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0175] DEBUG layer metadata: index=9 digest=sha256:b930f92ec447cf1fed613868593b3a15032a5ff12f352009e260e218cf4caa31 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0175] DEBUG layer metadata: index=10 digest=sha256:533cb6a2ac4e64b967ca5e912124db2c5edcb99e3f7bb6e397d58511c6b65ed3 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0175] DEBUG layer metadata: index=11 digest=sha256:5136b9668456a3c1ccc045b773bd6e05a391a2d2b23f92231d3d1865985c94e7 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0176] DEBUG layer metadata: index=12 digest=sha256:1343d1c6ac3e9f8ec9b37f5908fa13cd4b0a8de8471518dd900387439eda7dbe mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0176] DEBUG layer metadata: index=13 digest=sha256:86b44e4d826e474c3529f3944babf8bd24891cfc844647bce6af1005b7fee5c5 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0176] DEBUG layer metadata: index=14 digest=sha256:6e85a5670467b0db79177c17b1a474567ba2d6605d8544be7ae8135cdbe124e7 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0176] DEBUG layer metadata: index=15 digest=sha256:e5be975dcc6e34c8235764ffac1d77aa932a6578e91cee3402be9af6e8743424 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0176] DEBUG layer metadata: index=16 digest=sha256:549c7a37e009ce65c484adda626f6493a6a930160e35931d040f4a807442a3da mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0176] DEBUG layer metadata: index=17 digest=sha256:01becaacfdbafaef2fe5e8a13f171ed4512f97d67eb2fc1bd4ad45a218952e87 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0177] DEBUG layer metadata: index=18 digest=sha256:ee286c3dd82ca52a70acde8b5ace58a8338d72af4024eef767251dfd6c7235bc mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0177] DEBUG layer metadata: index=19 digest=sha256:b7474d2e89771312642f401f06907f99811142eb41996ccc994c866c52561463 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0177] DEBUG layer metadata: index=20 digest=sha256:545349473a878c20db1ce8f95e1007714f15764cbcb6c5a77a6d3d982bb4b988 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0178] DEBUG layer metadata: index=21 digest=sha256:60780752c65a106cb5896371edd8460f7a574cd156bb37df7f4d9471c30225fe mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0178] DEBUG layer metadata: index=22 digest=sha256:323ac21c584ec71655bd4d23d150797faf78effbb307983fcaf8f3341ca4d3aa mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0178] DEBUG layer metadata: index=23 digest=sha256:0fc0870b58a7ad7cd3238904f758dfed6c38d9e9d0f9ca6a7d5143da990d54ee mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0178] DEBUG layer metadata: index=24 digest=sha256:07bd02a5866143a7ec121de31b16a060cf7b598dbee71e10516daabe54f1f069 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0178] DEBUG layer metadata: index=25 digest=sha256:f6915300476fef3b3cd36c6a61170609468f70e689ce9f0b39fdbde94a46a93d mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0179] DEBUG layer metadata: index=26 digest=sha256:5d37974a514880e50d27bb5e496b7d9cf80cbe687d65f72cd831f6a1c082a654 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0180] DEBUG layer metadata: index=27 digest=sha256:7f38477b48fb64eca762f01066b5c64fe81f00a1d105230286e517ec93f079d1 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0180] DEBUG layer metadata: index=28 digest=sha256:52ed1d7147277bc2a818f1fc330238a89c40be90f4928f982eae396cbb7da6bd mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0180] DEBUG layer metadata: index=29 digest=sha256:221a17e21dd9925fa3199df00a1bbf065f7f71e6f25a556a9a6ad5cca6fec50b mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0180] DEBUG layer metadata: index=30 digest=sha256:5fb11e42dbbb3e2df26a9621836638f432826a3b3939f8b6b5a8bc8acd23605f mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0181] DEBUG layer metadata: index=31 digest=sha256:05039c19c5f34b75354f479e9ef722e416d2c27165c608bcf8c267c695e2b7d1 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0181] DEBUG layer metadata: index=32 digest=sha256:1311e0a5f5dbca220f161aea7ab65417ea9b0b402ffea46378c3a6b2ead511dd mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0181] DEBUG layer metadata: index=33 digest=sha256:a6dcef0e354c7cc6f4b68c8a0a23f6f192c4fb07d9bb5033ec0cfa6f0399fe74 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0181] DEBUG layer metadata: index=34 digest=sha256:e01377438e759293d54a5e66871e1e72b40b52db861729a9d8d2ecd2665675a8 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0181] DEBUG layer metadata: index=35 digest=sha256:98bbcafc20415dcac285195c87844e5f29e55c7a70e35502c621c69ab77db926 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0182] DEBUG layer metadata: index=36 digest=sha256:d23cc0cc9df5b81cf44ab0d572f707eda05d3ac29f5f2cab3d3f0d21271ddbf4 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0182] DEBUG layer metadata: index=37 digest=sha256:b64e05ad382f0dce00989c5488bd8e01291a51eead3e5a43c2e0d250fbacf908 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0183] DEBUG layer metadata: index=38 digest=sha256:d6fe31275ab5fcb2a6736b74c32f3105bb8d34d2078bb8cecb7f7fae870dc4f5 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0184] DEBUG layer metadata: index=39 digest=sha256:5e48bfac23a5609f424e6694c9932be913b05b344ac3b0782cfe565518fe75f4 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0184] DEBUG layer metadata: index=40 digest=sha256:d1f28de64c1e8778000ba53d1047fe9ff4c5acb7c90ffcd112805d9571bb76b6 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0185] DEBUG layer metadata: index=41 digest=sha256:4e12f52fb52dab2c52e7c7f3c4596cb3832cd6e6f6d858ff96d87ac31b04a429 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0190] DEBUG layer metadata: index=42 digest=sha256:8bc9158217c050e7e851e45459cb0080a7d6d3804e46f1fc86f9bd776840c171 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0192] DEBUG layer metadata: index=43 digest=sha256:e34df13fdb902aad03e42b453ba898dbb9565d3064db4b60e18d4d297ce293c5 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0193] DEBUG layer metadata: index=44 digest=sha256:9ba4341f57ce4e5459a3d808a3bacd01fd16325b144465fbd414853295503783 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0194] DEBUG layer metadata: index=45 digest=sha256:b674f777a1561a5a5da12b797c8cb33d0440bb90cb2481a2c9bd2a4b36c054be mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0195] DEBUG layer metadata: index=46 digest=sha256:f06db302d3a2dd0a27aafeca0bda106ae2ec929bf2a134218b68689e5d827a3b mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0196] DEBUG layer metadata: index=47 digest=sha256:a32724be0438cc4a3e537b799bae3ec1f534ce428b81fd96b08e6d670edb1f04 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0198] DEBUG layer metadata: index=48 digest=sha256:1b266c2ae73dae279b96355843c14d85b0be7540701302c9c6cfaf7e85a368d0 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0200] DEBUG layer metadata: index=49 digest=sha256:670c2c40c494ca2bc7c3c294a573f0bda8460b7b04c6d95c84ca065d1e98996b mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0202] DEBUG layer metadata: index=50 digest=sha256:606347154655f967abbdbcd4921302bb0c59fc741d2e44d3a6c569a306a031f9 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0205] DEBUG layer metadata: index=51 digest=sha256:4d4527e63785b1cb26a0d94473c0f7f4d2ee8431f2af72290e47bbf458faf1a0 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0206] DEBUG layer metadata: index=52 digest=sha256:98a2606a663f30c3377fe24f03083f956df244856a3dfe9363fd4f3845ca5780 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0207] DEBUG layer metadata: index=53 digest=sha256:58971769f2d052b97eeab0a2e5e8347e302c05966a6e23dac2e7606e3f0958c5 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0209] DEBUG layer metadata: index=54 digest=sha256:2d6196f4c83d364af11e20c0301a7425378106a460463d1e5140848870379436 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0210] DEBUG layer metadata: index=55 digest=sha256:038d679f04bcf481b624547db81c78bed540c1f9992654bde12ac4bfc507cfc2 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0212] DEBUG layer metadata: index=56 digest=sha256:0e26247460006d4dc6d66901014f7d154f918c8f5bd33c07944c2b80c7c99bc8 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0213] DEBUG layer metadata: index=57 digest=sha256:7f1e2500b204a2ba91612b3ed5602687354d490bcf2911849b0ca0db5fc54d7e mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0215] DEBUG layer metadata: index=58 digest=sha256:f4bb717b4262622543b9292088ffb63ad87fbe02b7ed89e12d7aa5c3ee89397f mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0217] DEBUG layer metadata: index=59 digest=sha256:e2173ce44450eecdaa2e2703968e3497633cee4d31630a505dffcc9f2ab64415 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0219] DEBUG layer metadata: index=60 digest=sha256:4ec50f5ee95781bbc0c41874c2d6fc521e88a02b61f7493ec9aeb9097d229825 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0221] DEBUG layer metadata: index=61 digest=sha256:4f44f8ff62bd6c81b9f19dbc68d5335e279b89a6f3330445d8daac233603c43f mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0223] DEBUG layer metadata: index=62 digest=sha256:e521e36f2f4bdfdc9d565ca32ee3d93a6af8614528ad657a16709ad9e6285d1d mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0227] DEBUG layer metadata: index=63 digest=sha256:bb7a1a07d6784f62f2ba3e956d98eaf73d26ca4b82301a8187357952b2bee10e mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0229] DEBUG layer metadata: index=64 digest=sha256:12787d84fa137cd5649a9005efe98ec9d05ea46245fdc50aecb7dd007f2035b1 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0231] DEBUG layer metadata: index=65 digest=sha256:21c20bb17d1a0eb941c8037be4eb9c8980dcd1e2efce60145c89658de5fed7c1 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0234] DEBUG layer metadata: index=66 digest=sha256:7ff670fc43dcbf3826e4ad3aec72704ec1d3e5521b036ac5d694032ba58670bf mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0235] DEBUG layer metadata: index=67 digest=sha256:6a6b6a2377c4ddf566de57c113223651ce8d7fe7ad5d1e1a51f5dfb404d71d8f mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0236] DEBUG layer metadata: index=68 digest=sha256:05f774391938808835060994d727ab92b5fd38ab820edfa87685ad805add8708 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0237] DEBUG layer metadata: index=69 digest=sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0239] DEBUG layer metadata: index=70 digest=sha256:d6f37474fbe1d1988005824c2172a0fc2c94281154a1058682a25d2a45cafb89 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0239] DEBUG layer metadata: index=71 digest=sha256:6ce597f60b82fb21ecf69040c80b5df01a59b0d2be130fc53362dbda2a6aee41 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0241] DEBUG layer metadata: index=72 digest=sha256:4f7149dda0e54eede248b76be1c88a689885c8f6ce91d69664a329920862a421 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0242] DEBUG layer metadata: index=73 digest=sha256:a11e137476bf1ae18d05cedfc9722e09d355fc8ff7942600c9fd04cb82bdd13e mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0243] DEBUG layer metadata: index=74 digest=sha256:02234c80d8b92cd4c408d9994b5b950f100b08431c6330a968a22bc1295857e4 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip
[0465] DEBUG selected 25 package cataloger tasks
[0465] DEBUG selected 4 file cataloger tasks
[0466]  INFO task completed elapsed=1.80226ms task=environment-cataloger
[0466] DEBUG discovered 0 packages cataloger=portage-cataloger
[0466]  INFO task completed elapsed=1.711459ms task=portage-cataloger
[0466] DEBUG discovered 0 packages cataloger=conan-info-cataloger
[0466]  INFO task completed elapsed=2.278145ms task=conan-info-cataloger
[0466] DEBUG discovered 0 packages cataloger=php-composer-installed-cataloger
[0466]  INFO task completed elapsed=595.127µs task=php-composer-installed-cataloger
[0466] DEBUG discovered 0 packages cataloger=r-package-cataloger
[0466]  INFO task completed elapsed=413.684µs task=r-package-cataloger
[0466] DEBUG discovered 0 packages cataloger=alpm-db-cataloger
[0466]  INFO task completed elapsed=3.256276ms task=alpm-db-cataloger
[0466] DEBUG discovered 0 packages cataloger=ruby-installed-gemspec-cataloger
[0466] DEBUG discovered 0 packages cataloger=apk-db-cataloger
[0466]  INFO task completed elapsed=3.57068ms task=apk-db-cataloger
[0466] DEBUG discovered 0 packages cataloger=php-pecl-serialized-cataloger
[0466]  INFO task completed elapsed=419.445µs task=php-pecl-serialized-cataloger
[0466]  INFO task completed elapsed=873.769µs task=ruby-installed-gemspec-cataloger
[0466] DEBUG discovered 0 packages cataloger=dotnet-packages-lock-cataloger
[0466]  INFO task completed elapsed=216.283µs task=dotnet-packages-lock-cataloger
[0466] DEBUG discovered 0 packages cataloger=dotnet-deps-binary-cataloger
[0466]  INFO task completed elapsed=852.29µs task=dotnet-deps-binary-cataloger
[0466] DEBUG discovered 0 packages cataloger=dpkg-db-cataloger
[0466]  INFO task completed elapsed=4.737932ms task=dpkg-db-cataloger
[0466] DEBUG discovered 0 packages cataloger=java-archive-cataloger
[0466]  INFO task completed elapsed=303.044µs task=java-archive-cataloger
[0466] DEBUG discovered 67 packages cataloger=python-installed-package-cataloger
[0466]  INFO task completed elapsed=185.388931ms task=python-installed-package-cataloger
[0466] DEBUG discovered 0 packages cataloger=lua-rock-cataloger
[0466]  INFO task completed elapsed=2.151464ms task=lua-rock-cataloger
fish: Job 1, 'TMPDIR=$(pwd)/syft_tmp ~/.local…' terminated by signal SIGKILL (Forced quit)

Exit code: 137

dmesg output:

[966369.682694] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=user.slice,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1001.slice/session-3.scope,task=syft,pid=2846858,uid=1001 [966369.682750] Out of memory: Killed process 2846858 (syft) total-vm:14810388kB, anon-rss:13505124kB, file-rss:4480kB, shmem-rss:0kB, UID:1001 pgtables:26752kB oom_score_adj:0

What you expected to happen:

The program finishes with a json output.

Steps to reproduce the issue:

run the command above

Anything else we need to know?:

It also did not work on my desktop computer. I get the same out of memory error. This one is x86_64 with 8GB. On gitlab.com with their runners, the scan ran into a timeout.

This seems to be related to #3651. I disabled already the javascript part as you can see in the command, but it did not help.

Environment:

NAME="Fedora Linux" VERSION="41.20250315.3.0 (CoreOS)" RELEASE_TYPE=stable ID=fedora VERSION_ID=41 VERSION_CODENAME="" PLATFORM_ID="platform:f41" PRETTY_NAME="Fedora CoreOS 41.20250315.3.0" ANSI_COLOR="0;38;2;60;110;180" LOGO=fedora-logo-icon CPE_NAME="cpe:/o:fedoraproject:fedora:41" HOME_URL="https://getfedora.org/coreos/" DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora-coreos/" SUPPORT_URL="https://github.com/coreos/fedora-coreos-tracker/" BUG_REPORT_URL="https://github.com/coreos/fedora-coreos-tracker/" REDHAT_BUGZILLA_PRODUCT="Fedora" REDHAT_BUGZILLA_PRODUCT_VERSION=41 REDHAT_SUPPORT_PRODUCT="Fedora" REDHAT_SUPPORT_PRODUCT_VERSION=41 SUPPORT_END=2025-12-15 VARIANT="CoreOS" VARIANT_ID=coreos OSTREE_VERSION='41.20250315.3.0'

popey commented 1 month ago

Hi @rriemann - thanks for reporting this issue, and for the reproducible steps.

I have tried on my (chunky) laptop and syft peaks at 8.7GB RAM usage. So I'm not surprised it gets oom-killed. Log below.

Memory profiling...

$ export SYFT_DEV_PROFILE=mem
$ export TMPDIR=$(pwd)/syft_tmp
$ ~/bin/syft scan --select-catalogers -javascript -vv --parallelism 6 -o cyclonedx-json=gl-sbom-report.cdx.json registry.gitlab.com/eu-os/workspace-images/eu-os-base-demo/eu-os-demo:acba8f13-41 > syft_log.txt 2>&1
$ go tool pprof ~/bin/syft ./syft_tmp/profile2797470040/mem.pprof
File: syft
Type: inuse_space
Time: 2025-04-17 12:53:12 BST
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) web
(pprof) top
Showing nodes accounting for 8700.52MB, 85.68% of 10155.10MB total
Dropped 583 nodes (cum <= 50.78MB)
Showing top 10 nodes out of 86
      flat  flat%   sum%        cum   cum%
 4278.56MB 42.13% 42.13%  6323.47MB 62.27%  github.com/anchore/stereoscope/pkg/tree.(*Tree).Copy
 2044.89MB 20.14% 62.27%  2044.89MB 20.14%  github.com/anchore/stereoscope/pkg/filetree/filenode.(*FileNode).Copy
 1059.07MB 10.43% 72.70%  1059.07MB 10.43%  github.com/anchore/stereoscope/pkg/tree/node.IDSet.Add (inline)
  335.34MB  3.30% 76.00%   335.34MB  3.30%  github.com/anchore/stereoscope/pkg/tree.(*Tree).addNode
  197.12MB  1.94% 77.94%   197.12MB  1.94%  github.com/anchore/stereoscope/pkg/tree/node.NewIDSet (inline)
  192.95MB  1.90% 79.84%  1449.14MB 14.27%  github.com/anchore/stereoscope/pkg/filetree.(*searchContext).buildLinkResolutionIndex
     164MB  1.61% 81.46%   970.95MB  9.56%  github.com/anchore/stereoscope/pkg/file.NewTarIndex.func1
  153.59MB  1.51% 82.97%   154.64MB  1.52%  github.com/anchore/stereoscope/pkg/file.NewMetadata
  141.79MB  1.40% 84.36%   262.85MB  2.59%  github.com/anchore/stereoscope/pkg/filetree.(*index).Add
  133.20MB  1.31% 85.68%   806.95MB  7.95%  github.com/anchore/stereoscope/pkg/image.(*Layer).readStandardImageLayer.layerTarIndexer.func1

Image

syft_log.tar.gz

rriemann commented 1 month ago

I do not really know the internals of syft and how to read this diagram, but my wild guess is that the ostree/composefs parts in the image contain some (recursive) duplicated scanning of the file tree.

kzantow commented 1 month ago

I do not believe this is the same issue as https://github.com/anchore/syft/issues/3651, which was due to a bug that caused squashfs to continue reading after EOF, and I believe should be fixed in syft v1.22.0, though I had some difficulty tracking down which commits would validate this. I was able to successfully run syft to catalog this image locally, on my MacBook Pro with 32 GB RAM, though I did see memory usage get upwards of 20GB, possibly up to 30GB including swap space.

A few more pertinent details: this is an 8.1 GB image we're scanning, with 74 layers:

$ docker save  > ~/Downloads/eu-os-base-demo.tar
$ ls -alFh ~/Downloads/eu-os-base-demo.tar
-rw-r--r--  1 kzantow  staff   8.1G Apr 16 10:18 /Users/kzantow/Downloads/eu-os-base-demo.tar
...
[0243] DEBUG layer metadata: index=74 digest=sha256:02234c80d8b92cd4c408d9994b5b950f100b08431c6330a968a22bc1295857e4 mediaType=application/vnd.docker.image.rootfs.diff.tar.gzip

I also created a memory profile, which matches what @popey posted above: it looks like inuse space grows to over 8GB, causing this specific image to fail when only 8GB is available. The memory is being predominantly used by the in-memory representation of the filesystem of all the layers.

When you say "ostree/composefs parts in the image contain some (recursive) duplicated", are you referring to this occurring via symlinks? I believe these are normalized to the absolute paths and shouldn't be responsible for large issues. I'm actively investigating ways to help improve memory usage and performance in this part of the app, but don't have a lot of concrete changes yet. See: https://github.com/anchore/stereoscope/issues/233 and https://github.com/anchore/syft/issues/1446

popey commented 1 month ago

We briefly discussed this on the live stream, and it feels like this could benefit from the perennial topic of "dialling down memory usage by swapping an in-memory database to disk" that we have discussed in the past.

rriemann commented 1 month ago

As someone not aware of the internals of the software, I am surprised that scanning an 8GB image could require 20GB of memory.

davidjeddy commented 3 weeks ago

Dropping a line here; also experiencing OOM when scanning a custom 6.8Gb image.

syft scan   --scope all-layers   --output cyclonedx-xml=sbom.xml   podman:"${TARGET}":"${TARGET_VERSION_TAG}"
 ✔ Loaded image                                                                               891377244928.dkr.ecr.eu-west-1.amazonaws.com/prd/toolbox/jenkins-agents-wl-gc-iac/m590:0.17.3 
A newer version of syft is available for download: 1.22.0 (installed version is 1.21.0)
[0045] ERROR could not determine source: an error occurred attempting to resolve '891377244928.dkr.ecr.eu-west-1.amazonaws.com/prd/toolbox/jenkins-agents-wl-gc-iac/m590:0.17.3': podman: unable to save image to tar: write /tmp/stereoscope-3378755864/podman-daemon-image-2776216565/image.tar: no space left on device

Would it be possible to use a custom TMP location?

popey commented 3 weeks ago

@davidjeddy Hello! Yes, you can set the OS TMPDIR to define where syft puts the temporary files.

$ mkdir -p /home/alan/Temp/tempsyft
$ TMPDIR=/home/alan/Temp/tempsyft syft nextcloud:latest
$ du -hs /home/alan/Temp/tempsyft
2.5G    /home/alan/Temp/tempsyft

However, this issue is more about running out of RAM, while your issue is running out of disk space.

m2Giles commented 1 day ago

On GitHub's free runners, there appears to be a large secondary disk mounted to /mnt.

My current strategy is to eliminate the existing swapfile at /mnt/swapfile and replace it with 70GB swapfile at the same location. The disk appears to usually be between 75 GB and 84 GB.

This solves the OOM for the most part. However, scans take an extremely long time with several duplicates. I do not know if this due to the different scanners. My resolution here is to only scan RPMs. This significantly reduces memory usage.

My last space savings technique is to not use stereoscope. I instead have syft scan an oci-archive. This doesn't concern memory, but for the limited disk space on runners.

rriemann commented 23 hours ago

This solves the OOM for the most part. However, scans take an extremely long time with several duplicates. I do not know if this due to the different scanners. My resolution here is to only scan RPMs. This significantly reduces memory usage.

My last space savings technique is to not use stereoscope. I instead have syft scan an oci-archive. This doesn't concern memory, but for the limited disk space on runners.

Thanks for sharing your work around. Can you please give the commands for scanning an oci-archive and for rpm scan only?

m2Giles commented 23 hours ago

My just recipe is here: https://github.com/m2Giles/m2os/blob/463a56fda28bd9cd28d9fb1951d2b49719679eb5/Justfile#L743

For scanning an oci-archive; my input is the oci-archive directly. The github workflow is here that does the swapfile changes, and passes the oci-archive to the recipe: https://github.com/m2Giles/m2os/blob/463a56fda28bd9cd28d9fb1951d2b49719679eb5/.github/workflows/gen-sbom.yml

popey commented 22 hours ago

This doesn't concern memory, but for the limited disk space on runners.

This might be interesting - the author strips out most of the software on the runner, to make even more disk space available. Obviously that may mean something you actually need in your workflow is gone, but it certainly gets rid of a lot, so putting back just the bits you need (or not deleting them) might be an option if you want optimal disk space.

https://wimpysworld.com/posts/nothing-but-nix-github-actions/

popey commented 22 hours ago

My current strategy is to eliminate the existing swapfile at /mnt/swapfile and replace it with 70GB swapfile at the same location. The disk appears to usually be between 75 GB and 84 GB.

Another option which might be worth considering as a workaround is https://blacksmith.sh who claim to be a drop-in replacement for the standard runners, but "twice as fast, half the cost". I haven't tried it, but I have certainly seen others rave about it.

kylegalbraith commented 20 hours ago

@popey Saw the comment on needing beefier runners. We have Depot GitHub Actions runners that are about 3-10x faster and also half the cost. Do some neat things inside of the runner like ramdisks for faster disk access, larger disk sizes, and they also integrate directly with our container build product + registry if you ever needed to use those. Some docs on the different runner types if you wanted to try them out: https://depot.dev/docs/github-actions/runner-types.