Open ykazakov opened 11 months ago
The problem is exacerbated if several images are built at once, e.g., using docker compose.
For example for this Dockerfile
:
FROM jupyter/minimal-notebook:2023-09-18 AS test1
RUN touch /tmp/test
FROM test1 AS test2
RUN touch /tmp/test
FROM test2 AS test3
RUN touch /tmp/test
and this docker-compose.yaml
:
version: '3'
services:
test1:
image: test1
build:
target: test1
test2:
image: test2
build:
target: test2
test3:
image: test3
build:
target: test3
after running docker compose build
, two docker-scout
processes start and memory consumption almost doubles:
I had a similar situation in my first screenshot.
Furthermore, when all system memory is consumed, docker crashes, leaving some temporary files in /private/var/folders/
, which, in my case occupied 20+ GB. First I did not notice this and my system ran out of disk space.
You can disable the SBOM background indexer under Settings > Features in development > Experimental features > Enable background SBOM indexing
Thanks @bsousaa, that worked! Finally the memory is under control! Did this feature appear in the last version? Please consider disabling SBOM by default.
Thanks @ykazakov, I will pass the feedback to the team
Just had this disable a macOS machine. Why are experimental features enabled by default?
This absolutely destroyed my 8GB MBA (I know... my fault!) but would be good to have this disabled by default!
@MattJustMatt Same - knew this machine had little memory but have rarely seen it slow to the absolute crawl this feature caused!
Just to add that it also occurs in Docker Desktop for Windows, the experimental feature is activated by default and the same workaround as proposed here fixes it.
Same I had this issue in Docker Desktop for Windows too
I've just installed the latest version of Docker Desktop 4.25.0 and this feature SBOM indexing
has been enabled by default😒
Docker team, please disable it by default🙏
On my MacBook m1 building time increased 10x times! I've waited to make a build for around 35 minutes instead of 3 after I disable it.
This is no longer found under "experimental features" but is still called "SBOM indexing" under the general settings tab. Until I disabled it, 2 docker-scout
processes were consuming all 24GB of memory on my M2 macbook, and I had to shut it off to get any work done.
I agree that this should be disabled by default.
Same issue here. Is docker desktop a botnet?
Dis enable SBOM indexing ?
I just updated to docker desktop 4.25.2 (129061) to observe that SBOM indexing has been reenabled after I manually disabled it in the settings:
What is going on??
➕ on this issue. Had the same problem when building a simple postgres image. Memory consumption going to its limits with the container unused. Lots of memory usage also happening with docker-scout process.
This was using 15GB of RAM out of a 16GB Intel Mac on 4.25.0. This needs attention.
For anyone still finding this issue, the setting in Docker Desktop has now moved to Settings > General > SBOM Indexing
(tested on 4.25.4).
Chiming in here:
I have disabled SBOM indexing, and Usage Statistics and restarted docker desktop. docker-scout
is still being run aggressively, as if on a timer even if no new containers are pulled or built, and it is trying to use up to 32 GB (!!!) of memory in one of two processes on a computer with only 16.
The only way I can somewhat attempt to use my work computer for work right now is running killall -9 docker-scout
repeatedly on the command line to buy myself 15 minutes of actually using my computer.
This is unacceptable.
Didn't realize this was happening till my machine was running slow. Docker Scout is throttling entire machine performance. Is there a timeline on this fix since it seems like a High priority bug since it affects entire machines.
@OsirisMaeluma memory consumption will depend on the size of the images in your machine. You can disable indexing in Settings > General > SBOM Indexing
What's the point of a memory usage limit if Docker is simply going to blow past it, with a 4GB memory limit, and Docker's scout component alone using 6GB.
Chiming in here:
I have disabled SBOM indexing, and Usage Statistics and restarted docker desktop.
docker-scout
is still being run aggressively, as if on a timer even if no new containers are pulled or built, and it is trying to use up to 32 GB (!!!) of memory in one of two processes on a computer with only 16.The only way I can somewhat attempt to use my work computer for work right now is running
killall -9 docker-scout
repeatedly on the command line to buy myself 15 minutes of actually using my computer.This is unacceptable.
So, this is interesting. After a few system restarts, it seems like Docker desktop is now honoring those settings and not firing up docker-scout. Restarting Docker Desktop isn't enough though, despite the UI implying it is. Apparently these settings in particular only take on a full reboot.
FWIW, it stopped immediately after unchecking the box for me
On Tue, 6 Feb 2024 at 7:26 pm, Jason DePeaux @.***> wrote:
Chiming in here:
I have disabled SBOM indexing, and Usage Statistics and restarted docker desktop. docker-scout is still being run aggressively, as if on a timer even if no new containers are pulled or built, and it is trying to use up to 32 GB (!!!) of memory in one of two processes on a computer with only 16.
The only way I can somewhat attempt to use my work computer for work right now is running killall -9 docker-scout repeatedly on the command line to buy myself 15 minutes of actually using my computer.
This is unacceptable.
So, this is interesting. After a few system restarts, it seems like Docker desktop is now honoring those settings and not firing up docker-scout. Restarting Docker Desktop isn't enough though, despite the UI implying it is. Apparently these settings in particular only take on a full reboot.
— Reply to this email directly, view it on GitHub https://github.com/docker/for-mac/issues/6987#issuecomment-1930611091, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC5QOWC4JQGVMTDASF4SRATYSJ7WFAVCNFSM6AAAAAA5BDYNXSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZQGYYTCMBZGE . You are receiving this because you commented.Message ID: @.***>
Oh god, I've just discovered that thing. It's bloody awful. I have quite big build image (around 50gb of code) and decided to build it on Docker Desktop (usually do it in Engine on some Linux distribution like fedora). Docker Engine on Linux used 9 gigs entirely (with the system!). And build on Desktop with scout ballooned to... 110gigs. Of ram. When pulling an image. This is atrocious and it should be disabled by default.
@GetAHat AGREED! 100%. WTH!
This is still a massive problem. Computers just grind to a halt because of 97% memory usage because of docker scout. That, and the fact that disabling it often does not work at all makes Docker Desktop at this moment pretty unusable unless you have task manager open to terminate docker scout every few minutes.
Devs, please look at this and at least DISABLE it by default since it is causing so much problems.
It is wild that this is still enabled by default. I have a 32 GB machine and it almost froze up completely yesterday, until I discovered that docker-scout.exe was taking up multiple GBs of RAM and I found this issue here. How could an experimental feature like this even be promoted to a stable feature, when it is clearly broken?
Also was hit by this with about 9GB of ram used
Just managed to catch it after last time it wasn't visible in the task manager. This started eating over 16gb of ram on a 32gb machine, nearly crashing it. This issue also has been open for 8 months. At this point, this is just malicious.
No joke, it just used up all of my 64GB(!!!!!) of RAM on my windows machine. And I am running the normal Nvidia PyTorch containers (nvcr.io/nvidia/pytorch). SBOM Indexing in its current form is a hazardous feature! Fix it, disable it, or get rid of it!
On a fresh install of Docker Desktop, the indexing is no longer enabled by default. When upgrading an existing install, the current setting is preserved and it was originally introduced as enabled by default. In Settings->General at the bottom of the list, you can disable it explicitly. Sorry for the frustration.
But why was the experimental feature on by default, and that then migrated? It's clearly a broken feature that should still be in experimental and also not turned on by default. An update should turn it off for everyone, and people actually wanting it can turn it back on. In the meantime, Docker Desktop is just crashing PCs of people who can't figure out what's happening.
I just found this thread as I found docker-scout.exe using 6GB of RAM. I've never enabled it, and I agree such an experimental feature should stay disabled. Even after disabling, and clicking Apply & Restart, it's still there doing stuff. Had to quit and re-start Docker Desktop before it was really gone
same here, my 16gm RAM was overflow in a few minutes after each docker-compose up --build so i have to turn off pc and restart it. totally should be disabled by default
Just had the same issue — docker-scout sucked up all my RAM with SBOM indexing enabled, completely paralysed my machine with memory pressure being completely red. Please disable it by default or put some sane limits on this feature as it clearly shouldn't behave like that.
Description
After updating to the latest docker, I noticed that my system (MacBook Pro M1, 16 GB) runs out of working memory when building docker images. Checking Activity Monitor reveals a huge memory consumption by the process
docker-scout
:This process starts shortly after the image is built, so it seems like docker-scout tries to analyse the built images. However I do not think I ever enabled docker-scout. Actually, I was not even aware about its existence. The docker scout configuration is empty, however there is clearly something in the cache:
Other commands like
docker scout quickview
ask me for my Docker ID, which I do not have.Please, make it possible to disable docker-scout completely!
Reproduce
RUN touch /tmp/test
docker info
Diagnostics ID
6734B03B-C87B-4461-884C-139350A00371/20230921080327
Additional Info
Interestingly, I did not notice
docker-scout
starting if I use other base docker images such asubuntu:latest
,debian:latest
oralpine:latest
. Is this behaviour image/organization-specific?