search

docker / for-win

Bug reports for Docker Desktop for Windows
https://www.docker.com/products/docker#/windows
1.86k stars 290 forks source link

Shared Volumes Slow #188

Closed dimasnake closed 2 years ago

dimasnake commented 8 years ago

Expected behavior

File access in volumes should be comparable to access times in non-volumes, similarly to Linux installations of docker

Actual behavior

File access in volumes is many times slower than on non-volumes.

Information

Version: 1.12.3-beta29.2 (8280) Channel: Beta Sha1: 902414df0cea7fdc85b87f0077b0106c3af9f64c Started on: 2016/11/01 21:19:46.408 Resources: C:\Program Files\Docker\Docker\Resources OS: Windows 10 Pro Edition: Professional Id: 1607 Build: 14393 BuildLabName: 14393.351.amd64fre.rs1_release_inmarket.161014-1755

Steps to reproduce the behavior

Get on the commandline of a lightweight docker container

root@a6b2e82c167b:/# dd if=/dev/zero of=test.dat bs=1024 count=100000
100000+0 records in
100000+0 records out
102400000 bytes (102 MB) copied, 0.569183 s, 180 MB/s

and mount a volume:

root@a6b2e82c167b:/var/www# dd if=/dev/zero of=test.dat bs=1024 count=100000
100000+0 records in
100000+0 records out
102400000 bytes (102 MB) copied, 5.11662 s, 20.0 MB/s

In 9 times slower.

ForceConstant commented 4 years ago

I am having a similar issue, not sure what the best way to address. I am not using WSL, but whatever the default docker for windows install is. But writing to mounted volume directory is 10-50x slower than other directories inside the container. How should I address this?

cortex93 commented 4 years ago

@Piepongwong — but it assumes that you don't need a performance on host. Good luck on IntelliJ project re-indexing every time you focus main window…

Try vscode remote wsl/container or you may also forward x11 with IntelliJ running inside the container.

er1z commented 4 years ago

Tried forwarding, there are some minor issues that makes work a struggle. Apart from GUI lag…

hsk-kr commented 4 years ago

Is there anyone else who solved this problem...? I'm using SFTP instead of docker-volume in windows10....

mapic100 commented 3 years ago

it is my solution to increase speed (in my case from 8MB/s even up to 2GB/s on removable SSD disk)

all what we need it is format disk/partition to have on Ext4 (i used free AOMEI Partition Assistant Sotf on W10)

in my case it is (g:) drive on windows is able to access/manage from path: "\\wsl$\docker-desktop\mnt\host\g\"

on docker-compose.yml we have to add volume like that: " volumes:

before: root@65aad241c5b5:/var/www# dd if=/dev/zero of=test.dat bs=1024 count=100000 100000+0 records in 100000+0 records out 102400000 bytes (102 MB, 98 MiB) copied, 12.0597 s, 8.5 MB/s

after: root@02c53e37e74f:/var/www# dd if=/dev/zero of=test.dat bs=1024 count=100000 100000+0 records in 100000+0 records out 102400000 bytes (102 MB, 98 MiB) copied, 0.146129 s, 701 MB/s for 100MB

ps i tested also for 5GB and i had result 2.3 GB/s

er1z commented 3 years ago

This is still a workaround, @mapic100, and doesn't solve a problem of two-way sync between container and Windows host.

docker-robott commented 3 years ago

Issues go stale after 90 days of inactivity. Mark the issue as fresh with /remove-lifecycle stale comment. Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. /lifecycle stale

paq85 commented 3 years ago

I guess everyone lost their hopes for a solution since this has been over 4 years now =D

Let's get some hope back: /remove-lifecycle stale

mapic100 commented 3 years ago

one thing i cannot resolve are lost projects (folders & files) after update Docker Desktop to newest version, maybe someone had the same issue and have some solution?

dovidweisz commented 3 years ago

Can someone confirm if this is still an issue when using the new WSL2 based engine?

Izopi4a commented 3 years ago

yes it is still very slow. With mounted volumes it probably will be fine, but i am not installing dev previous ( i am on preview circle, but i am not going below that). The workaround with home dir is decent tho

peter-gribanov commented 3 years ago

/remove-lifecycle stale

ken9ross commented 3 years ago

/remove-lifecycle stale

stevekm commented 3 years ago

I was having similar issues and binding all volumes from with wsl itself instead through the windows mount solved it for me. This is recommended best practice: https://docs.docker.com/docker-for-windows/wsl/#best-practices .

I'm not experiencing any significant performance differences between a real linux machine and docker for windows anymore.

from that link;

Performance is much higher when files are bind-mounted from the Linux filesystem, rather than remoted from the Windows host. Therefore avoid docker run -v /mnt/c/users:/users (where /mnt/c is mounted from Windows).

Instead, from a Linux shell use a command like docker run -v ~/my-project:/sources where ~ is expanded by the Linux shell to $HOME.

This is not a feasible solution. By default, the WSL filesystem is located on the Windows OS install drive, and is limited to 256GB in size. It is common for systems with a lot of high throughput disk IO to have dedicated drives such as fast SSD's meant for this purpose; the solution here makes it impossible to use your dedicated fast storage hardware. If you followed this guide, you will likely lock up your entire OS pretty quickly and fill the boot disk.

docker-robott commented 3 years ago

Issues go stale after 90 days of inactivity. Mark the issue as fresh with /remove-lifecycle stale comment. Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. /lifecycle stale

peter-gribanov commented 3 years ago

/remove-lifecycle stale

docker-robott commented 2 years ago

Issues go stale after 90 days of inactivity. Mark the issue as fresh with /remove-lifecycle stale comment. Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. /lifecycle stale

msxdan commented 2 years ago

/remove-lifecycle stale

I switched from WSL to Hyper-V as backend because the bind was so slow, after a while working with Hyper-V, today it went for a very good speed to a very low speed, not sure what happened..

This is the comparison of the current speed

Environment

Windows 10
SSD Force MP600
Docker version 20.10.11, build dea9396

Hyper-V Backend

Docker Volume (Using SSD 500 MB/s R/W )

root@e93a61c12105:~# time dd if=/dev/zero of=speedtest bs=1024 count=100000
100000+0 records in
100000+0 records out
102400000 bytes (102 MB, 98 MiB) copied, 0.217735 s, 470 MB/s

real    0m0.220s
user    0m0.040s
sys     0m0.179s

Docker Bind (Using SSD 4 GB/s R/W)

root@78596b019d76:/pwd# time dd if=/dev/zero of=speedtest bs=1024 count=100000
100000+0 records in
100000+0 records out
102400000 bytes (102 MB, 98 MiB) copied, 30.9469 s, 3.3 MB/s

real    0m30.960s
user    0m0.220s
sys     0m4.726s

WSL Backend

Docker Volume (Using SSD 4 GB/s R/W )

root@47c2706508cc:/pwd# time dd if=/dev/zero of=speedtest bs=1024 count=100000
100000+0 records in
100000+0 records out
102400000 bytes (102 MB, 98 MiB) copied, 0.118422 s, 865 MB/s

real    0m0.126s
user    0m0.010s
sys     0m0.116s

Docker Bind (Using SSD 4 GB/s R/W )

root@f4da61498f0c:/pwd# time dd if=/dev/zero of=speedtest bs=1024 count=100000
100000+0 records in
100000+0 records out
102400000 bytes (102 MB, 98 MiB) copied, 14.6283 s, 7.0 MB/s

real    0m14.636s
user    0m0.352s
sys     0m2.240s
mattodinaturas commented 2 years ago

So.. in 6 years and counting isn't there a solution yet? It's just nuts.. I am working with hyper-v (wsl has too many limitations for my use case) and file access to host directories is painfully slow. The only option is using linux it seems

freeze000 commented 2 years ago

Guys I have the same problem. And I resolved it. I use this guide https://docs.docker.com/desktop/windows/wsl/ (I installed Debian from windows store, I think it's use smaller resources than ubuntu.)

My resultats if I run docker from windows cmd with volume root@1a72a1c8cef1:/app# dd if=/dev/zero of=test.dat bs=1024 count=100000 100000+0 records in 100000+0 records out 102400000 bytes (102 MB, 98 MiB) copied, 21.049 s, 4.9 MB/s

If i run docker from debian with volume dd if=/dev/zero of=test.dat bs=1024 count=100000 100000+0 records in 100000+0 records out 102400000 bytes (102 MB, 98 MiB) copied, 0.461592 s, 222 MB/s

And in last case gulp-watch works! I am really happy.

msxdan commented 2 years ago

@freeze000

Volumes works fine from windows too

These are my results:

Hyper-V Bakend (Bind)

docker run --rm -it -v C:\Users\PC\Desktop\docker-volume-test:/docker-test ubuntu
root@8b980d71c272:/# cd docker-test/
root@8b980d71c272:/docker-test# dd if=/dev/zero of=test.dat bs=1024 count=100000
100000+0 records in
100000+0 records out
102400000 bytes (102 MB, 98 MiB) copied, 28.8806 s, 3.5 MB/s

WSL Bakend (Bind)

docker run --rm -it -v C:\Users\PC\Desktop\docker-volume-test:/docker-test ubuntu
Unable to find image 'ubuntu:latest' locally
latest: Pulling from library/ubuntu
7c3b88808835: Pull complete
Digest: sha256:8ae9bafbb64f63a50caab98fd3a5e37b3eb837a3e0780b78e5218e63193961f9
Status: Downloaded newer image for ubuntu:latest
root@91d3613ae9c6:/# cd docker-test/
root@91d3613ae9c6:/docker-test# dd if=/dev/zero of=test.dat bs=1024 count=100000
100000+0 records in
100000+0 records out
102400000 bytes (102 MB, 98 MiB) copied, 11.5315 s, 8.9 MB/s

WSL Bakend (Volume)

docker run --rm -it -v docker-volume-test:/docker-test ubuntu
root@b882a775a77e:/# cd docker-test/
root@b882a775a77e:/docker-test# dd if=/dev/zero of=test.dat bs=1024 count=100000
100000+0 records in
100000+0 records out
102400000 bytes (102 MB, 98 MiB) copied, 0.119611 s, 856 MB/s
freeze000 commented 2 years ago

Sorry, not volume I meant bind of course

dovidweisz commented 2 years ago

Your stats show that io is a little faster for binds than Hyper-V.

But still not nearly as fast as a volume.

What I've been doing is running vs-code off Docker so I actually have all my code in a docker volume.

On Sun, Mar 13, 2022, 6:37 PM freeze000 @.***> wrote:

Sorry, not volume I meant bind of course

— Reply to this email directly, view it on GitHub https://github.com/docker/for-win/issues/188#issuecomment-1066196669, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABUTPCINGZ64SVWHVE3V6RDU7ZU3JANCNFSM4CUZ2CFQ . You are receiving this because you commented.Message ID: @.***>

msxdan commented 2 years ago

@dovidweisz

You may have a volume for multiple reasons, using a Development Container is a good choose for coding but not great for any other stuff, it just copies all inside a volume and creates a container.

Dealing with big files is horrible, we cannot be copying +100 GB to a volume.

I haven't tested in Windows 11 tho, they added some improvements

drungrin commented 2 years ago

Share the folder on windows side, mount the shares on the Wsl (ubuntu) distro mount -t cifs -o username=<omitted>,password=<omitted>,file_mode=0777,dir_mode=0777,vers=3.11 //host.docker.internal/Users/<omitted>/my-share /mnt/my-share and then create the volume docker volume create --opt type=none --opt o=bind --opt device=/mnt/my-share my-share and then use the volume.

Speed went from 40-60 MB/s using a CIFS volume to 1 GB/s. Still a lot slower than the mount point, 3GB/s, but enough for me.

speed tested with dd if=/dev/zero of=./testfile bs=1G count=1 oflag=direct inside busybox docker run --rm -it -v my-share:/my-share busybox

Windows 11 and everything on latest version.

embluk commented 2 years ago

I am still facing the same issues with Windows 11 and Ubuntu wsl. A web app that I am working on will take 15 seconds to load a page, when it usually would be less than 2 seconds. Is there still no solid solution?

docker-robott commented 2 years ago

Issues go stale after 90 days of inactivity. Mark the issue as fresh with /remove-lifecycle stale comment. Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. /lifecycle stale

DeyV commented 2 years ago

/remove-lifecycle stale /lifecycle frozen

embluk commented 2 years ago

Yeah, this is still a major problem.

dovidweisz commented 2 years ago

At this point it's more likely a problem with WSL2. I ran the test mentioned by OP in WSL2; in my Windows home directory and in my Linux home directory.

username@978TQJ3:/mnt/c/Users/username$ dd if=/dev/zero of=test.dat bs=1024 count=100000
100000+0 records in
100000+0 records out
102400000 bytes (102 MB, 98 MiB) copied, 9.48475 s, 10.8 MB/s
username@978TQJ3:/mnt/c/Users/username$ cd
username@978TQJ3:~$ dd if=/dev/zero of=test.dat bs=1024 count=100000
100000+0 records in
100000+0 records out
102400000 bytes (102 MB, 98 MiB) copied, 0.10946 s, 936 MB/s

The difference is approximately 100x.

docker-robott commented 1 year ago

Closed issues are locked after 30 days of inactivity. This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

/lifecycle locked