Open jmichalicek opened 1 year ago
Just tested on 4.17.0, still super broken :(
I have the same issue running webpck inside a docker in Mac, I can confirm 4.17.0 is still broken. gRPC FUSE does not have this problem, but it has a poor performance, a weback build with VirtioFS would take 30 seconds, while with gRPC FUSE 400 seconds.
@jmichalicek I believe I have the same issue but I cannot reproduce it using your example.
I think downgrading Docker Desktop to 4.14.1
may help. Could you see if that works?
I can consistently reproduce in an internal project when npm
is configured to store its cache in a VirtioFS shared volume. When switching to gRPC FUSE, the problem disappears.
(...)
npm WARN tarball tarball data for string-width@https://registry.npmjs.org/string-width/-/string-width-4.2.3.tgz (sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==) seems to be corrupted. Trying again.
npm WARN tarball tarball data for string-width@https://registry.npmjs.org/string-width/-/string-width-4.2.3.tgz (sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==) seems to be corrupted. Trying again.
npm WARN tarball tarball data for string-width@https://registry.npmjs.org/string-width/-/string-width-4.2.3.tgz (sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==) seems to be corrupted. Trying again.
npm WARN tarball tarball data for string-width@https://registry.npmjs.org/string-width/-/string-width-4.2.3.tgz (sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==) seems to be corrupted. Trying again.
npm WARN tarball tarball data for string-width@https://registry.npmjs.org/string-width/-/string-width-4.2.3.tgz (sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==) seems to be corrupted. Trying again.
npm WARN tarball tarball data for js-yaml@https://registry.npmjs.org/js-yaml/-/js-yaml-4.1.0.tgz (sha512-wpxZs9NoxZaJESJGIZTyDEaYpl0FKSA+FB9aJiyemKhMwkxQg63h4T1KJgUGHpTqPDNRcmmYLugrRjJlBtWvRA==) seems to be corrupted. Trying again.
npm WARN tarball tarball data for js-yaml@https://registry.npmjs.org/js-yaml/-/js-yaml-4.1.0.tgz (sha512-wpxZs9NoxZaJESJGIZTyDEaYpl0FKSA+FB9aJiyemKhMwkxQg63h4T1KJgUGHpTqPDNRcmmYLugrRjJlBtWvRA==) seems to be corrupted. Trying again.
npm ERR! code ENOENT
npm ERR! syscall rename
npm ERR! path /app/.npm-cache/_cacache/tmp/66556a64
npm ERR! dest /app/.npm-cache/_cacache/content-v2/sha512/b1/1b/b592970ef722ed63104abea7d37a1f4acd91303b7493c97d474fee02683cc2e87a5319884884f2338fd5ee294eca603c2769e87985c3b08f2d50b89cc13c
npm ERR! errno -2
npm ERR! enoent ENOENT: no such file or directory, rename '/app/.npm-cache/_cacache/tmp/66556a64' -> '/app/.npm-cache/_cacache/content-v2/sha512/b1/1b/b592970ef722ed63104abea7d37a1f4acd91303b7493c97d474fee02683cc2e87a5319884884f2338fd5ee2964eca603c2769e87985c3b08f2d50b89cc13c'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
node:18.14-alpine
(node 18.14.2 + npm 9.5.0)I experienced this bug with Docker Desktop 4.17.0 + VirtioFS, but it did not occur with version 4.18.0 Please try upgrading to version 4.18.0
I missed the activity going on here, but I just tested with Docker Desktop 4.19.0 + VirtioFS and my first run of heavy disk usage worked great.
Not sure if this is the same issue you all are talking about, but when I run the Spring Cloud Config Server in a Docker container (using virtiofs), and I modify some config files on the host (which are mounted into the config server container), the container randomly sees corrupted and partial content files for a while. Even waiting a few seconds is not enough for the container to see the correct file content.
Discussed in https://github.com/docker/roadmap/issues/7#issuecomment-1641132358
If I disabled the virtiofs support in Docker Desktop, the problem goes away and things work correctly. (Though many other common operations are slower of course, due to the inefficient file sync needed.)
@cdbennett if you still see the problem, could you provide a simplified reproduction with more details, in a separate issue please?
If I get a chance, I'll try to create a reproducible test case.
@fredericdalleau - I tried to write a minimal reproduction case but wasn't able to break it with that. So far only the full Docker Compose for our proprietary application causes the issue with virtiofs (it is 100% reproducible in this case). If I ever am able to make a minimal test case I'll follow up with a new issue.
I also experienced this problem but instead of npm with a mount in the container to a directory on host mac book built using maven, with VirtioFS and Docker 4.30.0. As VirtioFS is currently the default for fresh Docker installs, maybe the default should change to gRPC FUSE (at least for Mac Intel) until this issue is fixed?
Expected behavior
I do not believe this to be specific to npm install, it is just easy to replicate using npm. Running
npm install
ornpm ci
should be able to successfully install npm packages. This does work in multiple containers when usinggRPC FUSE
Actual behavior
When only installing one or two things at a time, installation usually works. ie.
npm install webpack
, but the more things that are added, the more frequently failures occur which suggests this is related to the amount of disk activity. Every package says that it is corrupted. Because this works when usinggRPC FUSE
I do not believe it to be an npm issue.With npm the issue manifests in multiple ways.
and
Information
Output of
/Applications/Docker.app/Contents/MacOS/com.docker.diagnose check
Steps to reproduce the behavior
docker build . -t virtiofs-demo
If it succeeds,
rm -rf /app/.npm /app/node_modules
and usenpm install
ornpm ci
This is the Dockerfile I used for testing which is a very stripped down version of what I usually work with. I have tested this with multiple base docker images including the latest ubuntu and python 3.11 images. Everything works fine with gRPC FUSE and also worked perfectly using VirtioFS when it was a beta feature.
If you just do a simple
npm install webpack
, etc. then usually everything works. The more packages inPackage.json
or specified on the command line, the more failures occur.Package.json