docker / for-mac

Bug reports for Docker Desktop for Mac
https://www.docker.com/products/docker#/mac
2.44k stars 118 forks source link

/var/lib/docker/tmp/docker-builderXXXXXXX/... no such file or directory #1922

Closed bjcatar closed 5 years ago

bjcatar commented 7 years ago

Expected behavior

COPY command to copy a file from local to image

Actual behavior

doing a simple COPY command in Dockerfile is throwing this error when the file is in a folder (not same level as Dockerfile file)

Information

Docker for Mac: version: 17.06.0-ce-mac19 (4cdec4294a50b2233146b09469b49937dabdebdd) macOS: version 10.12.6 (build: 16G29) logs: /tmp/04FF32A9-CA64-40EB-B0DC-AD3E13A6C7D5/20170731-232145.tar.gz [OK] db.git [OK] vmnetd [OK] dns [OK] driver.amd64-linux [OK] virtualization VT-X [OK] app [OK] moby [OK] system [OK] moby-syslog [OK] db [OK] env [OK] virtualization kern.hv_support [OK] slirp [OK] osxfs [OK] moby-console [OK] logs [OK] docker-cli [OK] menubar [OK] disk

FROM python:latest RUN apt-get update RUN apt-get install nano RUN mkdir /code WORKDIR /code COPY /helpdesk/requirements.txt /code/

Steps to reproduce the behavior

  1. docker build -t test .
bggolding commented 7 years ago

I'm seeing the same problem with ADD in my Dockerfile as well

bjcatar commented 7 years ago

Is there any news related to this issue? I had to downgrade to make it work. Today I upgraded and surprisingly it worked for a little bit, but soon it started failing again.

bggolding commented 7 years ago

I tried your example above and I changed the COPY line to:

COPY helpdesk/requirements.txt /code/

and made a helpdesk/requirements.txt in the directory next to the Dockerfile and I was able to build a docker image.

I'm not sure what the error message is trying to communicate but it is singularly unhelpful.

chb0github commented 7 years ago

it's definitely related to using a fully qualified path. if I put:

ADD /Volumes/dev/sterling/java-user-login-service/user-login-server/build/libs/user-login-server-1.0.5-SNAPSHOT.jar /user-login-server-1.0.5-SNAPSHOT.jar
ENTRYPOINT ["sh", "-c", "java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -jar /user-login-server-1.0.5-SNAPSHOT.jar"]

then I get Step 1/2 : ADD /Volumes/dev/sterling/java-user-login-service/user-login-server/build/libs/user-login-server-1.0.5-SNAPSHOT.jar /user-login-server-1.0.5-SNAPSHOT.jar ADD failed: stat /var/lib/docker/tmp/docker-builder663214013/Volumes/dev/sterling/java-user-login-service/user-login-server/build/libs/user-login-server-1.0.5-SNAPSHOT.jar: no such file or directory

But if I change it to:

ADD build/libs/user-login-server-1.0.5-SNAPSHOT.jar /user-login-server-1.0.5-SNAPSHOT.jar
ENTRYPOINT ["sh", "-c", "java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -jar /user-login-server-1.0.5-SNAPSHOT.jar"]

it all works

docker build -t mothra/user-service:1234 -f ./user-login-server/Dockerfile ./user-login-server
Sending build context to Docker daemon    363MB
Step 1/2 : ADD build/libs/user-login-server-1.0.5-SNAPSHOT.jar /user-login-server-1.0.5-SNAPSHOT.jar
pbranforn commented 7 years ago

Fails on me, too. Worked with previous versions. Is this still present or any update avail? Which version do I have to revert to, to make this work again?

Gaibhne commented 7 years ago

This is happening to me too with simple case of

COPY content /app/

Any variations with ./content, /app produce the same error.

Note that I am seeing this problem both on CentOS and Ubuntu.

bjcatar commented 7 years ago

I installed the edge version and I have been able to, for right now, continue to work.

Sent from my iPhone

On Aug 10, 2017, at 7:05 AM, Gaibhne notifications@github.com wrote:

This is happening to me too with simple case of

COPY content /app/

Any variations with ./content, /app produce the same error.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

puneetjindal commented 7 years ago

@bjcatar what is the version which resolved this?

bsodmike commented 7 years ago

Ugh, as @Gaibhne pointed out, even a relative path such as COPY content /app or COPY ./content /app produces the same error.

Using Docker version 17.06.0-ce, build 02c1d87 in Ubuntu 16.04.3 LTS. Thoughts?

bsodmike commented 7 years ago

@bjcatar Do please let us know which version you're running via docker version?

I've just removed my stable CE install and replaced with test, since edge is the same version as stable.

Client:                                                                       │
 Version:      17.07.0-ce-rc2                                                 │
 API version:  1.31                                                           │
 Go version:   go1.8.3                                                        │
 Git commit:   36ce605                                                        │
 Built:        Mon Aug  7 23:44:33 2017                                       │
 OS/Arch:      linux/amd64

I still get the same error though with 17.07.0-ce-rc2, build 36ce605 or 17.06.0-ce, build 02c1d87

bsodmike commented 7 years ago

Ha, figured it out. Was attempting the build from the wrong path. /facepalm.

chb0github commented 7 years ago

@bsodmike Explain. What does your current file look like? I am trying to add with with full paths and it doesn't work

jimschubert commented 7 years ago

I just had this problem, although I could copy some files and not others. Turned out that I had a parent directory excluded in .dockerignore and a directory I renamed was no longer being explicitly allowed.

stewartmoreland commented 7 years ago

Also having the same issues on both stable & edge versions. I'm using the spotify/dockerfile-maven plugin and getting similar results and I don't have a .dockerignore file.

bjcatar commented 7 years ago

Sorry about the late reply, was away from my mac the last few days. Here is the info for the version I now have that works:

Client: Version: 17.07.0-ce-rc1 API version: 1.31 Go version: go1.8.3 Git commit: 8c4be39 Built: Wed Jul 26 05:20:09 2017 OS/Arch: darwin/amd64

Server: Version: 17.07.0-ce-rc1 API version: 1.31 (minimum version 1.12) Go version: go1.8.3 Git commit: 8c4be39 Built: Wed Jul 26 05:25:01 2017 OS/Arch: linux/amd64 Experimental: true

On Aug 10, 2017, at 8:33 PM, Michael de Silva notifications@github.com wrote:

@bjcatar https://github.com/bjcatar Do please let us know which version you're running via docker version?

I've just removed my stable CE install and replaced with test, since edge is the same version as stable.

Client: │ Version: 17.07.0-ce-rc2 │ API version: 1.31 │ Go version: go1.8.3 │ Git commit: 36ce605 │ Built: Mon Aug 7 23:44:33 2017 │ OS/Arch: linux/amd64 I still get the same error though with 17.07.0-ce-rc2, build 36ce605 or 17.06.0-ce, build 02c1d87

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/docker/for-mac/issues/1922#issuecomment-321708500, or mute the thread https://github.com/notifications/unsubscribe-auth/AFSejO2pNkBSIUVhEGQN4HATGLsuG_doks5sW6FXgaJpZM4OpL3k.

amellnik commented 7 years ago

I'm seeing this on Docker for Windows 17.06.1-ce, build 874a737. ADD and COPY commands that worked before an update a few days ago now give errors like COPY failed: stat /var/lib/docker/tmp/docker-builder200899165/opt/julia: no such file or directory when I try to do something like COPY Make.user /opt/julia.

dinesh-papineni commented 7 years ago

I see this issue on Docker version 17.06.1-ce, build 874a737 too . The COPY failed following error COPY failed: stat /var/lib/docker/tmp/docker-builder426864723/build: no such file or directory

Does a downgrade to previous versions fix this?

matthieu-merlyn commented 7 years ago

Confirming that I am facing similar issues with version 17.06.1-ce, build 874a737 on Sierra 10.12.6

COPY failed: stat /var/lib/docker/tmp/docker-builder384870922/build: no such file or directory

EDIT: same issue on edge version 17.07.0-ce-rc4, build fe143e3

wpecker commented 7 years ago

This question is because the .dockerignore file is excluded, so this is the result of the suggestion that "COPY failed: stat /var/lib/docker/tmp/docker-builder902282144/tag: no such file or directory"

Alternative: COPY failed: stat /var/lib/docker/tmp/docker-builder902282144/tag: no such file or directory, please check whether the dockerignore file has excluded, There may be aaa file rule excluded "

RandyLarson commented 7 years ago

RESOLUTION FOR ME -- the .dockerignore file was the culprit. It included a path that included my COPY src folder. Revising the .dockerignore resolved my issue.

wpecker commented 7 years ago

@RandyLarson same as you, I recommend giving detailed hints, or possibly the situation, including the .dockerignore file

orangewolf commented 7 years ago

For me the offending line was "**/log/*" and the directory it would not COPY was vendor/gems/blog. Note that a) this ignore worked fine with dinghy which is docker-machine based, but didn't work on Docker.app for Mac. b) given the slashes I do not believe that "*/log/" should match blog

rkevinburton commented 7 years ago

I don't have a .dockerfile so this isn't the cause for me.

Ethreal commented 7 years ago

+1

netspark commented 7 years ago

+1, no dockerignore and still fails to COPY

TyIsI commented 7 years ago

Resolution for me was to make sure that the directory to be copied was a sub-directory of the build directory:

FROM nginx

COPY web /usr/share/nginx/html

docker build -t <image> -f docker/images/nginx/Dockerfile .

The docker build process copies all the files in the build directory to the docker-machine and it gets built from within the VM.

rkevinburton commented 7 years ago

The problem for me is that the folder that I want to copy from is a sub-directory of the directory where the Dockerfile is. But, I only want to copy one file from that directory not the entire contents.

From: TyIsI [mailto:notifications@github.com] Sent: Friday, September 29, 2017 1:01 PM To: docker/for-mac for-mac@noreply.github.com Cc: Kevin (Ronald) Burton rburton@agsource.com; Comment comment@noreply.github.com Subject: Re: [docker/for-mac] /var/lib/docker/tmp/docker-builderXXXXXXX/... no such file or directory (#1922)

Resolution for me was to make sure that the directory to be copied was a sub-directory of the build directory:

FROM nginx

COPY web /usr/share/nginx/html

docker build -t -f docker/images/nginx/Dockerfile .

The docker build process copies all the files in the build directory to the docker-machine and it gets built from within the VM.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/docker/for-mac/issues/1922#issuecomment-333196367, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AdmtTfWApPv0HtXgWorjARl_-oKxGXfNks5snTBNgaJpZM4OpL3k.

SamFromDaUk commented 7 years ago

Deleting .dockerignore file resolved the issue for me

wpecker commented 7 years ago

@SamFromDaUk same as you

swbfraser commented 7 years ago

For other newbs... the stat /var/lib/docker/tmp/docker-builderXXXXXX is not important. The rest of the text is the location seems to show where it is looking relative to the dockerfile. The important bit is the message that the file cannot be found. As @rkevinburton mentioned, it seems the file must be relative to the location of the dockerfile file.

rkevinburton commented 7 years ago

Also it is worth considering the permissions on the destination.

From: Steve Fraser [mailto:notifications@github.com] Sent: Wednesday, October 4, 2017 9:34 PM To: docker/for-mac for-mac@noreply.github.com Cc: Kevin (Ronald) Burton rburton@agsource.com; Mention mention@noreply.github.com Subject: Re: [docker/for-mac] /var/lib/docker/tmp/docker-builderXXXXXXX/... no such file or directory (#1922)

For other newbs... the stat /var/lib/docker/tmp/docker-builderXXXXXX is not important. The rest of the text is the location seems to show where it is looking relative to the dockerfile. The important bit is the message that the file cannot be found. As @rkevinburtonhttps://github.com/rkevinburton mentioned, it seems the file must be relative to the location of the dockerfile file.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/docker/for-mac/issues/1922#issuecomment-334342371, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AdmtTcRgHXRBvDyFnnerFIo-nD3xBmMXks5spEARgaJpZM4OpL3k.

sp-7indigo commented 7 years ago

i have the this strange behaviour when pipe a Dockerfile from stdin and trying to copy a simple file for example

 docker build --network=host -t my/slate -<Slate.Dockerfile

returns

Step 5/11 : COPY ./info.txt /slate/source/info.txt
COPY failed: stat /var/lib/docker/tmp/docker-builder373088334/info.txt: no such file or directory

by renaming Slate.Dockerfile to Dockerfile and

docker build --network=host -t my/slate .

everything works fine!

rasekar2000 commented 7 years ago

Same issue in Version 17.09.0-ce-mac35 (19611) -- Channel: stable - a98b7c1b7c as well Tried your suggestion @sp-7indigo as well. Still not working !!

rodibrin commented 7 years ago

while following some samples running docker build -< dockerfile i got the same issue on windows. i still dont know what the "-" is for but running docker build . -f dockerfile it worked.

lfoppiano commented 7 years ago

Hi all, I have the same issue here with the ADD command.

I've tried several workaround, like setting the absolute path of the file I wanted to add, or try to pass an environment variable with the absolute prefix. Nothing worked. Regarding the permission issues, I'm trying with root and it returns the same result.

I checked and I do not have any .dockerignore file around.

Here my docker file:

FROM jetty:9.4-jre8

ARG GROBID_VERSION
ENV GROBID_VERSION=$GROBID_VERSION
ENV JAVA_OPTS=-Xmx4g

#RUN env
LABEL Description="GROBID (or Grobid) - GeneRation Of BIbliographic Data." Version="${GROBID_VERSION}"

USER root
RUN apt-get update && apt-get -y --no-install-recommends install libxml2

#USER jetty
ADD ./grobid-home/target/grobid-home-${GROBID_VERSION}.zip /opt
#ADD /Users/lfoppiano/development/grobid/grobid-home/target/grobid-home-${GROBID_VERSION}.zip /opt
RUN unzip /opt/grobid-home-${GROBID_VERSION}.zip -d /opt && rm /opt/grobid-home-${GROBID_VERSION}.zip

COPY ./grobid-service/target/grobid-service-${GROBID_VERSION}.war /var/lib/jetty/webapps/ROOT.war

# Workaround otherwise the tmp directory is not writeable (owner is root)
RUN mkdir /opt/grobid-home/tmp
RUN chown -R jetty:jetty /opt/grobid-home/
VOLUME /opt/grobid-home/tmp

Thanks Luca

--- Update 06/11/2017: The command I used to build: docker build -t lfoppiano/grobid:0.4.4 --build-arg GROBID_VERSION=0.4.4 .

JasperBeaurain commented 7 years ago

I am having the same issue on ubuntu:16.04 with builds started in jenkins.

Step 4/8 : COPY nginx/nginx.conf /etc/nginx/nginx.conf Service 'nginx' failed to build: COPY failed: stat /var/lib/docker/tmp/docker-builder508249444/nginx/nginx.conf: no such file or directory script returned exit code 1

gautamsinghania95 commented 7 years ago

+1 facing issue in my MAC OS Sierra 10.12.6 ADD command : path /var/lib/docker/tmp/docker-builderXXXXXXXXX/ gets prepended Searched in many issue chains, most popular solutions were as follows :

  1. Add/remove the slash in pathname - Has no effect
  2. Remove .dockerignore file - No such file present in my case
gautamsinghania95 commented 7 years ago

Replying to my previous comment, I have resolved the issue.

Fix : Apparently, there is some problem with the string encoding of the Dockerfile. I simply wrote the whole Dockerfile again in a new file (editor : Sublime), and this worked. The Dockerfile obtained previously was generated from a Gradle script.

Please be mindful of these changes and try it.


+1 facing issue in my MAC OS Sierra 10.12.6ADD command : path /var/lib/docker/tmp/docker-builderXXXXXXXXX/ gets prependedSearched in many issue chains, most popular solutions were as follows :Add/remove the slash in pathname - Has no effectRemove .dockerignore file - No such file present in my case

lfoppiano commented 7 years ago

@gautamsinghania95 mmmm, no offense, but I find it hard to believe, honestly.

I have written the Dockerfile by myself with intellij in utf-8 and it was working 3-6 months ago. How this can have been changed for me?

Could you please share both docker files?

Thanks

chb0github commented 7 years ago

@lfoppiano Until I read @gautamsinghania95 post it wouldn't have occurred to me but I generated my docker file using gradle on mac identically.

EamonKeane commented 7 years ago

Something similar happened to me also, I'll share my resolution in case it helps others. I was originally implementing:

docker build --tag mytag - < Dockerfile.dev

This started to build the file and then gave the error:

COPY failed: stat /var/lib/docker/tmp/docker-builder842485110/sites: no such file or directory

When I changed the name of the Dockerfile.dev to just Dockerfile and replaced the command with:

docker build --tag mytag .

It worked. This appears to be due to the fact the 'build context' was not being piped into docker when not using the . command.

(on Docker for Mac Version 17.09.0-ce-mac35 (19611) Channel: stable a98b7c1b7c)

lfoppiano commented 7 years ago

@EamonKeane I didn't write it initially in my comment but I've been always using docker build --tag bla .. I've added an update to it.

@chb0github did the suggestion from @gautamsinghania95 work?

jcollum commented 7 years ago

Same here and after getting through this massive thread I see a few solutions:

None of these are working. Frustrating; my dockerfile is so simple.

Docker for Mac 2017-10-06 17.09.0-ce-mac35

Here's the complete output:

➜  cat Dockerfile 
FROM mysql/mysql-server:5.7.20

COPY ./sql/create_db.sql /docker-entrypoint-initdb.d
RUN  alias ll='ls -alh'
ENV MYSQL_ROOT_PASSWORD mypassword
EXPOSE 1307

➜ cat sql/createdb.sql  
create database duplicates;
create user 'apiuser'@'%' IDENTIFIED BY 'lakjsdlfkuo2384789798!';
GRANT ALL PRIVILEGES ON duplicates.* TO 'apiuser'@'%' WITH GRANT OPTION;
FLUSH PRIVILEGES;

➜  docker build -t mysql-docker .
Sending build context to Docker daemon  82.94kB
Step 1/5 : FROM mysql/mysql-server:5.7.20
 ---> a3ee341faefb
Step 2/5 : COPY ./sql/create_db.sql /docker-entrypoint-initdb.d
COPY failed: stat /var/lib/docker/tmp/docker-builder844903334/sql/create_db.sql: no such file or directory
jovanchohan commented 7 years ago

I changed my command to "docker build ." and it worked

lfoppiano commented 7 years ago

Yes so I've tried recently without changing anything and it magically worked....

I'm using

19:04 $ docker --version
Docker version 17.09.0-ce, build afdb6d4

I don't know which other useful information can be actually shared.

dracan commented 6 years ago

@swbfraser For other newbs... the stat /var/lib/docker/tmp/docker-builderXXXXXX is not important. The rest of the text is the location seems to show where it is looking relative to the dockerfile. The important bit is the message that the file cannot be found. As @rkevinburton mentioned, it seems the file must be relative to the location of the dockerfile file.

This helped me solve my problem - which was just a case sensitivity issue in my path. However, I do feel the fact that the path in the error message is prefixed with /var/lib/docker/tmp/docker-builderXXXXXX is VERY misleading. It stopped me from spotting the real issue, because I had no idea what this path was, which certainly wasn't my current working directory. I'd argue that this error message could be classified as a bug in itself. Anyway, thanks @swbfraser for pointing this out, which led me to find my real issue.

caramcc commented 6 years ago

This error message was confusing, and none of the above helped. Here's why I was getting this cryptic error message:

  1. I was using an absolute path
  2. I was trying to copy a directory outside of the build context (a different subdirectory of the parent directory)

Switching to a relative path (e.g. from /home/me/workspace/dockerstuff to ../foo_dir) got me the helpful error message: COPY failed: Forbidden path outside the build context: ../foo_dir

hardikmodha commented 6 years ago

I wanted to copy only one of the sub-directory where the Dockerfile is present.

|
|---build
       |---dist
             |---app.tar
|--Dockerfile

Dockerfile:

FROM openjdk:8u151-jre
ADD build/dist/app.tar /dummy/

// Rest of the instructions

The following command worked for me,

docker build -t <image:version> -f ./Dockerfile .
louisbuchbinder commented 6 years ago

I was having this same issue it seems like inline comments on the COPY line was causing the error.

Failing

COPY . /app # Copy application contents to container

Working

# Copy application contents to container
COPY . /app 
wilder commented 6 years ago

removing some paths from .dockerignore solved for me