docker / for-mac

Bug reports for Docker Desktop for Mac
https://www.docker.com/products/docker#/mac
2.43k stars 118 forks source link

/var/lib/docker/tmp/docker-builderXXXXXXX/... no such file or directory #1922

Closed bjcatar closed 5 years ago

bjcatar commented 7 years ago

Expected behavior

COPY command to copy a file from local to image

Actual behavior

doing a simple COPY command in Dockerfile is throwing this error when the file is in a folder (not same level as Dockerfile file)

Information

Docker for Mac: version: 17.06.0-ce-mac19 (4cdec4294a50b2233146b09469b49937dabdebdd) macOS: version 10.12.6 (build: 16G29) logs: /tmp/04FF32A9-CA64-40EB-B0DC-AD3E13A6C7D5/20170731-232145.tar.gz [OK] db.git [OK] vmnetd [OK] dns [OK] driver.amd64-linux [OK] virtualization VT-X [OK] app [OK] moby [OK] system [OK] moby-syslog [OK] db [OK] env [OK] virtualization kern.hv_support [OK] slirp [OK] osxfs [OK] moby-console [OK] logs [OK] docker-cli [OK] menubar [OK] disk

FROM python:latest RUN apt-get update RUN apt-get install nano RUN mkdir /code WORKDIR /code COPY /helpdesk/requirements.txt /code/

Steps to reproduce the behavior

  1. docker build -t test .
psypersky commented 6 years ago

Having the docker-compose and the Dockerfile on the same folder solved it for me

realtebo commented 6 years ago

so, no solution? I cannot move / copy my file before building

pelzerim commented 6 years ago

I encountered this problem when trying to building from another folder. When specifying build context it worked: docker build -f ./path/to/dockerfile/Dockerfile ./path/to/dockerfile/

NargiT commented 6 years ago

I have the same issue, docker cannot find mu file.

Client: Version: 17.12.0-ce API version: 1.35 Go version: go1.9.2 Git commit: c97c6d6 Built: Wed Dec 27 20:03:51 2017 OS/Arch: darwin/amd64

Server: Engine: Version: 17.12.0-ce API version: 1.35 (minimum version 1.12) Go version: go1.9.2 Git commit: c97c6d6 Built: Wed Dec 27 20:12:29 2017 OS/Arch: linux/amd64 Experimental: true

In my Dockerfile COPY buid/distributions/akuna-matata.tar /opt/

The build docker build -t nargit/akuna-mata:latest -f Dockerfile .

And the output Step 3/5 : COPY buid/distributions/akuna-matata.tar /opt/ COPY failed: stat /var/lib/docker/tmp/docker-builder751334749/buid/distributions/akuna-matata.tar: no such file or directory

and I do not have any .dockerignore

any news ?

safaozturk93 commented 6 years ago

I recommend you to put your id_rsa and id_rsa.pub files into the same directory with Dockerfile if you are just trying to do something not commercial or not for production. In my case, it worked.

joseccruz commented 6 years ago

It is the .dockerignore file ... even if its contents don't make sense for the path you are having issues with, just delete the whole file and try again.

shashank-k commented 6 years ago

For all those who have problems with build context using a .

The simplest fix is to append a / to the end.

For example, instead of

docker build -t nargit/akuna-mata:latest -f Dockerfile .

do this

docker build -t nargit/akuna-mata:latest -f Dockerfile ./

This worked for me well.

emyller commented 6 years ago

I stumbled upon the same error message today, and I was absolutely sure there was no error in my Dockerfile, .dockerignore or the build context (directory jazz):

COPY requirements.txt /app

The simple statement above was erroring out just because the file requirements.txt was actually a symbolic link to somewhere-else/requirements.txt, and somewhere-else was not part of the build context. Took me a while and a coffee to figure this one out.

peterbe commented 6 years ago

I was here on this issue, amongst many other google/stackoverflow searches, not being able to see how it relates to my frustrating "no such file or directory".

My solution was to discover that the whole directory was mentioned in .dockerignore. I blogged about it here: https://www.peterbe.com/plog/docker-gotcha-with-building-a-dockerfile-in-sub-directory

mreilaender commented 6 years ago

I have the same problem and the same use case as @gautamsinghania95. My Dockerfiles are also built by gradle and I get the same error. When creating a new file manually and inserting the contents from the gradle generated dockerfile it works.

johnknoop commented 6 years ago

Make sure you don't have a .dockerignore file which excludes the files you want to copy. In my case I had, and it produces the same error message.

mreilaender commented 6 years ago

I deleted the Dockerfile and it didnt work. Btw I'm not on MAC im using it on Ubuntu.

Hithesh26 commented 6 years ago

hi, i have same issue,i tried most of above solutions,but still im getting error

Step 12/15 : COPY mvn-entrypoint.sh /usr/local/bin/mvn-entrypoint.sh COPY failed: stat /mnt/sda1/var/lib/docker/tmp/docker-builder833767886/mvn-entry point.sh: no such file or directory

also i searched .dockerignore file, i did't find it

AnandHirekatur commented 6 years ago

I am using a Mac and had forgotten to create/copy the .sh file(s) in/to the same directory as Dockerfile and this was giving me the COPY failed error.

ashahba commented 6 years ago

I'm trying a simple but important scenario here on a Mac:

docker build --build-arg SSH_PRIV_KEY=~/.ssh/id_rsa -f Dockerfile.dev .

Why because I need to access some remote ssh resources during the build. Sadly I get:

COPY failed: stat /var/lib/docker/tmp/docker-builder760998862/Users/ashahba/.ssh/id_rsa: no such file or directory

But I have:

$ ls ~/.ssh/id_rsa 
/Users/ashahba/.ssh/id_rsa

The alternative is to copy ~/.ssh/id_rsa to directory where docker build is being called, but I hate doing that since I don't want to rely or my memory to remove the copy after I'm done πŸ™

andrew-laughlin commented 6 years ago

Hit this issue on Windows, using version 18.03.0-ce-win59.

From the docs for COPY

Multiple resources may be specified but the paths of files and directories will be interpreted as relative to the source of the context of the build.

So changing my docker file COPY from:

WORKDIR /full/path/from/build/context COPY ${MY_FILE_VARIABLE} NewFile.tmp

to

WORKDIR /full/path/from/build/context COPY full/path/from/build/context/${MY_FILE_VARIABLE} NewFile.tmp

worked. However, it doesn't make sense to me why WORKDIR seems to be ignored.

kestrelbright commented 6 years ago

same issue. `docker-compose build Building test

Step 1/9 : FROM daocloud.io/library/ubuntu:16.04

---> f975c5035748

Step 2/9 : ENV HOME=/home

---> Using cache

---> 0ee73be8050b

Step 3/9 : ENV USER coco

---> Using cache

---> c36911cb7f4c

Step 4/9 : COPY a.sh $HOME

ERROR: Service 'test' failed to build: COPY failed: stat /var/lib/docker/tmp/docker-builder663672165/a.sh: no such file or directory` Version 17.12.0-ce-mac55, stable.

i put a.sh in a folder, use COPY folderName $HOME, then it's ok.

Dhrumil1808 commented 6 years ago

Hi, I also had the same issue as above. For me, the Dockerfile was incorrect.

I was not getting the proper location of the JAR file in the Dockerfile.

micoli commented 6 years ago

Same problem.

Solution: For me, the file I want to copy was in .gitignore file, check yours

SpaceNet commented 6 years ago

same problem

My docker app for MacOS is at /usr/local/bin/docker

but docker try to COPY failed: stat /var/lib/docker/tmp/docker-builder307536658

Does this the matter?

$ which docker
/usr/local/bin/docker

I cannot go beyond the first step of docker instruction....

jamesoflol commented 6 years ago

For me it was because I had a comment on the same line. Turns out this isn't allowed in Dockerfile spec.

Good:

# Next command will copy stuff
COPY . /code/

Bad:

COPY . /code/ # This line copies stuff

Resulting in:

COPY failed: stat /var/lib/docker/tmp/docker-builder615180871/code: no such file or directory

So the lesson is that no such file or directory means absolutely anything went wrong

YRM64 commented 6 years ago

Thanks developers, you all had something very valuable to contribute to the community. Chb0github and @swbfraser's comments on utilizing a fully qualified path, while caramss emphasized "a relative path" as a solution. jimschubert's comments on the significance of including a parent directory in .dockerignore. And, what was also interesting was the similarty in experiences with the .dockerignore file. Tylsl and rkevinburton's comments on sub-directory of the build directory, sp-7indigo and EamonKeane added the renaming of their files to just Dockerfile and made other slight modifications, all were successful. @gautamsinghania95 and chb0github discussed how they created their Dockerfile; jovanchohan's discussion centered around changing his command to "Docker build". And, interestingly enough, psypersky's comment on having Docker-compose and the Dockerfile on the same folder solved his issue. Pelzerim added ,"specifying build context worked for me", and an important note by johnknoop and micoli of not including files that you want to copy in .dockerignore, and the significance of .sh files. Other, just as important comments by developers, too many folks to mention here, I say thank you to all of you for making our jobs easier. Collaboration and networking are a few of our strongest tools. In other words, we need each other. Thanks again to all of you.

cunningj commented 6 years ago

For docker-compose users building with "docker-compose build"...

The context you define in docker-compose.yml overrides the normal context of the Dockerfile e.g.

user$ ls myDockerDir 
          Dockerfile file.txt

COPY ./file.txt /var/file.txt 

...normally works fine because of where file.txt is located relative to the Dockerfile. However, if you are trying to overcome other context related issues with your containers/setup and alter your docker-compose.yml this could affect the build context as well. Let's cd a level up from myDockerDir/ to myDir/ and assume docker-compose.yml lives there...

user$ ls myDir 
          docker-compose.yml myDockerDir

COPY ./file.txt /var/file.txt (from Dockerfile in myDockerDir) 

api:
  build:
    context: ../
    dockerfile: ./myDockerDir/Dockerfile

This will make "docker-compose build" fail due to a "no such file". Nothing has changed in terms of where file.txt is in relation to Dockerfile but the COPY statement needs to conform to what is defined in docker-compose.yml. For this setup the correct COPY statement would then be

COPY ./myDir/myDockerDir/file.txt /var/file.txt

geretz commented 6 years ago

For me it was a trailing semicolon at the end of the COPY line. Please improve the error message

nbekris commented 6 years ago

Have a fix.

Here is what my original code looks like: COPY /TestCode TestCode/

I decided to try this and it magically worked:

WORKDIR /TestCode

COPY . TestCode/

Tried most of the other solutions here so this might work for other people as well.

Vanco commented 6 years ago

I have same issue. block me two days. Finally, I fixed it by comment out the line

//mappings in Docker := (mappings in Universal).value

For detail: my Dockerfile generated by SBT

FROM java
LABEL MAINTAINER="Van Fan "
WORKDIR /opt/docker
ADD --chown=daemon:daemon opt /opt
EXPOSE 9988
RUN ["mkdir", "-p", "/opt/docker/logs", "/opt/docker/conf"]
RUN ["chown", "-R", "daemon:daemon", "/opt/docker/logs", "/opt/docker/conf"]
VOLUME ["/opt/docker/logs", "/opt/docker/conf"]
USER daemon
ENTRYPOINT ["/opt/docker/bin/akka-udpserver"]
CMD []

when I run

docker:publishLocal

The error occur at the ADD command.

I studied my old project, found that, the 'target/docker/stage' directory don't have opt/docker sub-directory. It mean there has something wrong in mappings .

So I try to comment out the mappings setting in build.sbt file. It works.

Help this will helpful.

patil16nit commented 6 years ago

In case you get these error of missing file/dir /var/lib/docker/tmp/docker-builderXXXXXXX/ use relative path from the Dockerfile directory to the actual directory to be copied

Docker version 17.12.0-ce

Directory tree

web-project
β”œβ”€β”€ docker-compose.yml
└── web
    β”œβ”€β”€ Dockerfile
    β”œβ”€β”€ index.php
    └── src
        └── info.php

Dockerfile:

FROM php:5.6-apache
COPY ./index.php  index.php
COPY ./src src
EXPOSE 80

docker-compose.yml

version: '3'
services:
  php-web:
    build:
      context : web
      dockerfile: Dockerfile
    ports:
      - 80:80
    tty: true

build image docker-compose up -d

alejandrovelez7 commented 6 years ago

@nbekris are you sure your files from the host were copied? I feel like you would just have switched to that directory in the container, but still copy the files from whatever directory the Dockerfile was in the host.

rcjamessr commented 6 years ago

Thanks nbekris! This worked for me
WORKDIR /x/repo/docker/buildfiles

But without .

COPY somefile.txt /usr/share/nginx/somefile.txt

yuzp1996 commented 6 years ago

This is my first time write something here

I meet this question today my dockerfile have copy like this

COPY requirements /requirements

when I run

docker build .

I get this

/var/lib/docker/tmp/docker-builder32123123/requments  no such file or directory 

I solve this by moving requirements to the folder where my dockerfile is and it work for me

jimmychang851129 commented 6 years ago

In my case, it's because there's something wrong in python2.7.15:onbuild , perhaps requirements.txt I solve this problem by changing the image to python:2.7-jessie

ghuser commented 6 years ago

On ubuntu I got this message because I had a typo in the source path. When I corrected the path, it worked. It seems the error message is kind of misleading. If it docker doesn't find the source file in the specified relative path it throws an error about another path /var/lib/docker/tmp/docker-builder*

jrichardsz commented 6 years ago

Hi @nbekris @rcjamessr I have just tried your solution using WORKDIR to allow copy files from external location to Dockerfile in build stage.

Not works for me :(

According to official documentation https://docs.docker.com/v17.09/engine/reference/builder/#workdir WORKDIR is just to set folder path inside docker container , not to mapping absolute paths from host.

RUN pwd
ENV MY_HOME "/host_path"
WORKDIR $MY_HOME
RUN pwd

Result is :

Step 3/24 : RUN pwd
---> Running in a1f4f1077c2c
/
Removing intermediate container a1f4f1077c2c
 ---> 3d7529df9ebc
Step 4/24 : ENV MY_HOME "/host_path"
 ---> Running in 2afcf2d319a4
Removing intermediate container 2afcf2d319a4
 ---> b67ca7ff4a3f
Step 5/24 : WORKDIR $MY_HOME
 ---> Running in 4737b707b4dd
Removing intermediate container 4737b707b4dd
 ---> 3f2fb47640a2
Step 6/24 : RUN pwd
 ---> Running in 3a88be01bd8a
/host_path

And as I know, RUN command execute inside container.

My docker version : Docker version 18.06.0-ce

So, copy files from external location to Dockerfile build folder , during build image stage , is unsolved!!

Thanks.

obaid1990 commented 6 years ago

many reasons for this issue 1)can be because there is a running container already (have to stop running container) 2)there is a problem with your dockerfile 3)usually with anguallar projects you have to do a gulp build on client-side to add dist file to your project go to the client side: npm install (Install all nodejs dependencies) sudo npm install bower -g (Install bower golbally) bower install (Automatically installs all bower dependencies) gulp build

jrichardsz commented 6 years ago

Hi @obaid1990

I think you do not understand what We are saying. Is simple :

How to include files outside of Docker's build context?

Currently, at build stage We can COPY or ADD files that are in the same folder of Dockerfile. If We need to pass some file in other location, an error is throwed:

/var/lib/docker/tmp/docker-builderXXXXXXX/... no such file or directory

WORKAROUND : Pass file as text using build args

In this example :

Dockerfile

ARG SSH_KEY
ENV SSH_KEY=$SSH_KEY
RUN echo "$SSH_KEY" > /root/.ssh/id_rsa

Pass file as text using build args

docker build -t some-app --build-arg SSH_KEY="$(cat ~/file/outside/build/context/id_rsa)" .

This worked for me.

Source: https://stackoverflow.com/a/47544999/3957754

nabito commented 6 years ago

I'd like to add some idea here. Mine just got the same error but finally found that the error is referring to the non-existing file because of typo in file extension.

It would not be misleading if docker haven't shown /var/lib/docker/tmp/docker-builderXXXXXXX/

And it would be even greater if it could give more useful error message to tell where exactly it was looking for the file (context) and couldn't find it.

jrichardsz commented 6 years ago

Is simple to reproduce :

Dockerfile

FROM python:latest
COPY /tmp/file/outside/docker/build/context/miau miau

Create this file /tmp/file/outside/docker/build/context/miau before build Dockerfile

Error will be :

/var/lib/docker/tmp/docker-builderXXXXXXX/... no such file or directory

Reason : /tmp/file/outside/docker/build/context/miau is out of Dockerfile context.

yarix commented 6 years ago

FYI: I experienced the same error on Windows: COPY failed: stat /var/lib/docker/tmp/docker-builderXXXX The problem in my case was a Windows path separator, so: COPY myApp\target\myApp.jar /apps/ should be replaced with: COPY myApp/target/myApp.jar /apps/

KanybekMomukeyev commented 6 years ago

I experienced the same error on Ubuntu:

ADD proceeded ERROR: Service 'smth' failed to build: ADD failed: stat /var/lib/docker/tmp/docker-builder832229414/proceeded: no such file or directory

Just to fix, my Git ignored that folder, so there was no such folder. So i edited my gitignore file, and everything is ok

prasad-marne commented 5 years ago

Any updates on the issue. I am facing similar situation with latest version.

sleekweasel commented 5 years ago

I got this because I tried to comment my COPY lines:

COPY this fails/   # Example of failing line

whereas

# This line succeeds
COPY this succeeds/

doesn't fail.

sabrehagen commented 5 years ago

Here's a repro:

FROM gliderlabs/logspout
RUN apk add curl
> docker build -t my-logspout .

Sending build context to Docker daemon  2.048kB
Step 1/2 : FROM gliderlabs/logspout
# Executing 3 build triggers
COPY failed: stat /var/lib/docker/tmp/docker-builder830904653/build.sh: no such file or directory
zrashwani commented 5 years ago

I am facing the same issue, docker version: Docker version 18.06.1-ce, build e68fc7a

geetee24 commented 5 years ago

i have same issue. docker v: docker:18.01.0-ce

jrichardsz commented 5 years ago

@zrashwani @geetee24 are you trying to copy file from external external location of your Dockerfile?

zrashwani commented 5 years ago

@jrichardsz this error happens when trying to using ADD or COPY with files outside the Dockerfile directory either using relative or absolute paths, it works will when the file is on same directory as Dockerfile

jrichardsz commented 5 years ago

Hi @sabrehagen

I reviewed your Dockerfile:

FROM gliderlabs/logspout
RUN apk add curl

And I confirmed your error (missing build.sh):

COPY failed: stat /var/lib/docker/tmp/docker-builder830904653/build.sh: no such file or directory

I can say that your error is not related to this issue or missing feature in docker because when I changed your Dockerfile to :

FROM alpine:3.8
RUN apk add curl

Works well.

Looking at logspout origin Dockerfile : https://github.com/gliderlabs/logspout/blob/master/Dockerfile

FROM alpine:3.8
ENTRYPOINT ["/bin/logspout"]
VOLUME /mnt/routes
EXPOSE 80

COPY . /src
RUN cd /src && ./build.sh "$(cat VERSION)"

ONBUILD COPY ./build.sh /src/build.sh
ONBUILD COPY ./modules.go /src/modules.go
ONBUILD RUN cd /src && chmod +x ./build.sh && sleep 1 && sync && ./build.sh "$(cat VERSION)-custom"

We can see that needs a build.sh file and also other files like modules.go.

My advice is review why build.sh and other files are required and include it to your workspace like:

https://github.com/gliderlabs/logspout

Regards

jrichardsz commented 5 years ago

@jrichardsz this error happens when trying to using ADD or COPY with files outside the Dockerfile directory either using relative or absolute paths, it works will when the file is on same directory as Dockerfile

@zrashwani ok.

I thought that your error was related to wrong path like others in this thread.

jrichardsz commented 5 years ago

@sabrehagen Will you close the issue related to copy files with absolute path during image build?

Imagine a 10 docker images. Each one of them needs a file called library.jar at build stage. At this moment We need to copy this library.jar in the same folder of the Dockerfile to perform the build. Repeting this for each Dockerfile.

If there were a way to copy files with absolute path during build stage, We just need only one library.jar located in any folder and each Dockerfile could refer it.

So I think the real issue or missing feature (copy files with absolute path during image build) should stay open.

El sΓ‘b., 10 nov. 2018 a las 16:44, Jackson Delahunt (< notifications@github.com>) escribiΓ³:

As per the resolution by @jrichardsz https://github.com/jrichardsz above I am closing this issue.

β€” You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/docker/for-mac/issues/1922#issuecomment-437623615, or mute the thread https://github.com/notifications/unsubscribe-auth/ADKz1OX9tCUUWz7-c2YduOKqgQhOaex9ks5ut0i5gaJpZM4OpL3k .

michaelhogg commented 5 years ago

@louisbuchbinder @jamesoflol @sleekweasel: Thank you – an inline comment in the COPY instruction was the source of the problem for me.

The relevant Dockerfile documentation is here:

Docker treats lines that begin with # as a comment. A # marker anywhere else in a line is treated as an argument.


Off-topic: it's possible to add an inline # comment in a RUN instruction, because the command is executed in a shell (and so the # comment is interpreted by the shell, not by Docker), eg:

RUN echo 'This is OK'  # This inline comment works