search

docker / for-mac

Bug reports for Docker Desktop for Mac
https://www.docker.com/products/docker#/mac
2.43k stars 118 forks source link

/var/lib/docker/tmp/docker-builderXXXXXXX/... no such file or directory #1922

Closed bjcatar closed 5 years ago

bjcatar commented 7 years ago

Expected behavior

COPY command to copy a file from local to image

Actual behavior

doing a simple COPY command in Dockerfile is throwing this error when the file is in a folder (not same level as Dockerfile file)

Information

Docker for Mac: version: 17.06.0-ce-mac19 (4cdec4294a50b2233146b09469b49937dabdebdd) macOS: version 10.12.6 (build: 16G29) logs: /tmp/04FF32A9-CA64-40EB-B0DC-AD3E13A6C7D5/20170731-232145.tar.gz [OK] db.git [OK] vmnetd [OK] dns [OK] driver.amd64-linux [OK] virtualization VT-X [OK] app [OK] moby [OK] system [OK] moby-syslog [OK] db [OK] env [OK] virtualization kern.hv_support [OK] slirp [OK] osxfs [OK] moby-console [OK] logs [OK] docker-cli [OK] menubar [OK] disk

FROM python:latest RUN apt-get update RUN apt-get install nano RUN mkdir /code WORKDIR /code COPY /helpdesk/requirements.txt /code/

Steps to reproduce the behavior

  1. docker build -t test .
irshsheik commented 5 years ago

I tried using ${pwd} so it could get the relative path w.r.t var/lib/docker folder. It works.

openocuser commented 5 years ago

We have a Docker file for Clam-av (anti virus scan for uploaded files). It is build on top of spring boot API. Docker File has the following commands: FROM mkodockx/docker-clamav:latest

MAINTAINER lokori antti.virtanen@iki.fi

RUN echo "deb http://ftp.de.debian.org/debian jessie-backports main" >> /etc/apt/sources.list

RUN apt-get update && apt install -t jessie-backports -y openjdk-8-jdk ca-certificates-java

Set environment variables. ENV HOME /root

Get the JAR file CMD mkdir /var/clamav-rest COPY target/clamav-rest-1.0.2.jar /var/clamav-rest/

Define working directory. WORKDIR /var/clamav-rest/

Open up the server EXPOSE 8082 ADD bootstrap.sh /bootstrap2.sh ENTRYPOINT ["/bootstrap2.sh"] From the above lines mkdir is not creating directory and it is not showing any error messages. Please share your experiences if you have faced similar issues. I am using the following Linux distribution. SMP Debian 3.16.56-1+deb8u1 (2018-05-08) x86_64 GNU/Linux

Thanks in advance

jrichardsz commented 5 years ago

Hi @openocuser Your error is not related to this issue : /var/lib/docker/tmp/docker-builderXXXXXXX/... no such file or directory

You could use https://stackoverflow.com/ to open a question.

excalq commented 5 years ago

This is an interesting one. I have this error happening, even though I've changed the Dockerfile to no longer reference this path, the old path is still in this error.

I have a project, which for a throwaway demo on a cheap ECS Fargate box, I'm copying files normally in a volume, into a big combined image to easily load.

The COPY part of my Dockerfile (see in full below.) is throwing the error seen in this issue. I'm making the context the whole project, and building with a Dockerfile contained within, 2 dirs deep.

The error:

COPY failed: stat /var/lib/docker/tmp/docker-builder609371816/etc/nginx/sites-available/default: no such file or directory

But even after changing my COPY line to a different path, to avoid editing previously COPY'd files, the build fails, and still references the /etc/nginx/sites-available/default path which is no longer in my Dockerfile. --no-cache does not help either.

The Dockerfile at issue:

FROM 12345678.dkr.ecr.us-west-2.amazonaws.com/namespace/myimage:tagname

COPY volumes/media-uploads/ /data/media-uploads
COPY volumes/media_catalog/ /app/media/catalog
COPY the-project-git-files/ /app

# Copy an HTTPS+ALB enabled version of the nginx config
# COPY docker-files/ecs_demo/nginx.https.config /app/app/etc/local.xml
## CHANGED TO:
COPY docker-files/ecs_demo/nginx.https.config /nginx

docker --version
Docker version 18.09.0, build 4d60db4
excalq commented 5 years ago

WORKDIR / also does not change the failure.

matthewaudio commented 5 years ago

Have a fix.

Here is what my original code looks like: COPY /TestCode TestCode/

I decided to try this and it magically worked:

WORKDIR /TestCode

COPY . TestCode/

Tried most of the other solutions here so this might work for other people as well.

I scrolled ten thousand comments just for this to work.

lucasbasquerotto commented 5 years ago

@excalq Make sure that your image 12345678.dkr.ecr.us-west-2.amazonaws.com/namespace/myimage:tagname (or some base image of this image) doesn't have an ONBUILD in the dockerfile that requires the file /etc/nginx/sites-available/default.

See: https://github.com/docker-library/python/issues/268

excalq commented 5 years ago

@lucasbasquerotto My app's image is based on https://github.com/timHerman/docker-nginx-magento/blob/master/Dockerfile, which does not use the ONBUILD.

The original "/app/app/etc/local.xml" COPY destination is in a previous layer's VOLUME path, so that may be a useful clue. Not sure why changing this destination path (/nginx) didn't stop this same error from happening. Anyway, I'm going to rebuild all this from the base layers now.

estatematt commented 5 years ago

This might seem silly but I fixed this issue yesterday by realizing that docker does not support same-line comments so

ADD /x/y/z # this line returns an error.

ghuser commented 5 years ago

@estatematt I think the real issue is that there is a misleading error message thrown whenever some path does not exist or is not accessible, and it happens on mac/windows/linux:

seansong327 commented 5 years ago

In my case, I got this error because my path is "bin/debug" which should be "bin/Debug".

hanlina0737 commented 5 years ago

I have the same problems.now confused

jrichardsz commented 5 years ago

@hanlina0737 could you share us your Dockerfile?

kamenov commented 5 years ago

From the official Dockerfile reference documentation here

COPY obeys the following rules: The path must be inside the context of the build; you cannot COPY ../something /something, because the first step of a docker build is to send the context directory (and subdirectories) to the docker daemon.

So don't try to copy files from outside of the build context (which is usually . )

yoaquim commented 5 years ago

I'm trying to copy a file from my host's $HOME.

The file text.txt is under a hidden dir in $HOME called texts:

I don't think I'm outside the build context:

# in $HOME dir
sudo docker build ./app -t app

Where the app dir contains my project source-code and Dockerfile.

In my Dockerfile:

....

COPY ./.texts/text.txt /www/my-text.txt

....

And docker's erring out with:

COPY failed: stat /var/lib/docker/tmp/docker-builder741058145/.texts/text.txt: no such file or directory

Still not sure what's wrong; files are there and everything else checks out AFAICT.

Am I still not in the right docker context?

thaJeztah commented 5 years ago

You specify ./app (thus $HOME/app/) as your build context, so anything outside of $HOME/app/ won't be in the build context. If that hidden directory is $HOME/.texts/, it's not in the build context.

Lewiscowles1986 commented 5 years ago

I had this problem with a local repository which was expected to build a directory. Double-check some brainbox at your business has not decided to COPY a materialized folder

SandeshMolane commented 5 years ago

@matthewaudio thanks a lot, had to go through hundreds of comments. Finally this worked for me. Earlier FROM python:3.6.1-alpine RUN pip install flask CMD ["python","app.py"] COPY app.py /app.py

Later FROM python:3.7.2-alpine3.8 RUN pip install flask CMD ["python","app.py"] WORKDIR app.py COPY . /app.py

thaJeztah commented 5 years ago

I don't think you want this line;

WORKDIR app.py

That line will create a directory named app.py (if it doesn't exist), and change the working-directory to that

ibrahimkais commented 5 years ago

Having the docker-compose and the Dockerfile on the same folder solved it for me

This solved my problem. I just copies the dockerfile and changed the paths inside docker-compose to point the dockerfile and changed the paths inside the docker file to point for the project folder

nobso commented 5 years ago

I faced the same issue and solved it by

docker-compose.yml

app:
   build:
     context: .
     # give the complete PATH to your Dockerfile instead of a folder
     dockerfile: ./app/Dockerfile

Dockerfile

COPY /app  /app
imochoa commented 5 years ago

Fixed by removing the comments at the end of the lines, thanks @estatematt!

This might seem silly but I fixed this issue yesterday by realizing that docker does not support same-line comments so

ADD /x/y/z # this line returns an error.

thomas-anderson-bsl commented 5 years ago

I was seeing this same issue when trying to use:

COPY scripts /scripts

Turns out I had the scripts folder in my .dockerignore. Removed it from there (so that the scripts folder is in the context of the docker build command) and everything worked as expected.

ibrahimkais commented 5 years ago 
# stage 1
FROM node:latest as node
WORKDIR /app
COPY <PROJECT_DIRECTORY>/. .
RUN yarn install --network-timeout 300000 --no-lockfile
RUN npm install --save rebuild-node-sass node-sass
RUN yarn run build --prod

# stage 2
FROM nginx:alpine
COPY --from=node /app/dist/<APP_NAME> /usr/share/nginx/html

That worked for me. Dockerfile is in the same path as project I mixed between yarn and npm for psychological problems 🗡 . Don't ask why try to remove the mixing if this suites your project

siaminator commented 5 years ago

using sudo solved my problem !

logston commented 5 years ago

Found the cause of this symptom in my setup:

I was trying to use fluentd's onbuild docker image. Turns out that ONBUILD images can pass build steps down to the child Dockerfiles:

https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#onbuild

Think of the ONBUILD command as an instruction the parent Dockerfile gives to the child Dockerfile. ... Be careful when putting ADD or COPY in ONBUILD. The “onbuild” image fails catastrophically if the new build’s context is missing the resource being added. Adding a separate tag, as recommended above, helps mitigate this by allowing the Dockerfile author to make a choice.

This is exactly what the fluentd Dockerfile was doing:

https://github.com/fluent/fluentd-docker-image/blob/master/v1.3/alpine-onbuild/Dockerfile#L38

Even though I had my own child Dockerfile that used a COPY with the correct location of my fluent.conf file, the parent Dockerfile was looking somewhere else. Specifically in the root of the docker build context. I updated my context to have the fluent.conf file in the root. Problem solved.

scottpage commented 5 years ago

I used vscode context menu option "Build Image", and it produced this: docker build --rm -f "Api/Dockerfile" -t api:latest Api

Then I found this solution, https://stackoverflow.com/a/33761889/5403486, and ended up with this: docker build --rm -f "Api/Dockerfile" -t api:latest .

In my solution folder, I have 7 dependent projects, so Docker runs from the solution folder. Replace the path, Api in my case, with . (dot), and this may help resolve a similar issue for you.

Don't ask me, I just code.

naydoo commented 5 years ago

it's definitely related to using a fully qualified path. if I put:

ADD /Volumes/dev/sterling/java-user-login-service/user-login-server/build/libs/user-login-server-1.0.5-SNAPSHOT.jar /user-login-server-1.0.5-SNAPSHOT.jar
ENTRYPOINT ["sh", "-c", "java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -jar /user-login-server-1.0.5-SNAPSHOT.jar"]

then I get Step 1/2 : ADD /Volumes/dev/sterling/java-user-login-service/user-login-server/build/libs/user-login-server-1.0.5-SNAPSHOT.jar /user-login-server-1.0.5-SNAPSHOT.jar ADD failed: stat /var/lib/docker/tmp/docker-builder663214013/Volumes/dev/sterling/java-user-login-service/user-login-server/build/libs/user-login-server-1.0.5-SNAPSHOT.jar: no such file or directory

But if I change it to:

ADD build/libs/user-login-server-1.0.5-SNAPSHOT.jar /user-login-server-1.0.5-SNAPSHOT.jar
ENTRYPOINT ["sh", "-c", "java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -jar /user-login-server-1.0.5-SNAPSHOT.jar"]

it all works

docker build -t mothra/user-service:1234 -f ./user-login-server/Dockerfile ./user-login-server
Sending build context to Docker daemon    363MB
Step 1/2 : ADD build/libs/user-login-server-1.0.5-SNAPSHOT.jar /user-login-server-1.0.5-SNAPSHOT.jar

works for me

alexykot commented 5 years ago

Seems like there is a number of different problems that can lead to the error message in question. Possible culprits:

I suggest to check all above to find your problem.

rajatchauhan126 commented 5 years ago

This issue occurs because the certificate.crt and private.key is missing from the folder. Manually copy this file from your coleague if possible or find a way to generate this file in the folder "/home/yourusername/folder name/nginx"

Arunjos commented 5 years ago

I had the COPY failed: stat /var/lib/docker/tmp/docker-builder929708051/XXXX: no such file or directory and managed to resolve this by specifying the docker file.

Running docker build . -f docker/development/Dockerfile worked.

But running Running docker build docker/development/Dockerfile` caused this issue.

-f or --file to specify the name and location of the Dockerfile.

GabrielGhe commented 5 years ago

My problem was that I was calling build like this

docker build -t $TAG - < $CURRENT_DIR/Dockerfile

I just needed to change to

docker build -t $TAG .
pablolopesk8 commented 5 years ago

I faced the same issue and solved it by

docker-compose.yml

app:
   build:
     context: .
     # give the complete PATH to your Dockerfile instead of a folder
     dockerfile: ./app/Dockerfile

Dockerfile

COPY /app  /app

I had the same error, using Mac and the Engine: 18.09.2. This solutions works to me =) The problem was related with path structure. If your docker-compose in a different path that your Dockerfile, you need to specify a context!

Thanks for your answer ;)

stefjoosten commented 5 years ago

This might seem silly but I fixed this issue yesterday by realizing that docker does not support same-line comments so

ADD /x/y/z # this line returns an error.

@estatematt Not so silly... It worked for me too. :-) Thanks 👍

thaJeztah commented 5 years ago

@stefjoosten @estatematt perhaps you could open a ticket in the https://github.com/moby/buildkit repository to discuss adding a warning for those (i.e., if <whitespace># is found, produce a warning), similar to https://github.com/moby/moby/pull/35004.

Supporting inline comments has been discussed at some points, but was decided to not implement, as it cannot be supported consistently (e.g. supporting them in RUN lines would be impossible, given that different shells can be used (PowerShell/Bash/other)), however, adding a warning for ADD/COPY/ENV/VOLUME (possibly others) might be possible (we should investigate if it can be added easily though, without introducing a big chunk of complexity/code that has to be maintained)

Having said the above; it looks like with BuildKit enabled (the next generation builder), the error message is slightly clearer, and might on itself be sufficient to troubleshoot the problem if someone runs into this;

mkdir inline-comments && cd inline-comments
touch file

Without buildkit enabled:

DOCKER_BUILDKIT=0 docker build . -f-<<EOF
FROM busybox
COPY file . # copy the file
EOF
Sending build context to Docker daemon  2.095kB
Step 1/2 : FROM busybox
 ---> af2f74c517aa
Step 2/2 : COPY file . # copy the file
COPY failed: stat /var/lib/docker/tmp/docker-builder340679920/#: no such file or directory

With buildkit enabled:

DOCKER_BUILDKIT=` docker build . -f-<<EOF
FROM busybox
COPY file . # copy the file
EOF
[+] Building 0.7s (6/6) FINISHED                                                                                                                                                                                                                   
 => [internal] load build definition from Dockerfile                                                                                                                                                                                          0.1s
 => => transferring dockerfile: 83B                                                                                                                                                                                                           0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                             0.1s
 => => transferring context: 2B                                                                                                                                                                                                               0.0s
 => [internal] load metadata for docker.io/library/busybox:latest                                                                                                                                                                             0.0s
 => [1/2] FROM docker.io/library/busybox                                                                                                                                                                                                      0.0s
 => => resolve docker.io/library/busybox:latest                                                                                                                                                                                               0.0s
 => [internal] load build context                                                                                                                                                                                                             0.5s
 => => transferring context: 25B                                                                                                                                                                                                              0.5s
 => ERROR [2/2] COPY file . # copy the file                                                                                                                                                                                                   0.0s
------
 > [2/2] COPY file . # copy the file:
------
"/#" not found: not found
johnmutuma5 commented 5 years ago

You may want to keep your context in the same root directory of the root of your project because that's where docker will be looking for the items to copy. Then in order to have your Dockerfile in a separate directory from the context, point docker to the location with -f.

docker build -t test -f ./path/to/docker-files/directory/NameOfYourDockerfile .

Note the . at the end, that keeps the context in your current working directory (pwd) working directory so docker can begin there to copy your items, but -f points docker to the location where it'll find the Dockerfile for instructions.

chaminnk commented 5 years ago

COPY worked after removing or commenting the content in the default .dockerignore file. In my .dockerignore file, it has filtered everything except the target directory. Removing this restriction might help.

vertsynskyi commented 5 years ago

Had the same problem with Gradle in Intellij: Command: gradle build gradle dockerPush Error: stat /var/lib/docker/tmp/docker-builder663214013/.../.../../.../: no such file or directory Solved by running: sudo ./gradlew clean build dockerPush

felixvd commented 5 years ago

You may want to keep your context in the same root directory of the root of your project because that's where docker will be looking for the items to copy. Then in order to have your Dockerfile in a separate directory from the context, point docker to the location with -f.

docker build -t test -f ./path/to/docker-files/directory/NameOfYourDockerfile .

Note the . at the end, that keeps the context in your current working directory (pwd) working directory so docker can begin there to copy your items, but -f points docker to the location where it'll find the Dockerfile for instructions.

This is what solved this problem on Docker for Mac for me. Thanks!

klemmster commented 5 years ago

I just ran into a similar issue: the directory to copy included a link into a file that was now excluded by .dockerignore.

gireeshspr commented 5 years ago

RESOLUTION FOR ME -- the .dockerignore file was the culprit. It included a path that included my COPY src folder. Revising the .dockerignore resolved my issue.

bgmnbear commented 5 years ago

Hi.

There are so many review in this page and the problem about path is always puzzling. I came up with the error which most of us met, but I solve it by check my docker command, make sure that the Dockerfile path, project path, docker command path and COPY options path, is that all correct. Finally I found that my COPY option are wrong.

Best, Whister.

atariman5000 commented 5 years ago

I was just dealing with the same issue, and the solution was to check my .dockerignore file. I was excluding the file (e.g. .bak) I was trying to copy in the .dockerignore file. After removing the line with .bak from the .dockerignore file, my database backup files then copied as they should have. So this issue was my fault and not a result of some Docker bug.

jayunit100 commented 5 years ago

To summarize for folks... This isn't an issue w/ COPY or ADD.

The above scenario can happen if:

The workaround is of course, to make sure you have a path that is visible in the build context. Probably the simplest test is to do RUN ls -altrh path/to/file as a simple smoke test in your image build

gojedi commented 5 years ago

i had same issue and was resolved by removing a forgotten extra dot (.) in front of one path in .dockerignore file

ChristosMonogios commented 5 years ago

In my case, I was trying to integrate Docker for Visual Studio into an existing Solution and all the needed files where in the correct position including Dockerfile.

What it helped was to add the <AspNetCoreHostingModel>InProcess</AspNetCoreHostingModel>in the .csproj file.

chuksudeh commented 4 years ago

For me, the issue was caused by .dockerignore file. Apparently I was trying to copy from a directory that was already included in .dockerignore. Removing it from .dockerignore fixed the issue.

pjw1630 commented 4 years ago

If you are working with Docker for Windows, also be aware of case sensitivity with the path of your repo.

zeiv commented 4 years ago

In case anyone else finds this on Google with the same issue I had, in my case, I had an emdash sneak into my --from flag: —-from=builder: no such file or directory or without monotype —-from=builder: no such file or directory (In vscode's font, it was nearly indistinguishable) Tip: Emdashes to not count as flag delimiters

ericr6 commented 4 years ago

+1, no dockerignore and still fails to COPY, Workaround (for me) I created a new directory, put all my stuffs in and docker build is ok.