Docker fails when trying to run containers

[salatfreak]

Running Docker 17.06.0-ce on 64bit Linux 4.9.38.1-MANJARO

Following the instruction in DOCKER.md leads to an error. It is not mentioned, that a docker machine needs to be created as well. Here is what I do:

# Clone repository
git clone https://github.com/gravitystorm/openstreetmap-carto.git
cd openstreetmap-carto

# Copy osm data
cp ../<extract>.osm.pbf ./data.osm.pbf

# Create and start docker machine
docker-machine create default

# Configure environment
eval $(docker-machine env default)

# Import data
docker-compose up import

Everything downloads and installs until:

...
Creating openstreetmapcarto_db_1 ... 
Creating openstreetmapcarto_db_1 ... done
Creating openstreetmapcarto_import_1 ... 
Creating openstreetmapcarto_import_1 ... done
Attaching to openstreetmapcarto_import_1
import_1   | sh: 0: Can't open scripts/docker-startup.sh
openstreetmapcarto_import_1 exited with code 127

Rerunning docker-compose up import results in

openstreetmapcarto_db_1 is up-to-date
Starting openstreetmapcarto_import_1 ... 
Starting openstreetmapcarto_import_1 ... done
Attaching to openstreetmapcarto_import_1
import_1   | sh: 0: Can't open scripts/docker-startup.sh
openstreetmapcarto_import_1 exited with code 127

immediately.

[matthijsmelissen]

Thanks for reporting. Which extract did you use? Could you try with the Luxembourg extract?

[salatfreak]

I used the current Germany extract from geofabrik.de. I don't think it even starts the import process. It takes about 2 seconds to fail.

[salatfreak]

As expected, the same thing happens with the current Luxembourg extract from geofabrik.de.

[matthijsmelissen]

Is anybody able to reproduce this?

[salatfreak]

I installed the Docker Toolbox on Windows 7 on the same machine. Both the procedure and the resulting error are the same when trying to import the data.

[salatfreak]

The same thing happens if I use a version from 18 days ago. Has it been tested since then? Is there maybe some additional configuration required?

[kocio-pl]

I made a standard installation on fresh Manjaro (in VirtualBox) as documentation says:

# Clone repository
git clone https://github.com/gravitystorm/openstreetmap-carto.git
cd openstreetmap-carto

# Copy osm data
cp ../<extract>.osm.pbf ./data.osm.pbf

# Import data
docker-compose up import

and it works. So I guess this is a problem with these extra steps you took.

[kocio-pl]

It is not mentioned, that a docker machine needs to be created as well.

Why do you think so? Have you tried without using docker-machine at all?

[salatfreak]

I thought so because without it I got an

Error: Couldn't connect to Docker daemon at http+docker://localunixsocket - is it running?

If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.

and the first fix I found by startpage'ing the error message was to create a docker machine. What I should have done apparently is trying to run the command with sudo, which works as expected.

Is this just so obvious, that it wasn't even mentioned in DOCKER.md or should it not generally be required? I am part of the docker group.

[kocio-pl]

Hard to say where to draw the line. Installing Docker and Docker Compose depends on the system used.

On the other hand we could mention to check if Docker is ready before the import, I think simple docker run hello-world would be enough.

[salatfreak]

I don't quite understand your answer. Do you need to run sudo docker-compose up import or not? I think it would be nice if something like "docker-compose requires/might require root priviledges" was mentioned in the DOCKER.md then.

[nebulon42]

Docker Machine is definitely not required to run this. Also root privileges are not required. I don't know what the problem is, but it is quite likely related to your Docker setup.

Can you run other containers? Can you run other docker-compose setups?

[nebulon42]

Maybe this issue/comment helps you: https://github.com/docker/compose/issues/1214#issuecomment-195548786

[salatfreak]

I think I didn't put it very well. Everything works now but I got an error when I ran it without root privileges. For some reason it works without sudo now (after a reboot) but I had to start the docker daemon with sudo systemctl start docker first. Maybe it works now because I added myself to the "docker" group and it required a reboot for some reason?

[salatfreak]

@kocio-pl Do you need to be in the docker group in your manjaro VB? And did you have to start the service?

[kocio-pl]

You should look at some Docker installation guide for Manjaro, for example this one:

https://manjaro-tutorial.blogspot.com/2016/12/how-to-install-docker-on-manjaro-1610.html

Yes, you need to start Docker each time you start your system. If you want it to start automatically, you need to run this command:

sudo systemctl enable docker

And yes, you need to be in group docker to run Docker commands.

That's why I advise to run:

docker run hello-world

to know if Docker is up and you have the right to run it. That's all part of a Docker installation and configuration.

[salatfreak]

@kocio-pl Thank you very much and sorry for wasting your time :( I think this information should be mentioned in the DOCKER.md anyways for everybody else who is not familiar with it.

[OndrejSpanel]

I am a bit confused. I have tried to follow DOCKER.md both on Windows and Ubuntu, in both cases my result is what is described earlier in this thread:

Creating openstreetmapcarto_kosmtik_1
Attaching to openstreetmapcarto_kosmtik_1
kosmtik_1  | sh: 0: Can't open scripts/docker-startup.sh

I understand this is some omission or mis-configuration on my side, but even after reading this thread several time I do not see what am I doing wrong.

I have installed docker, verified I can run docker hello world without admin rights, then did docker-compose up import (worked fine) followed with docker-compose up kosmtik, which built fine, but then the error sh: 0: Can't open scripts/docker-startup.sh appeared.

Can someone clarify, please?

[matthijsmelissen]

What is the output of docker run hello-world?

Do you run Docker inside a virtual machine (like Virtualbox)? If so, is your git repository located in a shared folder?

[OndrejSpanel]

Do you run Docker inside a virtual machine (like Virtualbox) For Ubuntu - yes If so, is your git repository located in a shared folder? For Ubuntu - Yes

On Ubuntu the output of docker run hello-world is as expected:

Hello from Docker!
This message shows that your installation appears to be working correctly.

On Windows there is no VM (my computer is native Windows 10 Pro / x64). I run docker from Docker Machine Quickstart terminal (I was unable to install more recent Docker For Windows), docker run hello-world works as expected.

Output of docker-compose up kosmtik is:

$ docker-compose up kosmtik
Recreating openstreetmapcarto_db_1 ... done
Recreating openstreetmapcarto_kosmtik_1 ... done
Attaching to openstreetmapcarto_kosmtik_1
kosmtik_1  | sh: 0: Can't open scripts/docker-startup.sh
openstreetmapcarto_kosmtik_1 exited with code 127

[matthijsmelissen]

I ran into the same problem. As a workaround, try the git checkout directly in your virtual machine, instead of in the shared folder. Must be something permission related, if you know how to fix that I’d be interested too!

[OndrejSpanel]

As a workaround, try the git checkout directly in your virtual machine, instead of in the shared folder.

I confirm this works for me as a workaround. Given Docker Toolbox uses VirtualBox VM under the hood, the error I get when running Windows Docker Toolbox most likely has the same cause.

Checking the permissions, the difference I see is that with shared folders the user:group is reported as root:vboxsf:

-rwxrwx--- 1 root vboxsf 2059 úno 20 10:04 docker-startup.sh

When I create the folder in a VM, I get this instead:

-rw-rw-r-- 1 ondra ondra 2059 úno 26 08:59 docker-startup.sh

I am not sure why it is not working. My user is member of vboxsf, when I access the file, it works fine but docker somehow has trouble accessing it on my behalf, even when I try as sudo docker-compose up kosmtik

[matthijsmelissen]

My user is member of vboxsf,

@OndrejSpanel Does docker run under its own user? Perhaps this user should also be made a member of vboxsf?