Dockerfile for faster installation

[dmitry]

Any specific reason why there are no Dockerfile, that allows to setup and run overpass API with just few commands?

It took me some time to setup Docker image and run container: https://github.com/dmitry/docker-overpass-api

Even after that I don't feel it's the final solution, that allows to query everything. Would be great if the core source of overpass API will support Dockerfile that allows to setup docker container in a few minutes, because if something went wrong it could take half a day or even days to rebuild everything from scratch.

[mmd-osm]

Why don't you sync up with https://github.com/mediasuitenz/docker-overpass-api where your repo seems to originate from and create a common Docker image?

Also, I would remove the out-of-repo "areas.osm3s" file and at least enable meta data as well. In addition -O3 should not be used at this time.

[dmitry]

@mmd-osm took most of the data from https://github.com/vespakoen/ontheweb/tree/master/overpass-api, but still had an issue with areas. Why not to push Dockerfile directly into the https://github.com/drolbr/Overpass-API ?

BTW tried https://github.com/mediasuitenz/docker-overpass-api before, but had even more issues. So I think most of them are outdated, and if Dockerfile will be inside the core repository, it will be much more easier to support and update.

[mmd-osm]

@vespakoen, @mediasuitenz: are you maybe interested in joining in to create a best practise Dockerfile?

[dmitry]

areas.osm3s - that's a good suggestion, I don't like this idea too. It was fast solution, as in the archive there are no rules path with areas data.

Is there are any issues with overpass execution while optimization during compilation was used?

[mmd-osm]

areas.osm3s should be in the rules directory for 0.7.53 official release.

Yes. we had segmentation faults with -O3 for some versions of gcc during test runs. I think -O2 is still safe to use (didn't have issues with it), Roland removed the optimization altogether in the docs.

[dmitry]

@mmd-osm regarding the areas-osm3s haven't noticed it now in the 0.7.53, rules were missing in the previous release (http://dev.overpass-api.de/releases/osm-3s_v0.7.52.tar.gz). Thanks for the -O2 suggestion!

[mmd-osm]

It would be cool to have some script where you can decide the compression (zlib, lz4, no compression) / map compression / meta settings / area to be created, etc. There are plenty of different options, which would be worthwhile configuring.

Also the question, how updates should be handled (automatic for planet, no updates, ...).

[dmitry]

@mmd-osm I just started to work with overpass API, haven't dig too deeply into the settings. But all the current docker images I found on the github somehow broken.

Regarding automatic updates, looks like it's very CPU and memory intensitive task. Should be easily switchable, may be document something about switching it on / off using docker exec in the current container instance, so it could be turned on while overpass is working, without rebuilding everything from scratch.

[dmitry]

Regarding O3 - there are many main resources that suggests to use O3:

• http://overpass-api.de/full_installation.html
• http://wiki.openstreetmap.org/wiki/Overpass_API/Installation
• http://overpass-api.de/no_frills.html

[mmd-osm]

Right, that's due to the main instance still running on 0.7.52. This has been changed in this commit, which is part of 0.7.53. Removing optimization altogether is a bit pessimistic, imho. There's also another issue for it: #316

It's probably worthwhile using xenial these days, see. https://github.com/mmd-osm/docker-overpass-api/blob/master/Dockerfile - and compile in lz4 support. I also added dedicated users, etc. - running everything as root is not what I consider best practice.

[Frankkkkk]

Mentionned my dockerfile on the mailing list. It is here. It does nearly everything except XAPI and incremental areas. What do you think ?

[mmd-osm]

@Frankkkkk : some of my previous comments also apply to your Docker file:

• dedicated user instead of root (security)
• use of -O3 compilation option is currently discouraged, use -O2 for the time being
• use only released version via dev.overpass-api.de/releases (master branch should not be used, minor_issues is the one used in production).
• Don't download whole planet and store in docker image. Also, better use PBF + osmconvert and store outside of docker for smaller download
• Consider option for cloning, rather than setting up db from scratch (only takes 3-4 hours on fast network)

[Frankkkkk]

Hi,

• Root doesn't matter: it is inside a docker container so even if there is a security breach, the master host is not vulnerable
• will fix -O3
• will see for last version
• the whole planet is not stored in the docker image. It is downloaded in the host and deleted after the import
• for the clone, I should check that someday. I wanted first the option to be independent (i.e. start from scratch). It implies to use the same version as the cloning server. I will see about that.

Cheers

[mmd-osm]

Regarding security:

Docker containers are, by default, quite secure; especially if you take care of running your processes inside the containers as non-privileged users (i.e., non-root).

Source: https://docs.docker.com/engine/security/security/

We advocate to use non-root accounts everywhere in the installation docs. I'd expect exactly the same least privilege principle for a Docker image as well.

[Frankkkkk]

But also: "This means that even if an intruder manages to escalate to root within a container, it will be much harder to do serious damage, or to escalate to the host.".

Anyways, I don't want to spend more energy on this. I agree that its prettier to have its own user. If I've got the time, I'll do it.

[Frankkkkk]

Okay, I've updated some of your points (user, O3, last release). It is here.

What do you think ?

[Frankkkkk]

Some news ? ;-)