search

kartaview / upload-scripts

Uploader tools for KartaView
MIT License
59 stars 30 forks source link

Docker Support #71

Closed alrs closed 5 years ago

alrs commented 5 years ago

I took a look at removing the PEP 576 type annotations from the codebase so that it could run on Debian Stable or on a Raspberry Pi, and saw that that it is used extensively. Type annotations require Python 3.6, both Debian and Raspbian in their stable releases only support Python 3.5.

This PR adds a docker Makefile recipe to build a Docker container for upload-scripts that runs Python 3.7.2, as provided by not-yet-stable Debian Buster.

bogdans-telenav commented 5 years ago

@alrs, thanks you saved me a couple of hours of reaserch. I was planning to add Docker support again to the scripts but first I would have had to learn how to do that.

ToeBee commented 5 years ago

So the make docker command works for me but when I try to run the next command it blows up. I am not (yet) very familiar with Docker so I'm having trouble figuring out the problem.


$ docker run -Pit osc-up osc_tools.py
docker: Error response from daemon: oci runtime error: container_linux.go:247: starting container process caused "exec: \"osc_tools.py\": executable file not found in $PATH".```

Since it is complaining about $PATH, I tried adding the cloned repo location to my PATH environment variable before running this command but that didn't change anything. I am running docker 17.03.2. I'm wondering if there is some version difference in how to call it or something?
ToeBee commented 5 years ago

Well with the help of @iandees and @ianmcorvidae on the OSM US Slack channel I got it working. That first command does not seem to be necessary. The existing command also seems to require the images you want to upload to be placed in the images subdirectory of this repo which is not ideal since I have a couple terabytes of photos on a separate partition. I ended up using this command:

docker run -Pit --mount type=bind,source="$(pwd)",target=/opt/osc -v /path/to/my/images/:opt/osc/images/ osc-up /opt/osc/osc_tools.py upload -p /opt/osc/images/

It seems like it would be better to bundle the scripts into the docker image instead of mounting them at runtime. Instead the location of the photos to upload could be mounted at runtime. I guess that would mean building a new docker image any time the scripts are updated. That doesn't seem like the end of the world though. I will try to play with this later. Might be a good excuse to get my feet wet with docker.

ianmcorvidae commented 5 years ago

An option that might be worth considering is Docker Hub's automated builds. Since those can trigger on every push, it'd make keeping a public image up to date very easy.

alrs commented 5 years ago

@toebee For development use, you really don't want to bake the script into the container. @ianmcorvidae Docker Hub is, unfortunately, a somewhat terrible way to distribute software, unless you're rebuilding the images daily.

We're stuck using Docker just to run these scripts, because of the decision to develop against the latest version of Python 3 that isn't yet widely available. Hopefully no one will change the Dockerfile to use something other than Debian, and as Buster becomes stable the default Python in the container will eventually be a lot more conservative than it is right now. Once the world catches up and upload-scripts is using a more common version of Python, it will be no problem to use pip. Once that happens the Docker complexity won't be necessary anymore.

I'm not interested in running Slack, but I will start hanging out in the #osm-dev channel on irc.oftc.net. Even better: is there an email list for Openstreetcam development?