Simplify docker image - Githubissues

aruneko commented 5 years ago

I modified the docker images to simplify it.

For example, the image size of v3.0.0-dev-postgresql_11 decrease from 1.4GB to 920MB.

The changes is below.

Replace base image pgrouting/postgres with mdillon/postgis
- mdillon/postgis is based on official Postgres image, well-maintained, and built automatically.
- So we can use latest version of postgresql and postgis easily.
- And we can refer to an official postgresql docker image document.
- This is why, I removed postgres directory.
Remove osm2pgrouting from this docker image
- I think the responsibility of this docker image is only providing pgrouting.
- If you use osm2pgrouting to import OSM data, it is better to create a other container which includes osm2pgrouting.

In addition, I think it is required a CI for building 3.0.0-dev automatically (#4). But to detect push for the development branch, we have to place CI settings in pgRouting repository. So if we resolve this issue, we have to consider it.

cayetanobv commented 5 years ago

Thanks @aruneko . You have done in this PR a lot of work to improve the project! I've been trying to simplify the way we prepare the base image because it generates a lot of work and it is difficult to maintain. This could be a good opportunity to improve that.

Several things I would like to discuss before proceed:

mdillon/postgis is a well mantained postgis base image but it has things I don't like:
- init file creates a specific use-case template (this init file shouldn't be in the image base): https://github.com/appropriate/docker-postgis/blob/master/11-2.5/initdb-postgis.sh
- same with update file: https://github.com/appropriate/docker-postgis/blob/master/11-2.5/update-postgis.sh
- The only interesting part of Dockerfile is postgis installation so we could change base of our image by postgres official and add apt install of postgis (version as ENV in Dockerfile). Postgis installation from apt is fast.
osm2pgrouting: It is true that not all people are using it but I think this is a useful tool (and little in terms of disk). When you install PostGIS several tools are installed by default (shp2pgsql and pgsql2shp) and not all people use it.
Regarding automatic build of development branch: I agree with you. Maybe this is the point to discuss again issue #4

@cvvergara and @dkastl: What do you think about it?

aruneko commented 5 years ago

Thanks for your opinion!

I understand your concern, and I propose this Dockerfile.

for pgrouting 2.6.2

FROM postgres:11

ENV POSTGIS_MAJOR 2.5
ENV PGROUTING_VERSION 2.6.2

RUN apt update \
 && apt install -y \
        build-essential \
        cmake \
        perl \
        wget \
        libboost-graph-dev \
        libcgal-dev \
        libpq-dev \
        # install postgis from apt
        postgresql-${PG_MAJOR}-postgis-${POSTGIS_MAJOR} \
        postgresql-server-dev-${PG_MAJOR} \
 && cd /usr/local/src \
 && wget https://github.com/pgRouting/pgrouting/archive/v${PGROUTING_VERSION}.tar.gz \
 && tar xvf v${PGROUTING_VERSION}.tar.gz \
 && cd pgrouting-${PGROUTING_VERSION} \
 && mkdir build \
 && cd build \
 && cmake .. \
 && make \
 && make install \
 && cd ../../ \
 && rm -rf ./* \
 && apt purge -y \
        build-essential \
        cmake \
        perl \
        wget \
        libcgal-dev \
        libpq-dev \
        postgresql-server-dev-${PG_MAJOR} \
 && apt install -y \
        postgresql-${PG_MAJOR}-postgis-${POSTGIS_MAJOR}-scripts \
        libboost-atomic1.62.0 \
        libboost-chrono1.62.0 \
        libboost-graph1.62.0 \
        libboost-date-time1.62.0 \
        libboost-program-options1.62.0 \
        libboost-system1.62.0 \
        libboost-thread1.62.0 \
        libcgal12 \
 && apt autoremove -y \
 && rm -rf /var/lib/apt/lists/*

for pgrouting 3.0.0

FROM postgres:11

ENV POSTGIS_MAJOR 2.5
ENV PGROUTING_VERSION develop

RUN apt update \
 && apt install -y \
        build-essential \
        cmake \
        perl \
        wget \
        libboost-graph-dev \
        libpq-dev \
        # install postgis from apt
        postgresql-${PG_MAJOR}-postgis-${POSTGIS_MAJOR} \
        postgresql-server-dev-${PG_MAJOR} \
 && cd /usr/local/src \
 && wget https://github.com/pgRouting/pgrouting/archive/${PGROUTING_VERSION}.tar.gz \
 && tar xvf ${PGROUTING_VERSION}.tar.gz \
 && cd pgrouting-${PGROUTING_VERSION} \
 && mkdir build \
 && cd build \
 && cmake .. \
 && make \
 && make install \
 && cd ../../ \
 && rm -rf ./* \
 && apt purge -y \
        build-essential \
        cmake \
        perl \
        wget \
        libpq-dev \
        postgresql-server-dev-${PG_MAJOR} \
 && apt install -y \
        postgresql-${PG_MAJOR}-postgis-${POSTGIS_MAJOR}-scripts \
        libboost-graph1.62.0 \
        libboost-program-options1.62.0 \
 && apt autoremove -y \
 && rm -rf /var/lib/apt/lists/*

regarding osm2pgrouting

The base image postgres:11 is based on Debian stretch. The repository of Debian stretch includes an osm2pgrouting package but the version is 2.2.0, so we cannot install latest version of osm2pgrouting and have to build it for installing. I concern about to be more complex Dockerfile.

I understand the needs of osm2pgrouting, but for production deployment using public cloud (e.g. AWS, GCP), I hope to save as more disk space as possible. So, I think it is required both of docker image which includes it or not. Because, once we create a docker image, we cannot remove any files from it.

But I concern about increase of tags. If possible, I hope to remove support of 2.6.1.

dkastl commented 5 years ago

Would it be too much work or a bad idea to make a different Docker image for osm2pgrouting?

I think, that it's good to have a way to install the latest one as a Docker container, because distributions are sometimes slow to update. And I understand, that some people won't need it.

Not sure, what is the most reasonable solution. But I'm very glad about the discussion and the pull request of @aruneko . It actually shows users of this Docker image, that it is well thought and maintained.

cayetanobv commented 5 years ago

@aruneko I like new Dockerfiles you are proposing. Could you push these changes to PR?

Osm2pgrouting issue

I understand your concern about image size and I think in production is important. @dkastl It looks good to me to maintain 2 Dockerfiles (one of them including osm2pgrouting) by pRouting version (we can automate it). The question here is: How we could tag osm2pgrouting image in a clearly way? What do you think about it?

woodbri commented 5 years ago

On 7/9/2019 12:14 PM, Cayetano Benavent wrote:

@aruneko https://github.com/aruneko I like new Dockerfiles you are proposing. Could you push these changes to PR?
  Osm2pgrouting issue
I understand your concern about image size and I think in production is important. @dkastl https://github.com/dkastl It looks good to me to maintain 2 Dockerfiles by pRouting version (we can automate it). The question here is: How we could tag osm2pgrouting image in a clearly way? What do you think about it?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pgRouting/docker-pgrouting/pull/12?email_source=notifications&email_token=AAGETAUPC7ELROZ4HWHA3SLP6S2OPA5CNFSM4H6Y5QKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZQYQRQ#issuecomment-509708358, or mute the thread https://github.com/notifications/unsubscribe-auth/AAGETAV7ZPMCGNBZLYQMFE3P6S2OPANCNFSM4H6Y5QKA.

How about calling it something like pgRouting-plus? or -fat? to imply that there are additional things in it. Today it would have osm2pgrouting, but later we might want to add other stuff to it.

This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

aruneko commented 5 years ago

I pushed suggested Dockerfiles.

Then, I propose Dockerfile for a docker image includes osm2pgrouting.

FROM pgrouting/prgouring:3.0.0-dev-11

ENV OSM2PGROUTING_VERSION 2.3.6

RUN apt update \
 && apt install -y \
        build-essential \
        cmake \
        wget \
        libboost-program-options-dev \
        libexpat1 \
        libexpat-dev \
        libosmium2-dev \
        libpqxx-dev \
        zlib1g-dev \
 && cd /usr/local/src \
 && wget https://github.com/pgRouting/osm2pgrouting/archive/v${OSM2PGROUTING_VERSION}.tar.gz \
 && tar xvf v${OSM2PGROUTING_VERSION}.tar.gz \
 && cd osm2pgrouting-${OSM2PGROUTING_VERSION} \
 && mkdir build \
 && cd build \
 && cmake .. \
 && make \
 && make install \
 && cd ../tools/osmium/ \
 && mkdir build \
 && cd build \
 && cmake .. \
 && make \
 && make install \
 && cd /usr/local/src \
 && rm -rf ./* \
 && apt purge -y \
        build-essential \
        cmake \
        wget \
        libexpat-dev \
        libosmium2-dev \
        libpqxx-dev \
        zlib1g-dev \
 && apt install -y \
        libpqxx-4.0v5 \
 && apt autoremove -y \
 && rm -rf /var/lib/apt/lists/*

cayetanobv commented 5 years ago

Thanks @aruneko . Dockerfile for osm2pgrouting looks good to me (only a typo in name: pgrouting/prgouring:3.0.0-dev-11 -> pgrouting/pgrouting:3.0.0-dev-11). Could you push these changes.

Naming

I think we only need to find a name for this image to differentiate from "main". Following the proposal by @woodbri , I've added another name more (added to docker image "tag"):

-plus
-fat
-ext (stands for extended version)

What do you think?

dkastl commented 5 years ago

-plus sounds as it would be better. What about -extra or -osm?

cayetanobv commented 5 years ago

-plus sounds as it would be better. What about -extra or -osm?

@dkastl I like -extra. I think the goal is very clear.

woodbri commented 5 years ago

+1 on -extra. better to keep it generic then we can put other stuff into it also and not have to have a separate one for everything we might want to add in the future.

aruneko commented 5 years ago

I pushed Dockerfile for -extra, and fixed URL on README.

cayetanobv commented 4 years ago

Thank you @aruneko I'm going to review all changes to proceed with this PR.

cayetanobv commented 4 years ago

@aruneko I've changed base branch to development. I will merge with master after some testing.

aruneko commented 4 years ago

Thanks a lot!

pgRouting / docker-pgrouting

Simplify docker image #12

for pgrouting 2.6.2

for pgrouting 3.0.0

regarding osm2pgrouting

Osm2pgrouting issue

Naming