lizmap-docker-compose & postgres

[frazucca]

Hi, I'm testing the Lizmap docker stack, and apart a trouble with lizmap 3.4 (@dmarteau has open a ticket about), all is great at the moment, but I guess I've to add a postgres-postgis image into the docker-container.yml, in order to test this side too, right?

ciao fz

[nboisteault]

Hi, You mean docker-compose.yml? This file could help https://github.com/3liz/lizmap-web-client/blob/master/tests/docker-compose.yml to add postgresql/postgis to your stack.

[nboisteault]

@dmarteau Could you add postgres/postgis to current stack easily? Thanks.

[frazucca]

@nboisteault Yes docker-compose.yml. Great tips! @nboisteault @dmarteau Mercì

[dmarteau]

Could you add postgres/postgis to current stack easily? Thanks.

It must be a optional feature because if you want to use an alternative/private database this would just get in your way.

[gerald2545]

Hi, I have to add a new project in https://velomax.2p2r.org/websig/lizmap/www/ and I would like to update the lizmap stack to use the latest software versions (with postgis).

@frazucca did you manage to add postgresql/postgis to you docker-compose?

I successfully "docker-compose up" the file https://github.com/3liz/lizmap-web-client/blob/master/tests/docker-compose.yml ...at least no error is raised, and all the containers are running.

When I try to add pgsql to https://github.com/3liz/lizmap-docker-compose/blob/master/docker-compose.yml, I get an error :

PS C:\temp\perso\2p2r\baladesCarto\logiciels\lizmap-docker-compose-master\pgsql> docker-compose --env-file .env.windows up
Starting pgsql_pgsql_1 ... done
Attaching to pgsql_pgsql_1
pgsql_1  | Changing postgres group id (999) to 0
pgsql_1  | groupmod: invalid group ID 'postgres'
pgsql_pgsql_1 exited with code 3

I simplified the docker-compose.yml file to contain only (the error is exactly the same with the full yml content) :

version: '3'
services:
  pgsql:
    image: 3liz/postgis:11-2.5
    environment:
      POSTGRES_DB: lizmap
      POSTGRES_PASSWORD: lizmap1234!
      POSTGRES_USER: lizmap
    volumes:
      - ${LIZMAP_DIR}/pg_data:/var/lib/postgresql/data
    ports:
      - ${LZMPGPORT}:5432
    networks:
      default:
        aliases:
          - db.lizmap.local

and the .env.windows file contains :

LIZMAP_VERSION_TAG=3.4
LIZMAP_USER_ID=1000
LIZMAP_USER_GID=1000
LIZMAP_DIR=./lizmap
QGIS_VERSION_TAG=3.10
QGIS_MAP_WORKERS=4
LIZMAP_PORT=8090
OWS_PORT=8091
LZMPGPORT=8130

I'm not using docker quite often...maybe I miss something..

I tried to find the recipe of the postgis image in 3liz github space, to try to understand, but I didn't find it.

Thank you for your help...

Gérald PS : my fork and new branch : https://github.com/2p2r/lizmap-docker-compose/tree/integratePgsql

[dmarteau]

pgsql_1 | Changing postgres group id (999) to 0 pgsql_1 | groupmod: invalid group ID 'postgres'

The startup script try to change the postgres uid/gid to the uid/gid of the mounted volume /var/lib/postgresql/data so that it use your chosen uid/gid instead of some random uid generated from the container.

If you are mounting a volume from a path that does not exists then it will be created as root and the startup script will try to change the postgres uid to 0 (root), and that's not permitted.

You should create the ${LIZMAP_DIR}/pg_data before starting the container, so that the postgres user defined in the container may change to the correct ui/gid.

Hint: do not use the POSTGRES_USER env variable as it is going to change the superuser name and this is generally not what you want.

[dmarteau]

I tried to find the recipe of the postgis image in 3liz github space, to try to understand, but I didn't find it.

Yes, that part is missing atm...

[dmarteau]

The docker recipe has been published to github: https://github.com/3liz/postgis

[gerald2545]

You should create the ${LIZMAP_DIR}/pg_data before starting the container, so that the postgres user defined in the container may change to the correct ui/gid.

hum, even if I create the lizmap\pg_data folder before starting the container, the error is the same.

Thank you for the postgis recipe, I'll have a look at it

[gerald2545]

sorry I don't understand how it can work...

in the postgis recipe, the $PGDATA folder (ENV PGDATA /var/lib/postgresql/data) is created and the ownership is given to user postgres:postgres mkdir -p "$PGDATA" && chown -R postgres:postgres "$PGDATA" && chmod 777 "$PGDATA"; \

then when the container is run, docker-entrypoint.sh is executed : docker_create_db_directories > docker_check_postgres_uid is called (https://github.com/3liz/postgis/blob/master/docker-entrypoint.sh#L11)

https://github.com/3liz/postgis/blob/master/docker-entrypoint.sh#L11 seems to return 0 (id of root), whereas the ownership was given to postgres, no?

then it tries to change the gid of group postgres to 0 in https://github.com/3liz/postgis/blob/master/docker-entrypoint.sh#L21 which is forbidden.... I don't know how can I prevent that...

[gerald2545]

hum, if it matters....I'm on a windows 10 computer

[dmarteau]

in the postgis recipe, the $PGDATA folder (ENV PGDATA /var/lib/postgresql/data) is created and the ownership is given to user postgres:postgres mkdir -p "$PGDATA" && chown -R postgres:postgres "$PGDATA" && chmod 777 "$PGDATA"; \

Yes, the directory is created by default in the container, so it may used as persistent volume.

https://github.com/3liz/postgis/blob/master/docker-entrypoint.sh#L11 seems to return 0 (id of root), whereas the ownership was given to postgres, no?

https://github.com/3liz/postgis/blob/master/docker-entrypoint.sh#L11 is only run when id=0 (root) so that it can change the uid/gid of the postgres user/group acconding to the uid/gid of the data volume. If the container is not run as root (--user) then this part is not run.

then it tries to change the gid of group postgres to 0 in https://github.com/3liz/postgis/blob/master/docker-entrypoint.sh#L21

It will try to change to 0 only if the mounted volume uid/gid (volume_uid, volume_gid) is owned by root which is bad.

hum, if it matters....I'm on a windows 10 computer

Yes it may matters because on windows and Osx docker run in a virtualized environnement (ake docker-machine), so it depends how ownership of your mounted directories is transferred to the container: you must ensure that your mounted directory is not owned by root in the container.

[dmarteau]

sorry I don't understand how it can work...

I use these images on a linux platform on a daily basis for testing multiple environnements, so I suspect that you are hitting some "docker on Windows" caveat.

Note that you don't have to mount a host volume, you may use a named volume: https://docs.docker.com/storage/volumes/#use-a-volume-with-docker-compose if you want persistence.

[gerald2545]

PS : I send the following lines for memory (I began to wrote it before you responded...but children are here;)) Thanks for your answers, I will check that tips

_I just built the image postgis using your Dockerfile with some more logs in docker_check_postgres_uid

docker_check_postgres_uid() {
    if [ "$(id -u)" = '0' ]; then

      local postgres_uid=$(id -u postgres)
      local postgres_gid=$(id -g postgres)

      local volume_uid=$(stat -c '%u' $PGDATA)
      local volume_gid=$(stat -c '%g' $PGDATA)
      **echo "postgres_uid = $postgres_uid"
      echo "postgres_gid = $postgres_gid"
      echo "volume_uid = $volume_uid"
      echo "volume_gid = $volume_gid"**
      if [[ "$postgres_gid" != $volume_gid ]]; then
          echo "Changing postgres group id ($postgres_gid) to $volume_gid"
          groupmod -g $volumes_gid postgres
      fi

      if [[ "$postgres_uid" != $volume_uid ]]; then
          echo "Changing postgres user id ($postgres_uid) to $volume_uid"
          usermod -u $volumes_uid postgres
      fi
   fi
}

and it seems the postgres_uid is equal to volume_uid in this case output :

postgres_uid = 999
postgres_gid = 999
volume_uid = 999
volume_gid = 999

why is this not the same with docker-compose?_

[gerald2545]

right, using

pgsql:
    image: 3liz/postgis:11-2.5
    environment:
      POSTGRES_DB: lizmap
      POSTGRES_PASSWORD: lizmap1234!
      POSTGRES_USER: lizmap
    volumes:
      - pg_data:/var/lib/postgresql/data
[....]
volumes:
  pg_data:

is working!!!

thank you for your help, I can go on...but I will need a persistent volume for that at some point

[gerald2545]

test done under linux > same error trying to change group ID from 999 to 998 (group of the user running docker in the host) pgsql_1 | groupmod: invalid group ID 'postgres'

wrong variable name used in docker-entrypoint.sh (https://github.com/3liz/postgis/issues/1)

[gerald2545]

now the error on my windows host is :

pgsql_1   | Changing postgres group id (999) to 0
pgsql_1   | groupmod: GID '0' already exists

0 is for the root user

[gerald2545]

I managed to use postgis with the following files, in a linux host :

docker-compose.zip Makefile.zip

[dmarteau]

0fac11a24ff41682377ae1540abc36acecd36bc2 Postgis support added to docker-compose using the profile option: https://docs.docker.com/compose/compose-file/compose-file-v3/#profiles