search

Bogdanp / nemea

Privacy focused website analytics.
GNU General Public License v3.0
36 stars 3 forks source link

Example doesn't work due to problem loading hll extension #1

Open winny- opened 4 years ago

winny- commented 4 years ago

The log does indicate one needs to add shared_preload_libraries = 'hll' to $PGDATA/postgresql.conf, so I did try that through adding a COPY to Dockerfile.postgres, and made sure my script ran before the SQL script (prefixing the shell script with 00 and SQL script with 01), but the same error still occurred.

Attaching to example_postgres_1, example_nemea_1
postgres_1  | The files belonging to this database system will be owned by user "postgres".
postgres_1  | This user must also own the server process.
postgres_1  | 
postgres_1  | The database cluster will be initialized with locale "en_US.utf8".
postgres_1  | The default database encoding has accordingly been set to "UTF8".
postgres_1  | The default text search configuration will be set to "english".
postgres_1  | 
postgres_1  | Data page checksums are disabled.
postgres_1  | 
postgres_1  | fixing permissions on existing directory /var/lib/postgresql/data ... ok
postgres_1  | creating subdirectories ... ok
postgres_1  | selecting default max_connections ... 100
postgres_1  | selecting default shared_buffers ... 128MB
postgres_1  | selecting default timezone ... Etc/UTC
postgres_1  | selecting dynamic shared memory implementation ... posix
postgres_1  | creating configuration files ... ok
postgres_1  | running bootstrap script ... ok
postgres_1  | performing post-bootstrap initialization ... ok
postgres_1  | syncing data to disk ... ok
postgres_1  | 
postgres_1  | Success. You can now start the database server using:
postgres_1  | 
postgres_1  |     pg_ctl -D /var/lib/postgresql/data -l logfile start
postgres_1  | 
postgres_1  | 
postgres_1  | WARNING: enabling "trust" authentication for local connections
postgres_1  | You can change this by editing pg_hba.conf or using the option -A, or
postgres_1  | --auth-local and --auth-host, the next time you run initdb.
postgres_1  | waiting for server to start....2019-11-19 05:17:15.453 UTC [46] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1  | 2019-11-19 05:17:15.470 UTC [47] LOG:  database system was shut down at 2019-11-19 05:17:15 UTC
postgres_1  | 2019-11-19 05:17:15.476 UTC [46] LOG:  database system is ready to accept connections
postgres_1  |  done
postgres_1  | server started
postgres_1  | CREATE DATABASE
postgres_1  | 
postgres_1  | 
postgres_1  | /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/initdb-hll.sql
postgres_1  | 2019-11-19 05:17:15.913 UTC [71] ERROR:  HLL can only be loaded via shared_preload_libraries
postgres_1  | 2019-11-19 05:17:15.913 UTC [71] HINT:  Add hll to shared_preload_libraries configuration variable in postgresql.conf
postgres_1  | 2019-11-19 05:17:15.913 UTC [71] STATEMENT:  CREATE EXTENSION IF NOT EXISTS hll;
postgres_1  | psql:/docker-entrypoint-initdb.d/initdb-hll.sql:1: ERROR:  HLL can only be loaded via shared_preload_libraries
postgres_1  | HINT:  Add hll to shared_preload_libraries configuration variable in postgresql.conf
example_postgres_1 exited with code 3
nemea_1     | tcp-connect: connection failed;
nemea_1     |  host not found
nemea_1     |   address: postgres
nemea_1     |   port number: 5432
nemea_1     |   system error: Name or service not known; gai_err=-2
nemea_1     |   context...:
nemea_1     |    /usr/share/racket/pkgs/db-lib/db/private/postgresql/main.rkt:13:0: postgresql-connect25
nemea_1     |    /usr/share/racket/collects/racket/contract/private/arrow-val-first.rkt:430:3
nemea_1     |    /usr/share/racket/pkgs/db-lib/db/private/generic/connect-util.rkt:331:25: send-arg636
nemea_1     |    .../more-scheme.rkt:261:28
nemea_1     |    /usr/share/racket/pkgs/db-lib/db/private/generic/connect-util.rkt:38:19: proc
nemea_1     |    /usr/share/racket/pkgs/db-lib/db/private/generic/connect-util.rkt:23:7
example_nemea_1 exited with code 1
Bogdanp commented 4 years ago

Try removing the containers and then the volume:

docker-compose stop \
  && docker-compose rm \
  && docker volume rm example_nemea-pg-data \
  && docker-compose build

And then see if your changes work. Unfortunately, I'm unable to reproduce this. Even if I purge everything and start from scratch, everything works as expected. The custom dockerfile explicitly installs the postgres-hll package for debian, which should add the extension to shared_preload_libraries so maybe that's failing somehow.

winny- commented 4 years ago

I tried the above, and was sure to run docker-compose build --no-cache --pull, and I get the following output when building:

edit - pasted wrong output

winston@snowcrash ~/.../nemea/example $ docker-compose stop   && docker-co
mpose rm -f  && docker volume rm example_nemea-pg-data   && docker-compose
 build --no-cache --pull && docker-compose up
Going to remove example_nemea_1, example_postgres_1
Removing example_nemea_1    ... done
Removing example_postgres_1 ... done
example_nemea-pg-data
nemea uses an image, skipping
Building postgres
Step 1/4 : FROM postgres:11
11: Pulling from library/postgres
Digest: sha256:c383e369f70d80204265f5178a32ef3e9fcaa2fdee7925266894f30812ef6619
Status: Image is up to date for postgres:11
 ---> b0c5652ae7e8
Step 2/4 : RUN apt-get update -y   && apt-get install -y --no-install-recommends postgresql-$PG_MAJOR-hll   && rm -rf /var/lib/apt/lists/*
 ---> Running in 39d60eca817f
Ign:1 http://deb.debian.org/debian stretch InRelease
Get:2 http://security.debian.org/debian-security stretch/updates InRelease [94.3 kB]
Get:3 http://apt.postgresql.org/pub/repos/apt stretch-pgdg InRelease [51.4 kB]
Get:4 http://deb.debian.org/debian stretch-updates InRelease [91.0 kB]
Get:5 http://deb.debian.org/debian stretch Release [118 kB]
Get:6 http://deb.debian.org/debian stretch Release.gpg [2,365 B]
Get:7 http://security.debian.org/debian-security stretch/updates/main amd64 Packages [508 kB]
Get:8 http://deb.debian.org/debian stretch-updates/main amd64 Packages [27.9 kB]
Get:9 http://apt.postgresql.org/pub/repos/apt stretch-pgdg/main amd64 Packages [206 kB]
Get:10 http://apt.postgresql.org/pub/repos/apt stretch-pgdg/11 amd64 Packages [2,585 B]
Get:11 http://deb.debian.org/debian stretch/main amd64 Packages [7,086 kB]
Fetched 8,187 kB in 2s (2,926 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
The following NEW packages will be installed:
  postgresql-11-hll
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 79.5 kB of archives.
After this operation, 233 kB of additional disk space will be used.
Get:1 http://apt.postgresql.org/pub/repos/apt stretch-pgdg/main amd64 postgresql-11-hll amd64 2.13-1.pgdg90+1 [79.5 kB]
debconf: delaying package configuration, since apt-utils is not installed
Fetched 79.5 kB in 0s (440 kB/s)
Selecting previously unselected package postgresql-11-hll.
(Reading database ... 13656 files and directories currently installed.)
Preparing to unpack .../postgresql-11-hll_2.13-1.pgdg90+1_amd64.deb ...
Unpacking postgresql-11-hll (2.13-1.pgdg90+1) ...
Setting up postgresql-11-hll (2.13-1.pgdg90+1) ...
Processing triggers for postgresql-common (210.pgdg90+1) ...
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
Building PostgreSQL dictionaries from installed myspell/hunspell packages...
Removing obsolete dictionary files:
Removing intermediate container 39d60eca817f
 ---> 72b083097b5c
Step 3/4 : RUN mkdir -p /docker-entrypoint-initdb.d
 ---> Running in 38300124b83f
Removing intermediate container 38300124b83f
 ---> 2c02e59cecbb
Step 4/4 : COPY ./initdb-hll.sql /docker-entrypoint-initdb.d/
 ---> 966576958653

Successfully built 966576958653
Successfully tagged example_postgres:latest
Creating volume "example_nemea-pg-data" with default driver
Creating example_postgres_1 ... done
Creating example_nemea_1    ... done
Attaching to example_postgres_1, example_nemea_1
postgres_1  | The files belonging to this database system will be owned by user "postgres".
postgres_1  | This user must also own the server process.
postgres_1  |
postgres_1  | The database cluster will be initialized with locale "en_US.utf8".
postgres_1  | The default database encoding has accordingly been set to "UTF8".
postgres_1  | The default text search configuration will be set to "english".
postgres_1  |
postgres_1  | Data page checksums are disabled.
postgres_1  |
postgres_1  | fixing permissions on existing directory /var/lib/postgresql/data ... ok
postgres_1  | creating subdirectories ... ok
postgres_1  | selecting default max_connections ... 100
postgres_1  | selecting default shared_buffers ... 128MB
postgres_1  | selecting default timezone ... Etc/UTC
postgres_1  | selecting dynamic shared memory implementation ... posix
postgres_1  | creating configuration files ... ok
postgres_1  | running bootstrap script ... ok
postgres_1  | performing post-bootstrap initialization ... ok
postgres_1  | syncing data to disk ...
postgres_1  | WARNING: enabling "trust" authentication for local connections
postgres_1  | You can change this by editing pg_hba.conf or using the option -A, or
postgres_1  | --auth-local and --auth-host, the next time you run initdb.
postgres_1  | ok
postgres_1  |
postgres_1  | Success. You can now start the database server using:
postgres_1  |
postgres_1  |     pg_ctl -D /var/lib/postgresql/data -l logfile start
postgres_1  |
postgres_1  | waiting for server to start....2019-11-19 10:20:50.629 UTC [46] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1  | 2019-11-19 10:20:50.661 UTC [47] LOG:  database system was shut down at 2019-11-19 10:20:50 UTC
postgres_1  | 2019-11-19 10:20:50.669 UTC [46] LOG:  database system is ready to accept connections
postgres_1  |  done
postgres_1  | server started
postgres_1  | CREATE DATABASE
postgres_1  |
postgres_1  |
postgres_1  | /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/initdb-hll.sql
postgres_1  | 2019-11-19 10:20:51.095 UTC [71] ERROR:  HLL can only be loaded via shared_preload_libraries
postgres_1  | 2019-11-19 10:20:51.095 UTC [71] HINT:  Add hll to shared_preload_libraries configuration variable in postgresql.conf
postgres_1  | 2019-11-19 10:20:51.095 UTC [71] STATEMENT:  CREATE EXTENSION IF NOT EXISTS hll;
postgres_1  | psql:/docker-entrypoint-initdb.d/initdb-hll.sql:1: ERROR:  HLL can only be loaded via shared_preload_libraries
postgres_1  | HINT:  Add hll to shared_preload_libraries configuration variable in postgresql.conf
example_postgres_1 exited with code 3
nemea_1     | tcp-connect: connection failed;
nemea_1     |  host not found
nemea_1     |   address: postgres
nemea_1     |   port number: 5432
nemea_1     |   system error: Name or service not known; gai_err=-2
nemea_1     |   context...:
nemea_1     |    /usr/share/racket/pkgs/db-lib/db/private/postgresql/main.rkt:13:0: postgresql-connect25
nemea_1     |    /usr/share/racket/collects/racket/contract/private/arrow-val-first.rkt:430:3
nemea_1     |    /usr/share/racket/pkgs/db-lib/db/private/generic/connect-util.rkt:331:25: send-arg636
nemea_1     |    .../more-scheme.rkt:261:28
nemea_1     |    /usr/share/racket/pkgs/db-lib/db/private/generic/connect-util.rkt:38:19: proc
nemea_1     |    /usr/share/racket/pkgs/db-lib/db/private/generic/connect-util.rkt:23:7
example_nemea_1 exited with code 1

I did try installing apt-utils before hand in the Dockefile, but that yielded a different error/warning:

debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (This frontend requires a controlling tty.)
debconf: falling back to frontend: Teletype
dpkg-preconfigure: unable to re-open stdin:

I did try this Dockerfile.postgres instead but that did not appear to work either, though there was no error/warning during installation:

FROM postgres:11

RUN apt-get update -y \
  && apt-get install -y --no-install-recommends apt-utils \
  && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends postgresql-$PG_MAJOR-hll \
  && rm -rf /var/lib/apt/lists/*

RUN mkdir -p /docker-entrypoint-initdb.d
COPY ./initdb-hll.sql /docker-entrypoint-initdb.d/

Some Sources: