Idempotent Docker creation

[huneau]

Hi,

I tried to create an organisation but posgres tell me that zulip.com already exist (I tried to create mine, not zulip.com ). So I deleted the volume but the same thing happens over and over.

I'm right if I do

rm -rf /opt/docker/zulip

https://github.com/galexrt/docker-zulip/blob/master/docker-compose.yml#L10 ?

Why my DB is not deleted?

[galexrt]

@huneau How did you try to create an organisation?

[huneau]

Using the following doc : http://zulip.readthedocs.io/en/latest/prod-install.html#step-5-create-a-zulip-organization-and-login

[galexrt]

The error is that zulip shows you an error that the realm/domain zulip.com already exists?

[huneau]

Yes exactly.

[galexrt]

You only executed the command .../manage.py generate_realm_creation_link. Did it already return an error there or later in the process of creating the user?

[huneau]

Non other error, just zulip.com already exist.

[galexrt]

Are there any errors showing in the logs?

[huneau]

This is the only log a can retrieve.

https://pastebin.com/EPvZtbjV

We see the Database that say : 'zulip already exist'

[galexrt]

@huneau The "errors" shown aren't critical. They are expected. If there is a critical (real) error the container/script would exit.

[huneau]

This was the log just before I tried to add a new corp.

Now when I exec docker-compose up i've got that : https://pastebin.com/4Sk7xYsq

[galexrt]

Can you please post your docker-compose.yml (replace your passwords with x).

[huneau]

https://pastebin.com/mUddKj42

[galexrt]

@huneau Can you remove all directories and data under /opt/docker/zulip (your given data directory in the docker-compose.yml file) and then run the following commands:

docker-compose pull
docker-compose rm
docker-compose up

---

Note: After deleting my outdated Zulip data directory for all containers, I now can't reproduce this issue anymore.

[huneau]

nop, same issue : https://gist.github.com/huneau/185605d16c68244f42ad0660530d6d15

[galexrt]

@huneau Please post the logs of the database container.

[huneau]

posgres logs https://gist.github.com/huneau/41a91d65d921c04aa958c80f42bc340c

[galexrt]

That's weird it isn't creating the database for you. Can you run docker inspect DATABASE_CONTAINER (remove all password) and post it here.

---

Still can't reproduce this exact problem, but hit another problem.

[huneau]

https://gist.github.com/huneau/f58b7959febca0abff6ad5b517841516

[galexrt]

Please run the following command and post the output here:

docker run -it -e POSTGRES_DB=zulip -e POSTGRES_USER=zulip -e POSTGRES_PASSWORD=zulip quay.io/galexrt/postgres-zulip-tsearchextras:latest

---

I created an issue for the new issue I found: https://github.com/zulip/zulip/issues/4420

[huneau]

xxxxx@xxxxx-rnb:~/test-zulip$ docker run -it -e POSTGRES_DB=zulip -e POSTGRES_USER=zulip -e POSTGRES_PASSWORD=zulip quay.io/galexrt/postgres-zulip-tsearchextras:latest
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
creating template1 database in /var/lib/postgresql/data/base/1 ... ok
initializing pg_authid ... ok
initializing dependencies ... ok
creating system views ... ok
loading system objects' descriptions ... ok
creating collations ... ok
creating conversions ... ok
creating dictionaries ... ok
setting privileges on built-in objects ... ok
creating information schema ... ok
loading PL/pgSQL server-side language ... ok
vacuuming database template1 ... ok
copying template1 to template0 ... ok
copying template1 to postgres ... ok
syncing data to disk ... ok

WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.

Success. You can now start the database server using:

    pg_ctl -D /var/lib/postgresql/data -l logfile start

waiting for server to start....LOG:  database system was shut down at 2017-04-05 12:51:12 UTC
LOG:  MultiXact member wraparound protections are now enabled
LOG:  database system is ready to accept connections
LOG:  autovacuum launcher started
 done
server started
CREATE DATABASE

CREATE ROLE

/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/initdb-zulip-db.sh
CREATE SCHEMA
CREATE EXTENSION
CREATE EXTENSION

LOG:  received fast shutdown request
LOG:  aborting any active transactions
waiting for server to shut down....LOG:  autovacuum launcher shutting down
LOG:  shutting down
LOG:  database system is shut down
 done
server stopped

PostgreSQL init process complete; ready for start up.

LOG:  database system was shut down at 2017-04-05 12:51:14 UTC
LOG:  MultiXact member wraparound protections are now enabled
LOG:  database system is ready to accept connections
LOG:  autovacuum launcher started
^CLOG:  received fast shutdown request
LOG:  aborting any active transactions
LOG:  autovacuum launcher shutting down
LOG:  shutting down
LOG:  database system is shut down

[galexrt]

From the log the database gets created. Run docker-compose up and create a new terminal. In the new terminal run the following commands:

docker exec -it DATABASE_CONTAINER /bin/bash
su - postgres
psql
\l
\quit
exit
exit

(This shows the list of database in the postgres database container.)

[huneau]

postgres=# \l
                                 List of databases
   Name    |  Owner   | Encoding |  Collate   |   Ctype    |   Access privileges
-----------+----------+----------+------------+------------+-----------------------
 postgres  | postgres | UTF8     | en_US.utf8 | en_US.utf8 |
 template0 | postgres | UTF8     | en_US.utf8 | en_US.utf8 | =c/postgres          +
           |          |          |            |            | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.utf8 | en_US.utf8 | =c/postgres          +
           |          |          |            |            | postgres=CTc/postgres
(3 rows)

[galexrt]

The database should be created. Let's remove all images and gather some other info:

docker-compose stop # or just ctrl+c
docker-compose rm
docker rmi $(docker images -q)

and for the info (post this output):

docker info

[huneau]

When I try to docker rmi the image cannot be deleted because a image is currently using it. (An image that is terminates) I can't show you my docker info, I ve got other running image.

[galexrt]

docker info just shows information about the docker engine.

[huneau]

Containers: 62
 Running: 1
 Paused: 0
 Stopped: 61
Images: 105
Server Version: 17.03.0-ce
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 331
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 977c511eda0925a723debdc94d09459af49d082a
runc version: a01dafd48bc1c7cc12bdb01206f9fea7dd6feb70
init version: 949e6fa
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.8.0-41-generic
Operating System: Ubuntu 16.10
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.953 GiB
Name: avisto-rnb
ID: R3EV:RAW4:PMND:KBS6:M4NT:SZVJ:26EB:LRKY:MLFG:XLMG:2242:PBWQ
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Http Proxy: http://proxy-sophia.net.elsys-design.com:81/
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

[galexrt]

@huneau Thanks for the info!

Can you please try using the latest updated docker-compose.yml file (before running docker-compose up then, please delete all current data in /opt/docker/zulip). The current docker-compose.yml can be found here.

Please let me know if this fixes your problem.

[huneau]

It seems working, I try to make some crapy stuff to be sure it work and I close this issue.

[huneau]

OK fine, I close.

[galexrt]

@huneau Thanks for the feedback! If there is anything else I can help you with, please let me know.