Test and document Docker deployments on user workstations

[MBARIMike]

Deploying STOQS in production on Linux based hosted servers is documented in the README.md. (These are the instructions followed to deploy MBARI's public STOQS server).

In order to better support use and understanding of oceanographic data we'd also like to provide instructions for scientists to operate with STOQS installed on their own workstations. This would give them the ability to load databases, and perform data science operations, and generate visualizations on their own campaign data.

The workstation installation instructions need to be tested on MacOS, Windows, and Linux.

[MBARIMike]

There is an issue with deploying on MacOS. Here is the environment:

• Late 2013 MacBook Pro with macOS Mojave 10.14.2
• 1 TB solid state drive and 16 GB 1600 Mhz DDR3 RAM
• Docker Desktop Community Version 2.00.2
• Great Internet connection to GitHub and DockerHub

I clone the github repo, pull the images and deploy the web app. The unix time(1) command is used to record the elapsed time for the time consuming steps:

mkdir /tmp/GitHub
cd /tmp/GitHub

time git clone https://github.com/stoqs/stoqs.git stoqsgit
Cloning into 'stoqsgit'...
remote: Enumerating objects: 1008, done.
remote: Counting objects: 100% (1008/1008), done.
remote: Compressing objects: 100% (92/92), done.
remote: Total 58864 (delta 938), reused 971 (delta 916), pack-reused 57856
Receiving objects: 100% (58864/58864), 197.44 MiB | 10.31 MiB/s, done.
Resolving deltas: 100% (44798/44798), done.
Checking out files: 100% (26469/26469), done.

real    0m26.203s
user    0m9.229s
sys 0m6.120s

export STOQS_HOME=/tmp/GitHub/stoqsgit
cd $STOQS_HOME/docker
cp template.env .env

# Edited .env to set STOQS_HOME variable and uncomment CAMPAIGNS_MODULE line

time docker-compose pull
Pulling postgis   ... done
Pulling mapserver ... done
Pulling stoqs     ... done
Pulling nginx     ... done

real    2m33.514s
user    0m1.839s
sys 0m0.278s

time docker-compose up -d
Creating network "docker_default" with the default driver
Creating volume "docker_static-files" with default driver
Creating volume "docker_media-files" with default driver
Creating stoqs-mapserver ... done
Creating stoqs-postgis   ... done
Creating stoqs           ... done
Creating stoqs-nginx     ... done

real    0m9.021s
user    0m0.558s
sys 0m0.159s

It will take several more minutes to load initial data into the default database and run all the tests. The log output may be monitored with docker-compose logs -f, e.g.:

docker-compose logs -f
...
stoqs        | Wed Jan 30 17:34:01 2019 - [emperor] vassal /etc/uwsgi/django-uwsgi.ini is ready to accept requests
stoqs        | WSGI app 0 (mountpoint='') ready in 3 seconds on interpreter 0x55aab8b92010 pid: 245 (default app)
stoqs        | WSGI app 0 (mountpoint='') ready in 3 seconds on interpreter 0x55aab8b92010 pid: 250 (default app)

The local stoqs web app is now available at https://localhost. Note that you'll need to enter a security exception in your bowser to open the page:

Everything looks great, and in fact we can load additional databases from MBARI's public STOQS server so that we can have local data available for our own analyses, e.g.:

time curl -k https://stoqs.mbari.org/media/pg_dumps/stoqs_canon_september2018.pg_dump | \
>     docker exec -i stoqs pg_restore -Fc -U postgres -d stoqs_canon_september2018
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  195M  100  195M    0     0  1190k      0  0:02:48  0:02:48 --:--:-- 1993k

real    8m38.634s
user    0m0.688s
sys 0m1.026s

However, a problem arises when we try and restart our docker containers. The problem on the Mac is related to how the postgres data volumes are mounted in the container and the permissions needed inside the container. The details are spelled out in the next Comment on this Issue.

[MBARIMike]

With docker-compose logs -f running in one terminal window, execute this in another:

docker-compose restart postgis

The docker-compose logs -f window shows:

stoqs-postgis | chown: changing ownership of '/var/lib/postgresql/data/pg_wal': No such file or directory
stoqs-postgis exited with code 1

and with the postgis database container not running the whole STOQS app does not work.

We do not encounter this problem when deploying on Linux servers or workstations.

[MBARIMike]

There seems to be some hope in setting the user of the postgis container using --user option as documented for the postgres image. Maybe with this we can match the user id of the Mac host with the user id in the postgis container.

[MBARIMike]

Here are the steps to test various settings to get STOQS to work in Docker on a Mac:

1. Leave one terminal window open running docker-compose logs -f
2. In another window execute docker-compose build and docker-compose up -d commands
3. In yet another window (or with your favorite editor) edit text files

In $STOQS_HOME/docker edit the docker-compose.yml file. Add the line to the postgis service:

user: 399

where 399 is the uid of my login account on my Mac. (This is gotten from the id -u command.) Then rebuild the postgis image, and re-deploy it:

docker-compose build postgis
docker-compose up -d

The log window will show the services restarting and instead of seeing something like repeated

stoqs        | *** Waiting for postgres container ...

You'll see successful database connection log messages and eventually a running web service at https://localhost.

One can iterate on these steps making edits to the files that compose the STOQS application and monitoring the log output until the configuration is successful.

[MBARIMike]

With the PR https://github.com/stoqs/stoqs/pull/838 I'm able to run and restart my Docker containers on MacOS. Here are the diffs of the files that need to be changed following cloning the repository on MacOS:

git diff
diff --git a/docker/docker-compose.yml b/docker/docker-compose.yml
index 7d23d994..f171564f 100644
--- a/docker/docker-compose.yml
+++ b/docker/docker-compose.yml
@@ -19,7 +19,7 @@ services:
     ports:
       - "${STOQS_PGHOST_PORT}:5432"
     # Set user for deployment on MacOS, assign HOST_ID=<result of `id -u`> in your .env file
-    ##user: ${HOST_UID}
+    user: ${HOST_UID}
     environment:
       - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
       - STOQSADM_PASSWORD=${STOQSADM_PASSWORD}

diff docker/template.env docker/.env
16c16
< STOQS_HOME=/vagrant/dev/stoqsgit
---
> STOQS_HOME=/Users/mccann/GitHub/stoqsgit
19c19
< ##HOST_UID=
---
> HOST_UID=399
43c43
< ##CAMPAIGNS_MODULE=stoqs/mbari_campaigns.py
---
> CAMPAIGNS_MODULE=stoqs/mbari_campaigns.py

[andrewmccann]

This is the result from running this test on a custom desktop workstation

OS: Ubuntu 18.04.1 CPU: AMD Ryzen 5 1600 Memory: 16 GB DDR4 @ 3466Mhz Storage: 512 GB NVMe SSD Docker CE version: 18.09.1 Internet: ~70 Mbps download

The only thing I did differently from above was I ran docker-compose up without the -d switch to measure the time it took to create the databases, I had to manually exit out with CTRL+C once I saw NGINX was running so this time may be slightly increased from actual run time.

mkdir /tmp/git
cd /tmp/git

time git clone https://github.com/stoqs/stoqs.git stoqsgit
Cloning into 'stoqsgit'...
remote: Enumerating objects: 1081, done.
remote: Counting objects: 100% (1081/1081), done.
remote: Compressing objects: 100% (144/144), done.
remote: Total 58937 (delta 985), reused 1020 (delta 937), pack-reused 57856
Receiving objects: 100% (58937/58937), 197.46 MiB | 8.42 MiB/s, done.
Resolving deltas: 100% (44845/44845), done.
Checking out files: 100% (26469/26469), done.

real    0m26.103s
user    0m9.998s
sys     0m3.263s

# Created STOQS_HOME variable
export STOQS_HOME=/tmp/git/stoqsgit
cd $STOQS_HOME/docker

# Setup .env set STOQS_HOME directory and uncommented CAMPAIGNS_MODULE
cp template.env .env
vi .env

# Output omitted

time docker-compose pull                                                                                                                                                      

# Output omitted

real    5m6.275s
user    0m1.388s
sys     0m0.160s

time docker-compose up

# Output omitted

real    6m31.452s
user    0m2.609s
sys     0m0.893s