DazWilkin
commented
6 years ago Ugh! I've got it mostly working but there's still a race condition with the trillian_db_seed that results in this not always working initially on a clean (docker system prune) system :-(
https://github.com/google/trillian/blob/master/examples/deployment/docker-compose.yml
I have docker-compose working but I'd value some insights into which of the several changes is required. I'm confident it's some combination of several.
host=mysql does not work
The docker-compose.yml uses references to mysql (I believe in reference to service: mysql) in an attempt to reference the database. For me this does not work. After much consternation, I realized that my debugging included a possible solution. This worked:
adminer:
image: adminer
restart: always
ports:
- 8080:8080
depends_on:
- mysql
links:
- mysql:dbThe other containers don't assume (unlike adminer) that the container is called db but it struck me that perhaps the links is necessary and I think (!?) this provides a way to uniquely reference other containers. When I run docker-compose without links the database is not actually called mysql but is called e.g. mysql_1. Using links for all the containers appears to solve host resolution.
So each service needs:
links:
- mysql:dbAnd then references to the database become e.g. test:zaphod@tcp(db:3306)/test
protocol=TCP appears to be required
I tried running MySQL and MariaDB containers standalone and then connecting to them from a 2nd container as a client. The only way I could get this to work was when I included --protocol=TCP to force comms over the TCP port. Unfortunately, resetdb.sh does not include this flag (see next)
docker run \
--env=MYSQL_ROOT_PASSWORD=$DB_PASSWORD \
-it \
--publish=3306:3306 \
mysql:8Or:
docker run \
--env=MYSQL_ROOT_PASSWORD=$DB_PASSWORD \
-it \
--publish=3306:3306 \
mariadb:10.3.7And:
docker run \
--interactive \
--tty \
--net=host mysql:8 \
mysql \
--user=root \
--password=${DB_PASSWORD} \
--host=localhost \
--port=3306 \
--protocol=TCP \
--execute="show databases;"Works reliably.
resetdb.sh
I'm loathe to propose changes to resetdb.sh knowing that this is used elsewhere but it appears that this must include the --protocol=TCP as a flag to address the above issue.
While I was in here I made the password property consistent with the other long-names so -p --> --password=
The result:
#!/bin/bash
set -e
usage() {
echo "$0 [--force] [--verbose] ..."
echo "accepts environment variables:"
echo " - DB_NAME"
echo " - DB_USER"
echo " - DB_PASSWORD"
echo " - DB_HOST"
echo " - DB_PORT"
echo " - DB_PROT"
}
collect_vars() {
# set unset environment variables to defaults
[ -z ${DB_USER+x} ] && DB_USER="root"
[ -z ${DB_NAME+x} ] && DB_NAME="test"
[ -z ${DB_HOST+x} ] && DB_HOST="localhost"
[ -z ${DB_PORT+x} ] && DB_PORT="3306"
[ -z ${DB_PROT+x} ] && DB_PROT="TCP"
FLAGS=()
# handle flags
FORCE=false
VERBOSE=false
while [[ $# -gt 0 ]]; do
case "$1" in
--force) FORCE=true ;;
--verbose) VERBOSE=true ;;
*) FLAGS+=("$1")
esac
shift 1
done
FLAGS+=(--user="${DB_USER}")
FLAGS+=(--host="${DB_HOST}")
FLAGS+=(--port="${DB_PORT}")
FLAGS+=(--protocol="${DB_PROT}")
# Optionally print flags (before appending password)
[[ ${VERBOSE} = 'true' ]] && echo "- Using MySQL Flags: ${FLAGS[@]}"
# append password if supplied
[ -z ${DB_PASSWORD+x} ] || FLAGS+=(--password="${DB_PASSWORD}")
}
main() {
collect_vars "$@"
readonly TRILLIAN_PATH=$(go list -f '{{.Dir}}' github.com/google/trillian)
# what we're about to do
echo "Warning: about to destroy and reset database '${DB_NAME}'"
[[ ${FORCE} = true ]] || read -p "Are you sure? [Y/N]: " -n 1 -r
echo # Print newline following the above prompt
if [ -z ${REPLY+x} ] || [[ $REPLY =~ ^[Yy]$ ]]
then
echo "Resetting DB..."
echo "Flags: ${FLAGS[@]}"
mysql "${FLAGS[@]}" -e "DROP DATABASE IF EXISTS ${DB_NAME};"
mysql "${FLAGS[@]}" -e "CREATE DATABASE ${DB_NAME};"
mysql "${FLAGS[@]}" -e "GRANT ALL ON ${DB_NAME}.* TO '${DB_NAME}' IDENTIFIED BY 'zaphod';"
mysql "${FLAGS[@]}" -D ${DB_NAME} < ${TRILLIAN_PATH}/storage/mysql/storage.sql
echo "Reset Complete"
fi
}
main "$@"
docker-compose.yml
Because of the minimal Dockerfiles and a desire to make Docker Compose and Kubernetes consistent, I've pulled much of the formatting into the docker-compose.yml file.
Unfortunately, whereas Kubernetes permits environment variables (values) to be incorporated into the runtime command-line, Docker Compose does not and so the Docker Compose variant of the file is more static than I'd prefer. These variables could possibly (!?) be recreated and provided by the environment (e.g. ${DB_FLAG} and ${DB_PROVIDER} etc. as is done with ${DB_PASSWORD} but for now:
Also dropped -u mysql from the trillian-db-seed command it is redundant if DB_HOST=db is provided too.
version: "3.2"
services:
mysql:
image: mariadb:10.3.7
restart: always
environment:
- MYSQL_ROOT_PASSWORD=${DB_PASSWORD}
trillian-db-seed:
build:
context: ../..
dockerfile: ./examples/deployment/docker/db_client/Dockerfile
depends_on:
- mysql
links:
- mysql:db
environment:
- DB_USER=root
- DB_PASSWORD=${DB_PASSWORD}
- DB_HOST=db
- DB_PORT=3306
- DB_PROT=TCP
command: ./scripts/resetdb.sh --verbose --force
trillian-log-server:
build:
context: ../..
dockerfile: examples/deployment/docker/log_server/Dockerfile.new
restart: always
ports:
- "8090:8090"
- "8091:8091"
depends_on:
- mysql
links:
- mysql:db
environment:
- DB_USER=root
- DB_PASSWORD=$DB_PASSWORD
command: "--mysql_uri=test:zaphod@tcp(db:3306)/test --storage_system=mysql --rpc_endpoint=0.0.0.0:8090 --http_endpoint=0.0.0.0:8091 --alsologtostderr"
trillian-log-signer:
build:
context: ../..
dockerfile: examples/deployment/docker/log_signer/Dockerfile.new
restart: always
ports:
- "8092:8091"
depends_on:
- mysql
links:
- mysql:db
environment: [
"DB_USER=root",
"DB_PASSWORD=$DB_PASSWORD",
]
command: "--mysql_uri=test:zaphod@tcp(db:3306)/test --storage_system=mysql --http_endpoint=0.0.0.0:8091 --sequencer_guard_window=0s --sequencer_interval=300ms --num_sequencers=10 --batch_size=2000 --force_master=true --alsologtostderr"Feedback and guidance welcome.
jsha
daviddrysdale
jxsl13
docker-compose up --buildyields:and
trillian-db-seedappears to never get a connection (throughwait-for-it) to the database but I think it's notwait-for-itthat's at fault as if I replace this with a longsleep 15sand try that way, I get errors too:IIUC it should not be attempting to connect through that socket but the port. The output suggests that it is using
localhost:3306but this doesn't work even if I change this tomysqlusing$DB_HOST.The database is ready:
If I add
adminerinto the compose file, I'm able to reach the database using it:and then I can
In this case, it appears to have gotten as far as creating (a)
localhost:8080with${DB_PASSWORD}testDB but it contains no tables; (b)testuser.There's something in the bowels of docker-compose and|or MySQL(MariaDB) and|or Trillian that I'm not seeing because I'm entirely unable to understand what's broken and how to fix it and would value some inspiration.
and