Open benpsnyder opened 5 months ago
Thanks for suggesting this @benpsnyder. I'm tagging two of our maintainers who I think might have more thoughts/opinions/ideas about this...
@pavish and @Anish9901 can you respond here with your thoughts on this proposal?
Here is my working configuration with environment variables (just replaced my company with ACME)
Please note I deploy with docker swarm init
on a single host
version: '3'
services:
############################
caddy:
image: lucaslorentz/caddy-docker-proxy:ci-alpine
ports:
- 80:80
- 443:443
networks:
- caddy
volumes:
- /var/run/docker.sock:/var/run/docker.sock
# this volume is needed to keep the certificates
# otherwise, new ones will be re-issued upon restart
- caddy_data:/data
deploy:
labels: # Global options
caddy.email: systems@acme.com
placement:
constraints:
- node.role == manager
replicas: 1
restart_policy:
condition: any
resources:
reservations:
cpus: '0.2'
memory: 200M
############################
whoami:
image: containous/whoami
networks:
- caddy
expose:
- 80
deploy:
replicas: 1
restart_policy:
condition: any
labels:
caddy: whoami.${ACME_DEPLOYMENT_DOMAIN_PARENT}
caddy.reverse_proxy: '{{upstreams}}'
mathesar:
image: mathesar/mathesar-prod:latest
environment:
# First we load the variables configured above.
# You can generate one at https://djecrety.ir/ or by running:
SECRET_KEY: ${ACME_MSAR_SECRET_KEY}
DOMAIN_NAME: ${ACME_MSAR_DOMAIN_NAME}
POSTGRES_DB: ${ACME_MSAR_DB_NAME}
POSTGRES_USER: ${ACME_MSAR_DB_USER}
POSTGRES_PASSWORD: ${ACME_MSAR_DB_PASS}
POSTGRES_HOST: ${ACME_MSAR_DB_HOST}
POSTGRES_PORT: ${ACME_MSAR_DB_PORT}
DJANGO_SETTINGS_MODULE: config.settings.production
# We set ALLOWED_HOSTS to * (allow all hosts) by default here since we are
# relying on caddy to manage which domains could access the mathesar web
# service. If you do not want to use caddy add the domain(s) that you
# want to ALLOWED_HOSTS. Doing so will restrict traffic from all other
# domains.
ALLOWED_HOSTS: ${ALLOWED_HOSTS:-*}
# WARNING: MATHESAR_DATABASES is deprecated, and will be removed in a future release.
MATHESAR_DATABASES: ${MATHESAR_DATABASES:-}
volumes:
- msar_static:/code/static
- msar_media:/code/.media
healthcheck:
test: curl -f http://mathesar:8000
interval: 10s
timeout: 5s
retries: 30
start_period: 5s
# If using caddy, expose the internal port 8000 only to other containers and
# not the docker host.
networks:
- caddy
expose:
- 8000
labels:
caddy: 'https://${ACME_DEPLOYMENT_SUBDOMAIN_MSAR}.${ACME_DEPLOYMENT_DOMAIN_PARENT}'
caddy.reverse_proxy: '{{upstreams http 8000}}'
caddy.encode: 'zstd gzip'
caddy_0.handle_path: '/media/*'
caddy_0.handle_path.@downloads.query: 'dl=*'
caddy_0.handle_path.header_@downloads: 'Content-disposition "attachment; filename={query.dl}"'
caddy_0.handle_path.file_server: ''
caddy_0.handle_path.file_server.precompressed: 'br zstd gzip'
caddy_0.handle_path.file_server.root: '/code/.media/'
caddy_1.handle_path: '/static/*'
caddy_1.handle_path.file_server: ''
caddy_1.handle_path.file_server.precompressed: 'br zstd gzip'
caddy_1.handle_path.file_server.root: '/code/static/'
networks:
caddy:
driver: overlay
attachable: true
volumes:
caddy_data: {}
msar_media: {}
msar_static: {}
Thanks for suggesting this @benpsnyder! I'll admit that this does look very cool, and I'm glad you got it working on your setup.
The thing is, we don't really expect our caddy configuration to change that often, nor do we expect our users to fiddle around with caddy's settings if we can provide something that works out of the box.
We expect docker-compose.yml
to serve as a flexible blueprint, allowing users to customize it for deploying Mathesar on alternate setups like e.g. k8s. So, I don't think it's advantageous for us to use the suggested image instead of our own.
Thanks for suggesting this @benpsnyder! I'll admit that this does look very cool, and I'm glad you got it working on your setup.
The thing is, we don't really expect our caddy configuration to change that often, nor do we expect our users to fiddle around with caddy's settings if we can provide something that works out of the box.
We expect
docker-compose.yml
to serve as a flexible blueprint, allowing users to customize it for deploying Mathesar on alternate setups like e.g. k8s. So, I don't think it's advantageous for us to use the suggested image instead of our own.
One of the chief reasons for suggesting this change is k8s and keeping total number of containers to a minimum 😎
Imagine if the majority of projects shipped an extra container just for the http ingress ... we'd have a bit of chaos with so much extra overhead.
I suggest we arrive at a solution using labels 🏷️ as this reduces overhead. Using this variant of a caddy image makes it incredibly easy to have one ingress container with labels on many apps and supporting infrastructure containers in one compose
If be happy to supply a traefik approach too, also with labels 🏷️, if the desired outcome is maximum k8s support
You won't even have to use Caddy when using Ingress! You'd directly point Ingress to the port(8000) which is exposed by mathesar/mathesar-prod:latest
Something like this:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: mathesar-ingress
spec:
defaultBackend:
service:
name: mathesar
port:
number: 8000
EDIT: @benpsnyder I did some more research, and It seems that you were talking about using docker swarm's ingress instead of k8s in which case using the suggested image would make sense if the goal is to reduce the number of containers. It seems that swarm's ingress behaves differently to the one used by k8s, in the sense that using caddy with k8s' ingress won't generate automatic SSL certificates, however, It seems that's not the case while using swarm's ingress.
Nonetheless, it looks like traefik can provide SSL certificates for both docker swarm's as well as k8s' ingress, thanks for suggesting this btw! It would be great if you could provide a config for traefik since you've already expressed interest in doing so.
Can you use this approach with normal, off the shelf, Caddy, and with an appropriate Caddyfile? This is what would work for me and my current setup. Many thanks!
Can you use this approach with normal, off the shelf, Caddy, and with an appropriate Caddyfile? This is what would work for me and my current setup.
Yes you most definitely can @amca01, here is the link to our Caddyfile.
Here is an example configuration using the Caddyfile mount in docker compose:
caddy-reverse-proxy:
image: caddy:<version>
# This service needs the config variables defined above.
environment: *config
ports:
- "80:80"
- "443:443"
volumes:
- <path to Caddyfile on your machine>:/etc/caddy/Caddyfile # Your Caddyfile mount.
- ./msar/media:/code/media
- ./msar/static:/code/static
- ./msar/caddy:/data
Thank you very much, @Anish9901. I do everything with docker compose files, and at the moment I'm using a plain "vanilla" Caddy, defined as
caddy:
container_name: caddy
image: caddy:2.7.6
restart: always
ports:
- "80:80"
- "443:443"
- "443:443/udp"
networks:
- caddy_net
volumes:
- /home/amca/Docker/Caddy/Caddyfile:/etc/caddy/Caddyfile
- /home/amca/Docker/Caddy/www:/srv
- caddy_data:/data
- caddy_config:/config
Do I have to add the Mathesar volumes as well to this Caddy block? I already, as you see, have a file structure which includes a Caddy subdirectory in my Docker directory. Do I need separate directories for Mathesar? Currently my Caddy data is kept in the bind mount "caddy_data"; would it be confusing to have two directories both pointing to Caddy's "/data"?
Thanks for providing this snippet @amca01, since you already have a caddy_data
directory, you won't need to add ./msar/caddy:/data
.
You'd, however, need to add the following as volumes:
./msar/media:/code/media
(stores user uploaded datafiles(.csv/.tsv) to Mathesar)./msar/static:/code/static
(stores static files for Mathesar)Please do note that, the paths mentioned above are relative and you may need to change ./msar/media
& ./msar/static
if you are running your caddy docker-compose file from a different directory to the one that was used to run mathesar's docker-compose file.
That makes a lot of sense, and again, many thanks @Anish9901. I'll give this a go tomorrow; right now (in Australia) it's bed time!
The current trouble is setting the POSTGRES_HOST
environment variable. What it is supposed to be? Both the values of mathesar_db
and my VPS hostname produce errors shown in docker logs mathesar_service
. With value mathesar_db
the error is
django.db.utils.OperationalError: could not translate host name "mathesar_db" to address: Name or service not known
I see from the example given by @benpsnyder above, that the value is to be set to ${ACME_MSAR_DB_HOST}
- whatever that is!
Of course, the problem could be my setup; here are the relevant lines from my docker compose file:
mathesar_service:
container_name: mathesar_service
image: mathesar/mathesar-prod:latest
environment:
SECRET_KEY: ${SECRET_KEY}
POSTGRES_DB: ${POSTGRES_DB}
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_PORT: ${POSTGRES_PORT}
POSTGRES_HOST: ${POSTGRES_HOST}
DJANGO_SETTINGS_MODULE: config.settings.production
ALLOWED_HOSTS: "*"
entrypoint: ./run.sh
volumes:
- ./msar/static:/code/static
- ./msar/media:/code/media
depends_on:
mathesar_db:
condition: service_healthy
healthcheck:
test: curl -f http://localhost:8000
interval: 10s
timeout: 5s
retries: 30
start_period: 5s
# If using caddy, expose the internal port 8000 only to other containers and
# not the docker host.
networks:
- caddy_net
expose:
- "8000"
mathesar_db:
image: postgres:13
container_name: mathesar_db
# This service needs the config variables defined above.
environment:
POSTGRES_DB: ${POSTGRES_DB}
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_PORT: ${POSTGRES_PORT}
# Expose the internal port 5432 only to other containers and not
# the underlying host.
expose:
- "5432"
volumes:
- ./msar/pgdata:/var/lib/postgresql/data
healthcheck:
test: [ "CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"]
interval: 5s
timeout: 1s
retries: 30
start_period: 5s
and from my .env
file (slightly redacted):
SECRET_KEY=<randomly generated>
DOMAIN_NAME=mathesar.mysite.net
POSTGRES_PASSWORD=mathesar
POSTGRES_DB=mathesar_django
POSTGRES_USER=mathesar
POSTGRES_PORT=5432
POSTGRES_HOST=mathesar_db
All of my files are in the same directory, and I know that Caddy is set up correctly as it works with other services. Should I be adding the mathesar_db
container to the caddy_net
network as well? I didn't think I needed to as there is not an outside connection to the database. But maybe a docker network is needed? Anyway, if there are any glaringly obvious errors in my files, I'd be glad to know of them! Many thanks.
Alasdair
@amca01 Since you are explicitly adding a network(caddy_net
) for the mathesar_service
container, it will be isolated from the db container and hence the error.
You can fix this by creating another network and adding it to the mathesar_service
and mathesar_db
containers.
Here is a similar setup that I've got working, please notice the networks
section in the following snippets:
docker-compose.yml
service:
container_name: mathesar_service
image: mathesar/mathesar-prod:latest
environment:
# First we load the variables configured above.
<<: *config
DJANGO_SETTINGS_MODULE: config.settings.production
# We set ALLOWED_HOSTS to * (allow all hosts) by default here since we are
# relying on caddy to manage which domains could access the mathesar web
# service. If you do not want to use caddy add the domain(s) that you
# want to ALLOWED_HOSTS. Doing so will restrict traffic from all other
# domains.
ALLOWED_HOSTS: ${ALLOWED_HOSTS:-*}
# WARNING: MATHESAR_DATABASES is deprecated, and will be removed in a future release.
MATHESAR_DATABASES: ${MATHESAR_DATABASES:-}
volumes:
- ./msar/static:/code/static
- ./msar/media:/code/media
depends_on:
db:
condition: service_healthy
healthcheck:
test: curl -f http://localhost:8000
interval: 10s
timeout: 5s
retries: 30
start_period: 5s
# If using caddy, expose the internal port 8000 only to other containers and
# not the docker host.
networks:
- caddy_net_
- mathesar_default
expose:
- "8000"
# Uncomment the following if not using caddy
# ports:
# - ${HOST_PORT:-8000}:8000
#-----------------------------------------------------------------------------
# PostgreSQL Database
#
# This service provides a Postgres database instance for holding both internal
# Mathesar data, as well as user data if desired, using the official
# PostgreSQL image hosted on Docker Hub
#
# As configured, this service exposes Postgres' default port (5432) to other
# services, allowing the Mathesar web sevice to connect to it.
#
db:
image: postgres:13
container_name: mathesar_db
# This service needs the config variables defined above.
environment: *config
# Expose the internal port 5432 only to other containers and not
# the underlying host.
expose:
- "5432"
volumes:
- ./msar/pgdata:/var/lib/postgresql/data
networks:
- mathesar_default
healthcheck:
test: [ "CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"]
interval: 5s
timeout: 1s
retries: 30
start_period: 5s
networks:
caddy_net_:
name: caddy_net
external: true
mathesar_default:
name: mathesar_default
external: true
caddy.yml
services:
caddy_msar:
container_name: caddy
image: caddy:2.7.6
# This service needs the config variables defined above.
environment:
DOMAIN_NAME: ${DOMAIN_NAME:-http://localhost}
ports:
- "80:80"
- "443:443"
networks:
- caddy_net_
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile
- ./msar/media:/code/media
- ./msar/static:/code/static
- ./msar/caddy:/data
networks:
caddy_net_:
name: caddy_net
external: true
Thank you so much, @Anish9901 - that works just as it should! Brilliant. I'm utterly delighted now to be able to run mathesar concurrently with my other services.
I'm glad it worked out @amca01 :)
Problem
Caddy is amazing! Your current
docker-compose.yml
requires a custommathesar/mathesar-caddy:latest
image. This can be eliminated!Proposed solution
Leverage the
lucaslorentz/caddy-docker-proxy
image and abandon the custommathesar/mathesar-caddy:latest
image. All we have to do is add labels to themathesar/mathesar-prod:latest
service definition.Additional context
The implementation will look something like this. I will post back when I confirm what works for me.