seancolsen
commented
6 months ago Thanks for reporting this, @jeromeperez. We'll have someone look into this and help troubleshoot it.
Closed jeromeperez closed 6 months ago
seancolsen
commented
6 months ago Thanks for reporting this, @jeromeperez. We'll have someone look into this and help troubleshoot it.
mathemancer
commented
6 months ago Hello, @jeromeperez . It would help us diagnose the issue if you would attach the entire docker-compose.yml file you're using. Would you mind providing that?
jeromeperez
commented
6 months ago No problem
version: "3.9"
# This file defines a viable production setup for Mathesar.
#
# It can be used in production directly, or used as an example to help define
# your own infrastructure.
#
#-------------------------------------------------------------------------------
# PREREQUISITES
#
# Please double-check that your docker setup meets the following criteria:
#
# OS: Linux, Mac, Windows(WSL).
# Docker v23+ $ docker version
# Docker Compose v2.10+ $ docker compose version
#
#-------------------------------------------------------------------------------
# HOW TO USE THIS FILE
#
# First, make sure you meet the prerequisites, add a secret key below, and then
# run:
#
# $ docker compose -f docker-compose.yml up
#
# Note: You may need to run Docker commands using sudo, depending on your setup.
# Running Docker in rootless mode isn't currently supported.
#
#-------------------------------------------------------------------------------
# CONFIG
#
# Customize your Mathesar installation with the following variables.
# See https://docs.mathesar.org/configuration/env-variables/ for more info.
#
x-config: &config
# (REQUIRED) Replace '?' with '-' followed by a 50 character random string.
# You can generate one at https://djecrety.ir/ or by running:
# echo $(cat /dev/urandom | LC_CTYPE=C tr -dc 'a-zA-Z0-9' | head -c 50)
SECRET_KEY: ${SECRET_KEY:-zAzcRC5SopNKvUoHzFYd1yQcFA7mKycayFG1Bd6DubHNJiY7Yl}
# (Optional) Replace 'http://localhost' with custom domain(s) e.g.
# 'yourdomain.com, 127.0.0.1' to manage the host(s) at which you want to
# access Mathesar over http or https
DOMAIN_NAME: ${DOMAIN_NAME:-http://mydomain, http://127.0.0.1, http://162.38.X.X}
# Edit the POSTGRES_* variables if you are not using the db service provided
# below, or if you want to use a custom database user.
# (Optional) Replace 'mathesar_django' with any custom name for the internal
# database managed by mathesar web-service
POSTGRES_DB: ${POSTGRES_DB:-mathesar_django}
# (Optional) Replace 'mathesar' with any custom username for the
# aforementioned database
POSTGRES_USER: ${POSTGRES_USER:-mathesar}
# (Optional) Replace 'mathesar' with any custom password for the
# aforementioned database
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-dbpassword}
# (Optional) Replace 'mathesar_db' with the name of the host running postgres
POSTGRES_HOST: ${POSTGRES_HOST:-db.data.isem-evolution.fr}
# (Optional) Replace '5432' with the port on which postgres is running
POSTGRES_PORT: ${POSTGRES_PORT:-5432}
#-------------------------------------------------------------------------------
# ADDITIONAL INFO ABOUT CONFIG VARIABLES
#
# SECRET_KEY:
# Default: N/A
# Info: A unique secret key required to be set by the user for Django's
# security protection features. It should be 50 random characters. You
# can read more about it here:
# https://docs.djangoproject.com/en/4.2/ref/settings/#secret-key
# Example: a_very_insecure_secret_key1*zobb123)k(_d1%wubkv6#
#
# DOMAIN_NAME:
# Default: http://localhost
# Info: Specifies the domains that can access Mathesar over http(port 80)
# or https(port 443), also automatically creating SSL certificates
# for the same. If you want to host an instance of Mathesar over the
# internet or over your local network, add those domains here.
# Example: yourdomain.com, *.subdomain.com, 127.0.0.1
#
# POSTGRES_DB:
# Default: mathesar_django
# Info: Specifies a name for the database that will be created and used by
# Mathesar for managing internal data.
# Example: zeus
#
# POSTGRES_USER:
# Default: mathesar
# Info: Specifies creation of a user with superuser privileges
# and a database with the same name.
# Example: athena
#
# POSTGRES_PASSWORD:
# Default: mathesar
# Info: Specifies the superuser password that is required to be set for the
# PostgreSQL docker image.
# Example: apollo
#
# POSTGRES_HOST:
# Default: mathesar_db (name of the db service provided below)
# Info: Specifies the host name on which portgres listen for connections
# from client applications.
# Example: kratos
#
# POSTGRES_PORT:
# Default: 5432
# Info: Specifies the port on which portgres listen for connections from
# client applications.
# Example: 5555
#
#-------------------------------------------------------------------------------
# INFO ABOUT VOLUMES
#
# Volumes are used by Mathesar to persist essential data.
#
# Running this compose file will automatically create a subdirectory named
# "msar" with the following file structure:
#
# msar
# ├── caddy/ (stores certificates, keys, and other information for Caddy)
# ├── media/ (stores user uploaded datafiles(.csv/.tsv) to Mathesar)
# ├── pgdata/ (stores PostgreSQL data)
# └── static/ (stores static files for Mathesar)
#
#-------------------------------------------------------------------------------
# MATHESAR SERVICES
#
# The next section defines various containers in a workable production setup.
#
services:
#-----------------------------------------------------------------------------
# Mathesar web service
#
# This service provides the main web server required to run Mathesar, using
# our official Docker image hosted on Docker Hub
#
# As configured, this service exposes port 8000 to other services but not the
# host system. This isolates it from being directly accessed by the host
# while allowing access via caddy.
#
service:
container_name: mathesar_service
image: mathesar/mathesar-prod:latest
environment:
# First we load the variables configured above.
<<: *config
DJANGO_SETTINGS_MODULE: config.settings.production
# We set ALLOWED_HOSTS to * (allow all hosts) by default here since we are
# relying on caddy to manage which domains could access the mathesar web
# service. If you do not want to use caddy add the domain(s) that you
# want to ALLOWED_HOSTS. Doing so will restrict traffic from all other
# domains.
ALLOWED_HOSTS: ${ALLOWED_HOSTS:-*}
# WARNING: MATHESAR_DATABASES is deprecated, and will be removed in a future release.
MATHESAR_DATABASES: ${MATHESAR_DATABASES:-}
entrypoint: ./run.sh
volumes:
- ./msar/static:/code/static
- ./msar/media:/code/media
depends_on:
db:
condition: service_healthy
healthcheck:
test: curl -f http://localhost:8080
interval: 10s
timeout: 5s
retries: 30
start_period: 5s
# If using caddy, expose the internal port 8000 only to other containers and
# not the docker host.
expose:
- "8000"
# Uncomment the following if not using caddy
# ports:
# - ${HOST_PORT:-8080}:8080
#-----------------------------------------------------------------------------
# PostgreSQL Database
#
# This service provides a Postgres database instance for holding both internal
# Mathesar data, as well as user data if desired, using the official
# PostgreSQL image hosted on Docker Hub
#
# As configured, this service exposes Postgres' default port (5432) to other
# services, allowing the Mathesar web sevice to connect to it.
#
db:
image: postgres:13
container_name: mathesar_db
# This service needs the config variables defined above.
environment: *config
# Expose the internal port 5432 only to other containers and not
# the underlying host.
expose:
- "5432"
volumes:
- ./msar/pgdata:/var/lib/postgresql/data
healthcheck:
test: [ "CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"]
interval: 5s
timeout: 1s
retries: 30
start_period: 5s
#-----------------------------------------------------------------------------
# Caddy
#
# This service provides a reverse proxy for the Mathesar web server, using our
# custom Caddy image hosted on Docker Hub. That image is customized to use a
# Caddyfile with an appropriate configuration for Mathesar.
#
# Specifically, this service routes the requests to backend and the web
# frontend of Mathesar while also serving essential staic files and user
# uploaded datafiles(.csv/.tsv). It also provides SSL certificates
# automatically for any custom domain(s) listed in DOMAIN_NAME that you might
# want to use to access Mathesar.
#
# This service maps the default port for http(80) and https(443) of the host
# system to that of docker's for allowing access to Mathesar over http or
# https.
#
caddy-reverse-proxy:
image: mathesar/mathesar-caddy:latest
# This service needs the config variables defined above.
environment: *config
ports:
- "8080:80"
- "44343:443"
volumes:
- ./msar/media:/code/media
- ./msar/static:/code/static
- ./msar/caddy:/dataSo, I'm failing to reproduce this using that docker compose file. It seems like it should be set up to publish 8080 and 44343 from Caddy, and that's what's happening on my machine. To help me further diagnose, would you mind posting the output of:
docker ps --format "table {{.Names}}\t{{.Ports}}" # May require sudoand
ss -tulpn | grep LISTEN # May require sudo?
It would also help if you would walk me through how you've determined that Mathesar is listening on port 80? If you already have another service consuming that port, your OS should prevent such behavior.
jeromeperez
commented
6 months ago So, I'm failing to reproduce this using that docker compose file. It seems like it should be set up to publish 8080 and 44343 from Caddy, and that's what's happening on my machine. To help me further diagnose, would you mind posting the output of:
docker ps --format "table {{.Names}}\t{{.Ports}}" # May require sudoand
ss -tulpn | grep LISTEN # May require sudo?
It would also help if you would walk me through how you've determined that Mathesar is listening on port 80? If you already have another service consuming that port, your OS should prevent such behavior.
Sorry, you are right docker is listening 8080 and 44343
sudo docker ps --format "table {{.Names}}\t{{.Ports}}"
NAMES PORTS
mathesar_service 3000/tcp, 5432/tcp, 6006/tcp, 8000/tcp, 8080/tcp
mathesar-caddy-reverse-proxy-1 80/tcp, 443/tcp, 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp, 2019/tcp, 443/udp, 0.0.0.0:44343->44343/tcp, :::44343->44343/tcp
mathesar_db 5432/tcp
jerome.perez@db:~$ sudo ss -tulpn | grep LISTEN
tcp LISTEN 0 128 0.0.0.0:22 0.0.0.0:* users:(("sshd",pid=617,fd=3))
tcp LISTEN 0 244 0.0.0.0:5432 0.0.0.0:* users:(("postgres",pid=14207,fd=5))
tcp LISTEN 0 4096 0.0.0.0:8080 0.0.0.0:* users:(("docker-proxy",pid=167609,fd=4))
tcp LISTEN 0 20 127.0.0.1:25 0.0.0.0:* users:(("exim4",pid=1856,fd=4))
tcp LISTEN 0 4096 0.0.0.0:44343 0.0.0.0:* users:(("docker-proxy",pid=167588,fd=4))
tcp LISTEN 0 128 127.0.0.1:631 0.0.0.0:* users:(("cupsd",pid=149934,fd=7))
tcp LISTEN 0 128 [::]:22 [::]:* users:(("sshd",pid=617,fd=4))
tcp LISTEN 0 511 *:80 *:* users:(("apache2",pid=149871,fd=4),("apache2",pid=149870,fd=4),("apache2",pid=149869,fd=4),("apache2",pid=633,fd=4))
tcp LISTEN 0 244 [::]:5432 [::]:* users:(("postgres",pid=14207,fd=6))
tcp LISTEN 0 4096 [::]:8080 [::]:* users:(("docker-proxy",pid=167616,fd=4))
tcp LISTEN 0 20 [::1]:25 [::]:* users:(("exim4",pid=1856,fd=5))
tcp LISTEN 0 128 [::1]:631 [::]:* users:(("cupsd",pid=149934,fd=6))
tcp LISTEN 0 4096 [::]:44343 [::]:* users:(("docker-proxy",pid=167595,fd=4))@jeromeperez The output you have from docker ps doesn't line up with the docker-compose.yml you posted. From that docker-compose.yml, it should look like this:
mathesar_service 3000/tcp, 5432/tcp, 6006/tcp, 8000/tcp
mathesar-caddy-reverse-proxy-1 443/udp, 2019/tcp, 0.0.0.0:8080->80/tcp, :::8080->80/tcp, 0.0.0.0:44343->443/tcp, :::44343->443/tcp
mathesar_db 5432/tcpNote the differences in the mathesar-caddy-reverse-proxy-1 line.
As a next step, I'd like you to run:
docker rm -f mathesar_service
docker rm -f mathesar-caddy-reverse-proxy-1
docker rm -f mathesar_db
docker network rm mathesar_defaultThen please verify that the docker-compose.yml file you're using is the correct one (i.e., the one with ports set up like the one you posted), and run the command to bring up the services. From the directory with the docker-compose.yml file, this would be something like docker compose -f docker-compose.yml up -d.
Please let me know if that works for you.
jeromeperez
commented
6 months ago @jeromeperez The output you have from
docker psdoesn't line up with thedocker-compose.ymlyou posted. From thatdocker-compose.yml, it should look like this:mathesar_service 3000/tcp, 5432/tcp, 6006/tcp, 8000/tcp mathesar-caddy-reverse-proxy-1 443/udp, 2019/tcp, 0.0.0.0:8080->80/tcp, :::8080->80/tcp, 0.0.0.0:44343->443/tcp, :::44343->443/tcp mathesar_db 5432/tcpNote the differences in the
mathesar-caddy-reverse-proxy-1line.As a next step, I'd like you to run:
docker rm -f mathesar_service docker rm -f mathesar-caddy-reverse-proxy-1 docker rm -f mathesar_db docker network rm mathesar_defaultThen please verify that the
docker-compose.ymlfile you're using is the correct one (i.e., the one with ports set up like the one you posted), and run the command to bring up the services. From the directory with thedocker-compose.ymlfile, this would be something likedocker compose -f docker-compose.yml up -d.Please let me know if that works for you.
Sorry it was an old screenshot, it wasn't the good docker-compose.yml
sudo docker ps --format "table {{.Names}}\t{{.Ports}}"
NAMES PORTS
mathesar_service 3000/tcp, 5432/tcp, 6006/tcp, 8000/tcp
mathesar-caddy-reverse-proxy-1 443/udp, 2019/tcp, 0.0.0.0:8080->80/tcp, :::8080->80/tcp, 0.0.0.0:44343->443/tcp, :::44343->443/tcp
mathesar_db 5432/tcp
jerome.perez@db:~$ sudo ss -tulpn | grep LISTEN # May require sudo
tcp LISTEN 0 128 0.0.0.0:22 0.0.0.0:* users:(("sshd",pid=617,fd=3))
tcp LISTEN 0 244 0.0.0.0:5432 0.0.0.0:* users:(("postgres",pid=14207,fd=5))
tcp LISTEN 0 4096 0.0.0.0:8080 0.0.0.0:* users:(("docker-proxy",pid=185880,fd=4))
tcp LISTEN 0 20 127.0.0.1:25 0.0.0.0:* users:(("exim4",pid=1856,fd=4))
tcp LISTEN 0 4096 0.0.0.0:44343 0.0.0.0:* users:(("docker-proxy",pid=185860,fd=4))
tcp LISTEN 0 128 127.0.0.1:631 0.0.0.0:* users:(("cupsd",pid=288819,fd=7))
tcp LISTEN 0 128 [::]:22 [::]:* users:(("sshd",pid=617,fd=4))
tcp LISTEN 0 511 *:80 *:* users:(("apache2",pid=288757,fd=4),("apache2",pid=288756,fd=4),("apache2",pid=288755,fd=4),("apache2",pid=633,fd=4))
tcp LISTEN 0 244 [::]:5432 [::]:* users:(("postgres",pid=14207,fd=6))
tcp LISTEN 0 4096 [::]:8080 [::]:* users:(("docker-proxy",pid=185888,fd=4))
tcp LISTEN 0 20 [::1]:25 [::]:* users:(("exim4",pid=1856,fd=5))
tcp LISTEN 0 128 [::1]:631 [::]:* users:(("cupsd",pid=288819,fd=6))
tcp LISTEN 0 4096 [::]:44343 [::]:* users:(("docker-proxy",pid=185867,fd=4)) Now it's OK thanks
Hi
Description
i have two web applications installed on my server, the first listen on port 80 and mathesar should listen port 8080 but mathesar listen port 80 too
Expected behavior
I want Mathesar listening on ports 8080 and/or 44343 for https
To Reproduce
Environment
Additional context
hostnamectl Static hostname: myserverdomain Icon name: computer-vm Chassis: vm 🖴 Machine ID: f9ed6d02fc5043519f856f29a61085a3 Boot ID: cbe9ea72e4d7434985758483c71a35ee Virtualization: kvm Operating System: Debian GNU/Linux 12 (bookworm) Kernel: Linux 6.1.0-18-amd64 Architecture: x86-64 Hardware Vendor: QEMU Hardware Model: Standard PC _i440FX + PIIX, 1996_ Firmware Version: rel-1.16.1-0-g3208b098f51a-prebuilt.qemu.orgDocker-compose.yuml
it the same with expose: -"8080"
It's the same with 8080:8080 44343:44343 and when i put