search

mathesar-foundation / mathesar

Web application providing an intuitive user experience to databases.
https://mathesar.org/
GNU General Public License v3.0
2.32k stars 320 forks source link

Impossible to install mathesar #3444

Closed jeromeperez closed 6 months ago

jeromeperez commented 6 months ago

Description

Impossible to install last release

Expected behavior

an installed and usable application mathesar

To Reproduce

sudo docker compose -f /mnt/data

Environment

Additional context

sudo docker compose -f docker-compose.yml up [+] Running 3/0 ✔ Container mathesar_db Created 0.0s ✔ Container mathesar_service Created 0.0s ✔ Container mathesar-caddy-reverse-proxy-1 Created 0.0s Attaching to caddy-reverse-proxy-1, mathesar_db, mathesar_service mathesar_db | mathesar_db | PostgreSQL Database directory appears to contain a database; Skipping initialization mathesar_db | caddy-reverse-proxy-1 | {"level":"info","ts":1707832850.6093478,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"} caddy-reverse-proxy-1 | {"level":"info","ts":1707832850.6108546,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]} caddy-reverse-proxy-1 | {"level":"info","ts":1707832850.6109264,"logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443} caddy-reverse-proxy-1 | {"level":"info","ts":1707832850.610935,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"} caddy-reverse-proxy-1 | {"level":"info","ts":1707832850.61099,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc000172700"} mathesar_db | 2024-02-13 14:00:50.615 UTC [1] LOG: starting PostgreSQL 13.14 (Debian 13.14-1.pgdg120+2) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit mathesar_db | 2024-02-13 14:00:50.615 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432 mathesar_db | 2024-02-13 14:00:50.615 UTC [1] LOG: listening on IPv6 address "::", port 5432 caddy-reverse-proxy-1 | {"level":"info","ts":1707832850.6193373,"logger":"pki.ca.local","msg":"root certificate is already trusted by system","path":"storage:pki/authorities/local/root.crt"} caddy-reverse-proxy-1 | {"level":"info","ts":1707832850.6194456,"logger":"http","msg":"enabling HTTP/3 listener","addr":":443"} caddy-reverse-proxy-1 | {"level":"info","ts":1707832850.6197035,"msg":"failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/quic-go/quic-go/wiki/UDP-Buffer-Sizes for details."} caddy-reverse-proxy-1 | {"level":"info","ts":1707832850.619911,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]} caddy-reverse-proxy-1 | {"level":"info","ts":1707832850.6199572,"logger":"http.log","msg":"server running","name":"remaining_auto_https_redirects","protocols":["h1","h2","h3"]} caddy-reverse-proxy-1 | {"level":"info","ts":1707832850.6199656,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["MyIP"]} caddy-reverse-proxy-1 | {"level":"warn","ts":1707832850.6204915,"logger":"tls","msg":"stapling OCSP","error":"no OCSP stapling for [MyIP]: no OCSP server specified in certificate","identifiers":["MyIP"]} caddy-reverse-proxy-1 | {"level":"info","ts":1707832850.6206715,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"} caddy-reverse-proxy-1 | {"level":"info","ts":1707832850.6206858,"msg":"serving initial configuration"} caddy-reverse-proxy-1 | {"level":"warn","ts":1707832850.6426978,"logger":"tls","msg":"storage cleaning happened too recently; skipping for now","storage":"FileStorage:/data/caddy","instance":"ad48cdae-b75e-4fd5-b495-4a5a0e56b3e2","try_again":1707919250.6426942,"try_again_in":86399.999999209} caddy-reverse-proxy-1 | {"level":"info","ts":1707832850.6428475,"logger":"tls","msg":"finished cleaning storage units"} mathesar_db | 2024-02-13 14:00:50.673 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" mathesar_db | 2024-02-13 14:00:50.736 UTC [27] LOG: database system was shut down at 2024-02-13 13:56:54 UTC mathesar_db | 2024-02-13 14:00:50.771 UTC [1] LOG: database system is ready to accept connections mathesar_service | /usr/local/bin/python: Error while finding module specification for 'mathesar.install' (ModuleNotFoundError: No module named 'mathesar') mathesar_service exited with code 1

seancolsen commented 6 months ago

Thanks for reporting this, @jeromeperez. We'll have someone look into this ASAP.

Anish9901 commented 6 months ago

Hey @jeromeperez! I'd like to know a little more about what your setup for installing Mathesar looks like,

jeromeperez commented 6 months ago

Hey @jeromeperez! I'd like to know a little more about what your setup for installing Mathesar looks like,

  • Are you installing Mathesar on GCP/AWS? No i'am installing on local lan server
  • I notice that you are trying to set up a domain that is hidden by "MyIP", is this IP pointed to by a domain/subdomain on a DNS? It's the Ip of my server and it was working on another local lan server with 0.1.3 version using .env and the same version upgraded to 0.1.4 version
  • Are there any other settings except the SECRET_KEY & DOMAIN_NAME that you have altered in the docker-compose.yml file? DB settings 
    
x-config: &config
(REQUIRED) Replace '?' with '-' followed by a 50 character random string.
You can generate one at https://djecrety.ir/ or by running:
echo $(cat /dev/urandom | LC_CTYPE=C tr -dc 'a-zA-Z0-9' | head -c 50)
SECRET_KEY: ${SECRET_KEY:-SECRETKEY}

(Optional) Replace 'http://localhost' with custom domain(s) e.g. 'yourdomain.com, 127.0.0.1' to manage the host(s) at which you want to access Mathesar over http or https DOMAIN_NAME: ${DOMAIN_NAME:-162.38.X.X, Data-X-X, 127.0.0.1}

Edit the POSTGRES_* variables if you are not using the db service provided below, or if you want to use a custom database user.

(Optional) Replace 'mathesar_django' with any custom name for the internal database managed by mathesar web-service POSTGRES_DB: ${POSTGRES_DB:-mathesar_django}

(Optional) Replace 'mathesar' with any custom username for the aforementioned database POSTGRES_USER: ${POSTGRES_USER:-mathesar}

(Optional) Replace 'mathesar' with any custom password for the aforementioned database

POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-PASSWORD} (Optional) Replace 'mathesar_db' with the name of the host running postgres POSTGRES_HOST: ${POSTGRES_HOST:-162.38.X.X}

(Optional) Replace '5432' with the port on which postgres is running POSTGRES_PORT: ${POSTGRES_PORT:-5432}



If i put only IP in domain_name it's the same
Anish9901 commented 6 months ago

If i put only IP in domain_name it's the same

That's true @jeromeperez, however you'd have to specify http:// before those IP(s) as the reverse proxy will try to serve any traffic for those IPs over https hence making the service inaccessible as SSL certificates for those IP(s) couldn't be obtained. Once you have made the change mathesar should then be available at http://162.38.X.X

I am also noticing the following error from your error logs:

mathesar_service | /usr/local/bin/python: Error while finding module specification for 'mathesar.install' (ModuleNotFoundError: No module named 'mathesar')
mathesar_service exited with code 1

Are you trying to mount any volumes to the service? specifically to the /code/ directory that lives inside the mathesar_service container?

jeromeperez commented 6 months ago

If i put only IP in domain_name it's the same

That's true @jeromeperez, however you'd have to specify http:// before those IP(s) as the reverse proxy will try to serve any traffic for those IPs over https hence making the service inaccessible as SSL certificates for those IP(s) couldn't be obtained. Once you have made the change mathesar should then be available at http://162.38.X.X

I am also noticing the following error from your error logs:

mathesar_service | /usr/local/bin/python: Error while finding module specification for 'mathesar.install' (ModuleNotFoundError: No module named 'mathesar')
mathesar_service exited with code 1

Are you trying to mount any volumes to the service? specifically to the /code/ directory that lives inside the mathesar_service container? I don't know how to do that i added http:// before the IP but mathesar is unreachable (i use a vpn because i am out of office) today but i have the same error 


sudo docker compose -f docker-compose.yml up
[sudo] Mot de passe de jerome.perez : 
[+] Running 3/0
✔ Container mathesar_db                     Created                                                                                                                                                                                  0.0s 
✔ Container mathesar-caddy-reverse-proxy-1  Created                                                                                                                                                                                  0.0s 
✔ Container mathesar_service                Created                                                                                                                                                                                  0.0s 
Attaching to caddy-reverse-proxy-1, mathesar_db, mathesar_service
mathesar_db            | 
mathesar_db            | PostgreSQL Database directory appears to contain a database; Skipping initialization
mathesar_db            | 
caddy-reverse-proxy-1  | {"level":"info","ts":1707918948.1273003,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
caddy-reverse-proxy-1  | {"level":"info","ts":1707918948.1282218,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
caddy-reverse-proxy-1  | {"level":"warn","ts":1707918948.1282744,"logger":"http.auto_https","msg":"server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server","server_name":"srv0","http_port":80}
caddy-reverse-proxy-1  | {"level":"info","ts":1707918948.1283703,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc00071f080"}
caddy-reverse-proxy-1  | {"level":"info","ts":1707918948.1285214,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
caddy-reverse-proxy-1  | {"level":"info","ts":1707918948.1285973,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
caddy-reverse-proxy-1  | {"level":"info","ts":1707918948.1285994,"msg":"serving initial configuration"}
caddy-reverse-proxy-1  | {"level":"warn","ts":1707918948.3772845,"logger":"tls","msg":"storage cleaning happened too recently; skipping for now","storage":"FileStorage:/data/caddy","instance":"f60e6e50-f47d-4c78-ac78-c9ab1f66b270","try_again":1708005348.377282,"try_again_in":86399.999999303}
caddy-reverse-proxy-1  | {"level":"info","ts":1707918948.377629,"logger":"tls","msg":"finished cleaning storage units"}
mathesar_db            | 2024-02-14 13:55:48.446 UTC [1] LOG:  starting PostgreSQL 13.14 (Debian 13.14-1.pgdg120+2) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
mathesar_db            | 2024-02-14 13:55:48.446 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
mathesar_db            | 2024-02-14 13:55:48.446 UTC [1] LOG:  listening on IPv6 address "::", port 5432
mathesar_db            | 2024-02-14 13:55:48.491 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
mathesar_db            | 2024-02-14 13:55:48.555 UTC [27] LOG:  database system was shut down at 2024-02-14 13:54:54 UTC
mathesar_db            | 2024-02-14 13:55:48.589 UTC [1] LOG:  database system is ready to accept connections
mathesar_service       | /usr/local/bin/python: Error while finding module specification for 'mathesar.install' (ModuleNotFoundError: No module named 'mathesar')
mathesar_service exited with code 1
pavish commented 6 months ago

Hi @jeromeperez, I'd like to ask a few questions to better understand your environment.

If you'd like more assistance, you could schedule a meeting with us where we can debug and fix the issue for you, over the call.

jeromeperez commented 6 months ago

Hi @jeromeperez, I'd like to ask a few questions to better understand your environment.

  • Are you performing a fresh installation of Mathesar? If yes, did you follow our documentation and face any errors during the process? Could you try it again in a different installation directory and provide us a detailed log of the errors?

it's exactly what i did and the error 1 was the only error on my first docker-compose -f up

  • If no, I assume you were running 0.1.3 on one of your servers and it automatically upgraded to 0.1.4, and you did not manually upgrade Mathesar. Is this correct?

  • If this error happened after the upgrade, it's because our 0.1.4 version has some breaking changes and requires manual intervention. Our instructions to upgrading are present here.

    • Could you follow the instructions and let us know if you hit any roadblocks? (It should be okay even if your existing setup shows the version as upgraded to 0.1.4, the instructions would still work on top of it).
  • If it's not any of the above assumptions, could you explain the details of the steps you took in more detail that resulted in this error?

If you'd like more assistance, you could schedule a meeting with us where we can debug and fix the issue for you, over the call.

i will try to delete all and reinstall completely tomorow

pavish commented 6 months ago

i will try to delete all and reinstall completely tomorow

Thanks @jeromeperez. Let us know how it goes.

I would suggest the following checklist while re-attempting:

jeromeperez commented 6 months ago

i will try to delete all and reinstall completely tomorow

Thanks @jeromeperez. Let us know how it goes.

I would suggest the following checklist while re-attempting:

  • Remove existing Mathesar containers, images, and volumes in your docker instance.

    • sudo docker compose -f /etc/mathesar/docker-compose.yml down -v --rmi all
      sudo docker compose -f /mnt/data/mathesar/docker-compose.yml down -v --rmi all
[+] Running 7/7
✔ Container mathesar-caddy-reverse-proxy-1  Removed                                                                                                                                                                                  0.0s 
✔ Container mathesar_service                Removed                                                                                                                                                                                  0.0s 
✔ Container mathesar_db                     Removed                                                                                                                                                                                  0.0s 
✔ Image postgres:13                         Removed                                                                                                                                                                                  0.1s 
! Image mathesar/mathesar-caddy:latest      Resource is still in use                                                                                                                                                                 0.0s 
! Image mathesar/mathesar-prod:latest       Resource is still in use                                                                                                                                                                 0.0s 
✔ Network mathesar_default                  Removed
  • Delete all existing Mathesar related files on your server i.e the entire installation directory (/etc/mathesar). sudo rm -r /mnt/data/mathesar

i completely uninstall/reinstall docker engine and docker compose

  • Attempt installation again following the docs. 
    jerome.perez@Data-X-X:/mnt/data$ docker --version
Docker version 25.0.3, build 4debf41
jerome.perez@Data-X-X:/mnt/data$ docker compose version
Docker Compose version v2.24.5
jerome.perez@Data-X-X:/mnt/data
    drwxr-xr-x 2 jerome.perez jerome.perez  4096 14 févr. 18:21 mathesar
    
jerome.perez@Data-X-X:/mnt/data/mathesar$ wget https://github.com/mathesar-foundation/mathesar/raw/0.1.4/docker-compose.yml
--2024-02-14 18:24:33--  https://github.com/mathesar-foundation/mathesar/raw/0.1.4/docker-compose.yml
Résolution de github.com (github.com)… 140.82.121.3
Connexion à github.com (github.com)|140.82.121.3|:443… connecté.
requête HTTP transmise, en attente de la réponse… 302 Found
Emplacement : https://raw.githubusercontent.com/mathesar-foundation/mathesar/0.1.4/docker-compose.yml [suivant]
--2024-02-14 18:24:33--  https://raw.githubusercontent.com/mathesar-foundation/mathesar/0.1.4/docker-compose.yml
Résolution de raw.githubusercontent.com (raw.githubusercontent.com)… 185.199.110.133, 185.199.108.133, 185.199.111.133, ...
Connexion à raw.githubusercontent.com (raw.githubusercontent.com)|185.199.110.133|:443… connecté.
requête HTTP transmise, en attente de la réponse… 200 OK
Taille : 8876 (8,7K) [text/plain]
Sauvegarde en : « docker-compose.yml »

docker-compose.yml 100%[=======================================================================================================================================>] 8,67K --.-KB/s ds 0s

2024-02-14 18:24:34 (61,2 MB/s) — « docker-compose.yml » sauvegardé [8876/8876]

CONFIG

#

Customize your Mathesar installation with the following variables.

See https://docs.mathesar.org/configuration/env-variables/ for more info.

# x-config: &config

(REQUIRED) Replace '?' with '-' followed by a 50 character random string.

You can generate one at https://djecrety.ir/ or by running:

echo $(cat /dev/urandom | LC_CTYPE=C tr -dc 'a-zA-Z0-9' | head -c 50)

SECRET_KEY: ${SECRET_KEY:-XXXXXXXX}

(Optional) Replace 'http://localhost' with custom domain(s) e.g.

'yourdomain.com, 127.0.0.1' to manage the host(s) at which you want to

access Mathesar over http or https

DOMAIN_NAME: ${DOMAIN_NAME:-http://162.38.X.X, http://127.0.0.1, http://Data-X-X}

Edit the POSTGRES_* variables if you are not using the db service provided

below, or if you want to use a custom database user.

(Optional) Replace 'mathesar_django' with any custom name for the internal

database managed by mathesar web-service

POSTGRES_DB: ${POSTGRES_DB:-mathesar_django}

(Optional) Replace 'mathesar' with any custom username for the

aforementioned database

POSTGRES_USER: ${POSTGRES_USER:-mathesar}

(Optional) Replace 'mathesar' with any custom password for the

aforementioned database

POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-XXXX}

(Optional) Replace 'mathesar_db' with the name of the host running postgres

POSTGRES_HOST: ${POSTGRES_HOST:-162.38.X.X}

(Optional) Replace '5432' with the port on which postgres is running

POSTGRES_PORT: ${POSTGRES_PORT:-5432}

jerome.perez@Data-X-X:/mnt/data/mathesar$ sudo docker compose -f docker-compose.yml up -d [sudo] Mot de passe de jerome.perez : [+] Running 44/44 ✔ db 14 layers [⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿] 0B/0B Pulled 6.7s ✔ e1caac4eb9d2 Pull complete 1.0s ✔ d6b2187f2808 Pull complete 0.5s ✔ 9f4bcfb39fe9 Pull complete 0.7s ✔ b0d613ab0b76 Pull complete 1.3s ✔ 5770ebef9ba7 Pull complete 1.3s ✔ c7632123849d Pull complete 1.5s ✔ e5fd362437c3 Pull complete 1.7s ✔ b8c9698a102e Pull complete 1.7s ✔ c0b1c5373a19 Pull complete 3.9s ✔ d27fc7380fa4 Pull complete 2.1s ✔ 615a0e0c38c6 Pull complete 2.1s ✔ 74525c365b65 Pull complete 3.6s ✔ 50e2001ce9b1 Pull complete 2.5s ✔ 650f55cfd6b4 Pull complete 2.9s ✔ caddy-reverse-proxy 7 layers [⣿⣿⣿⣿⣿⣿⣿] 0B/0B Pulled 27.7s ✔ 4fc28d14f482 Pull complete 23.8s ✔ 8e11250dc7a3 Pull complete 24.8s ✔ 5f93e4db8804 Pull complete 24.4s ✔ bd52dee18582 Pull complete 24.5s ✔ 1e001341c28d Pull complete 25.2s ✔ 555287074b18 Pull complete 25.9s ✔ bc70cbbcfad6 Pull complete 25.2s ✔ service 20 layers [⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿] 0B/0B Pulled 32.1s ✔ ac8bb7e1a323 Pull complete 5.2s ✔ 3b1c264c0ad4 Pull complete 4.5s ✔ b1e7e053c9f6 Pull complete 5.7s ✔ a2e1e233599c Pull complete 8.2s ✔ 0ebfe287e976 Pull complete 5.9s ✔ e4e5f0594035 Pull complete 16.6s ✔ 508cf9d1df67 Pull complete 16.2s ✔ b0eda9c97fd4 Pull complete 16.3s ✔ 4f4fb700ef54 Pull complete 16.6s ✔ 3c492f5392f8 Pull complete 17.3s ✔ 144cf691b742 Pull complete 17.0s ✔ 4fd2f32ead5a Pull complete 17.0s ✔ f9f3264c77ed Pull complete 18.8s ✔ 7dce729c787c Pull complete 17.6s ✔ 92676faddf48 Pull complete 18.9s ✔ 2cee1e4d1bc5 Pull complete 18.7s ✔ 40bc6eda97fc Pull complete 19.4s ✔ d7304d61c860 Pull complete 22.9s ✔ 9bfb46be6803 Pull complete 24.0s ✔ a4de657cfab2 Pull complete 22.4s [+] Running 3/4 ⠧ Network mathesar_default Created 6.7s ✔ Container mathesar_db Healthy 6.4s ✔ Container mathesar-caddy-reverse-proxy-1 Started 0.9s ✔ Container mathesar_service Started


I will try again tomorow on the same lan because site is unrechable via VPN

> * Keep detailed notes of every step and/or if possible, make a screen recording. This will help us identify the issue quickly and reduce multiple to-fro conversations.
pavish commented 6 months ago

@jeromeperez

It looks like the service started. Please note that it takes a few minutes for the service to fully start and reach healthy status, and the web frontend will only be available once the service is healthy. The first startup after installation could take somewhere between ~2 to 8 minutes before the service is healthy and the web frontend to be ready.

Let us know if you're able to connect to it on the same lan.

If yes, we can narrow the problem to be making it reachable via VPN. If no, we might have to dig into the logs of the mathesar_service container.

jeromeperez commented 6 months ago

@jeromeperez

It looks like the service started. Please note that it takes a few minutes for the service to fully start and reach healthy status, and the web frontend will only be available once the service is healthy. The first startup after installation could take somewhere between ~2 to 8 minutes before the service is healthy and the web frontend to be ready.

Let us know if you're able to connect to it on the same lan.

If yes, we can narrow the problem to be making it reachable via VPN. If no, we might have to dig into the logs of the mathesar_service container.

Everything is ok now on the same lan and i can connect throw VPN too ;-) Thanks