search

naturalsolutions / ecoSecrets

ecoSecrets is a web application which enables users to manage their camera traps data
https://demo.ecosecrets.natural-solutions.eu/
GNU General Public License v3.0
18 stars 5 forks source link

Problème test installation #44

Open PNPyrenees opened 1 year ago

PNPyrenees commented 1 year ago

Bonjour,

Par curiosité, je voulais me lancer dans une première installation de GeoCam sur une VM locale (ubuntu server 22.04). Je rencontre cependant une erreur au moment d'execution de ./scripts/docker.sh up -d

$ sudo ./scripts/docker.sh up -d
[+] Running 7/7
 ✔ Container geonature-annotation-minio-1     Healthy                                                0.4s 
 ✔ Container geonature-annotation-db-1        Healthy                                                0.3s 
 ✔ Container geonature-annotation-doc-1       Running                                                0.0s 
 ✔ Container geonature-annotation-keycloak-1  Healthy                                                0.4s 
 ✘ Container geonature-annotation-api-1       Error                                                  0.2s 
 ✔ Container geonature-annotation-frontend-1  Recreat...                                             0.0s 
 ✔ Container geonature-annotation-traefik-1   Recreate...                                            0.0s 
dependency failed to start: container geonature-annotation-api-1 is unhealthy

Et voici ce que me donne les logs

$ sudo docker logs geonature-annotation-api-1
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO:     Will watch for changes in these directories: ['/home/app']
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [1] using StatReload
Process SpawnProcess-1:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/local/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/.venv/lib/python3.8/site-packages/uvicorn/_subprocess.py", line 76, in subprocess_started
    target(sockets=sockets)
  File "/.venv/lib/python3.8/site-packages/uvicorn/server.py", line 61, in run
    return asyncio.run(self.serve(sockets=sockets))
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/.venv/lib/python3.8/site-packages/uvicorn/server.py", line 68, in serve
    config.load()
  File "/.venv/lib/python3.8/site-packages/uvicorn/config.py", line 467, in load
    self.loaded_app = import_from_string(self.app)
  File "/.venv/lib/python3.8/site-packages/uvicorn/importer.py", line 21, in import_from_string
    module = importlib.import_module(module_str)
  File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/app/src/main.py", line 6, in <module>
    from src.keycloak.idp import idp
  File "/home/app/src/keycloak/idp.py", line 4, in <module>
    idp = Keycloak(
  File "/home/app/src/keycloak/base.py", line 10, in __init__
    super().__init__(**kwargs)
  File "/.venv/lib/python3.8/site-packages/fastapi_keycloak/api.py", line 162, in __init__
    self._get_admin_token()  # Requests an admin access token on startup
  File "/.venv/lib/python3.8/site-packages/fastapi_keycloak/api.py", line 346, in _get_admin_token
    response = requests.post(url=self.token_uri, headers=headers, data=data, timeout=self.timeout)
  File "/.venv/lib/python3.8/site-packages/requests/api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
  File "/.venv/lib/python3.8/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/.venv/lib/python3.8/site-packages/requests/sessions.py", line 575, in request
    prep = self.prepare_request(req)
  File "/.venv/lib/python3.8/site-packages/requests/sessions.py", line 486, in prepare_request
    p.prepare(
  File "/.venv/lib/python3.8/site-packages/requests/models.py", line 368, in prepare
    self.prepare_url(url, params)
  File "/.venv/lib/python3.8/site-packages/requests/models.py", line 439, in prepare_url
    raise MissingSchema(
requests.exceptions.MissingSchema: Invalid URL 'None': No scheme supplied. Perhaps you meant https://None?

Pour info, je n'ai modifié que les paramètres suivants dans le .env:

N'hésitez pas si besoin d'éléments complémentaires. En vous remerciant par avance.

geobrun commented 1 year ago

Bonjour,

J'avais rencontré pas mal de problème il y a quelques mois lorsque j'avais réalisé une installation sur notre réseau local. Mais avec l'aide de Natural Solutions, j'avais fini par obtenir une instance fonctionnelle. Tu peux jeter un œil aux problèmes que j'ai rencontrés et aux solutions qui ont été apportées ici : #8

Pour info, Natural Solutions est train de finaliser une nouvelle version intégrant la gestion des vidéos pour notre compte (Pnr de Millevaches en Limousin). Elle devrait sortir sous peu avec une amélioration du processus d'installation. Je te conseillerai d'attendre cette nouvelle version si c'est possible pour toi ! :)

geobrun commented 11 months ago

Bonjour @PNPyrenees,

Si tu souhaites toujours tester GeoCam, la nouvelle version est prête. Il y a encore quelques petits détails à régler au niveau de l'installation pour que le processus soit parfaitement fluide, mais rien de bien méchant. Si tu es intéressé, n'hésite pas à me le dire : je devrais pouvoir te guider pour l'installation.

MathildeNS commented 11 months ago

Bonjour à tous,

Merci beaucoup @geobrun pour vos réponses !

Pour l'installation je vous conseille toutefois d'attendre le début d'année. En effet, @ophdlv réaliser la semaine prochaine les modifications concernant docker compose et la version de MinIo notamment afin de faciliter l'installation.

PNPyrenees commented 10 months ago

Merci pour vos retours,

J'attends votre feu vert pour me lancer dans un nouveau test d'installation et vous partager le retour d'expérience,

Bien à vous,

MathildeNS commented 10 months ago

Bonjour,

Vous pouvez tester l'installation, nous avons mergés les développements ce weekend. Ils sont disponibles dans la dernière release 0.1.2

PNPyrenees commented 10 months ago

Merci pour l'information ! Je vous tiendrez informé dès que j'aurais pu y remettre les main dedans,

PNPyrenees commented 7 months ago

Bonjour,

Le temps me permettant de revenir sur le sujet je voulais vous partager mes difficultés d'installation actuelle.

Je suis parti de la version 0.1.2 et voici les retours

$ sudo ./scripts/docker.sh up -d
WARN[0000] /home/ecosecrets/ecosecrets/docker/docker-compose.yml: `version` is obsolete 
WARN[0000] /home/ecosecrets/ecosecrets/docker/docker-compose.override.yml: `version` is obsolete 
[+] Running 6/12
 ⠙ Network geonature-annotation_default        Created                                                                                110.1s 
 ⠙ Network geonature-annotation_annotation_nw  Created                                                                                110.0s 
 ⠋ Volume "geonature-annotation_db_data"       Created                                                                                110.0s 
 ⠋ Volume "geonature-annotation_minio_data"    Created                                                                                110.0s 
 ✔ Container geonature-annotation-minio-1      Healthy                                                                                 48.1s 
 ✔ Container geonature-annotation-doc-1        Started                                                                                 17.8s 
 ✔ Container geonature-annotation-taxapi-1     Started                                                                                 17.9s 
 ✔ Container geonature-annotation-db-1         Healthy                                                                                 28.3s 
 ✔ Container geonature-annotation-keycloak-1   Healthy                                                                                 69.1s 
 ✘ Container geonature-annotation-api-1        Error                                                                                  109.9s 
 ⠏ Container geonature-annotation-frontend-1   Created                                                                                109.9s 
 ⠏ Container geonature-annotation-traefik-1    Created                                                                                 92.9s 
dependency failed to start: container geonature-annotation-api-1 is unhealthy

Et les logs :

$ sudo ./scripts/docker.sh logs
taxapi-1  | 
taxapi-1  | > taxapi@1.0.0 start
taxapi-1  | > export NODE_OPTIONS=--max-old-space-size=4096 && node build/index.js
taxapi-1  | 
taxapi-1  | Downloading taxref.txt
taxapi-1  | Server Up on localhost:5666
taxapi-1  | Downloaded taxref.txt, unzipping
taxapi-1  | Extracting TAXREFv16.txt
taxapi-1  | Converting to JSON...
taxapi-1  | Ready
minio-1   | Formatting 1st pool, 1 set(s), 1 drives per set.
minio-1   | WARNING: Host local has more than 0 drives of set. A host failure will result in data becoming unavailable.
minio-1   | MinIO Object Storage Server
minio-1   | Copyright: 2015-2023 MinIO, Inc.
minio-1   | License: GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>
minio-1   | Version: RELEASE.2023-08-23T10-07-06Z (go1.19.12 linux/amd64)
minio-1   | 
minio-1   | Status:         1 Online, 0 Offline. 
minio-1   | S3-API: http://172.25.0.2:9000  http://127.0.0.1:9000     
minio-1   | Console: http://172.25.0.2:9001 http://127.0.0.1:9001   
minio-1   | 
minio-1   | Documentation: https://min.io/docs/minio/linux/index.html
minio-1   | Warning: The standard parity is set to 0. This can lead to data loss.
minio-1   | 
minio-1   |  You are running an older version of MinIO released 7 months before the latest release 
minio-1   |  Update: Run `mc admin update` 
minio-1   | 
minio-1   | 
db-1      | The files belonging to this database system will be owned by user "postgres".
db-1      | This user must also own the server process.
db-1      | 
db-1      | The database cluster will be initialized with locale "en_US.utf8".
db-1      | The default database encoding has accordingly been set to "UTF8".
db-1      | The default text search configuration will be set to "english".
db-1      | 
db-1      | Data page checksums are disabled.
db-1      | 
db-1      | fixing permissions on existing directory /var/lib/postgresql/data ... ok
db-1      | creating subdirectories ... ok
db-1      | selecting dynamic shared memory implementation ... posix
db-1      | selecting default max_connections ... 100
db-1      | selecting default shared_buffers ... 128MB
db-1      | selecting default time zone ... UTC
db-1      | creating configuration files ... ok
db-1      | running bootstrap script ... ok
db-1      | sh: locale: not found
db-1      | 2024-04-08 09:18:59.407 UTC [29] WARNING:  no usable system locales were found
db-1      | performing post-bootstrap initialization ... ok
db-1      | syncing data to disk ... ok
db-1      | 
db-1      | 
db-1      | Success. You can now start the database server using:
db-1      | 
db-1      |     pg_ctl -D /var/lib/postgresql/data -l logfile start
db-1      | 
db-1      | initdb: warning: enabling "trust" authentication for local connections
db-1      | You can change this by editing pg_hba.conf or using the option -A, or
db-1      | --auth-local and --auth-host, the next time you run initdb.
db-1      | waiting for server to start....2024-04-08 09:19:00.916 UTC [35] LOG:  starting PostgreSQL 14.11 on x86_64-pc-linux-musl, compiled by gcc (Alpine 13.2.1_git20231014) 13.2.1 20231014, 64-bit
db-1      | 2024-04-08 09:19:00.917 UTC [35] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db-1      | 2024-04-08 09:19:00.921 UTC [36] LOG:  database system was shut down at 2024-04-08 09:19:00 UTC
db-1      | 2024-04-08 09:19:00.925 UTC [35] LOG:  database system is ready to accept connections
db-1      |  done
db-1      | server started
db-1      | CREATE DATABASE
db-1      | 
db-1      | 
db-1      | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
db-1      | 
db-1      | 2024-04-08 09:19:01.150 UTC [35] LOG:  received fast shutdown request
db-1      | waiting for server to shut down...2024-04-08 09:19:01.150 UTC [35] LOG:  aborting any active transactions
db-1      | 2024-04-08 09:19:01.158 UTC [35] LOG:  background worker "logical replication launcher" (PID 42) exited with exit code 1
db-1      | .2024-04-08 09:19:01.164 UTC [37] LOG:  shutting down
db-1      | 2024-04-08 09:19:01.184 UTC [35] LOG:  database system is shut down
db-1      |  done
db-1      | server stopped
db-1      | 
db-1      | PostgreSQL init process complete; ready for start up.
db-1      | 
db-1      | 2024-04-08 09:19:01.292 UTC [1] LOG:  starting PostgreSQL 14.11 on x86_64-pc-linux-musl, compiled by gcc (Alpine 13.2.1_git20231014) 13.2.1 20231014, 64-bit
db-1      | 2024-04-08 09:19:01.292 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
db-1      | 2024-04-08 09:19:01.292 UTC [1] LOG:  listening on IPv6 address "::", port 5432
db-1      | 2024-04-08 09:19:01.293 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db-1      | 2024-04-08 09:19:01.297 UTC [50] LOG:  database system was shut down at 2024-04-08 09:19:01 UTC
db-1      | 2024-04-08 09:19:01.302 UTC [1] LOG:  database system is ready to accept connections
db-1      | 2024-04-08 09:19:31.985 UTC [78] ERROR:  relation "migration_model" does not exist at character 25
db-1      | 2024-04-08 09:19:31.985 UTC [78] STATEMENT:  SELECT ID, VERSION FROM MIGRATION_MODEL ORDER BY UPDATE_TIME DESC
db-1      | 2024-04-08 09:19:34.268 UTC [78] ERROR:  relation "public.databasechangelog" does not exist at character 22
db-1      | 2024-04-08 09:19:34.268 UTC [78] STATEMENT:  SELECT COUNT(*) FROM public.databasechangelog
db-1      | 2024-04-08 09:19:35.636 UTC [79] ERROR:  relation "public.databasechangeloglock" does not exist at character 22
db-1      | 2024-04-08 09:19:35.636 UTC [79] STATEMENT:  SELECT COUNT(*) FROM public.databasechangeloglock
db-1      | 2024-04-08 09:19:35.705 UTC [78] ERROR:  relation "public.databasechangelog" does not exist at character 22
db-1      | 2024-04-08 09:19:35.705 UTC [78] STATEMENT:  SELECT COUNT(*) FROM public.databasechangelog
doc-1     | INFO     -  Building documentation...
doc-1     | WARNING  -  Config value: 'dev_addr'. Warning: The use of the IP address '0.0.0.0' suggests a production environment or the use of a proxy to connect to the MkDocs server. However, the MkDocs' server is intended for local development purposes only. Please use a third party production-ready server instead.
doc-1     | INFO     -  Cleaning site directory
doc-1     | INFO     -  Documentation built in 0.55 seconds
doc-1     | INFO     -  [09:19:00] Watching paths for changes: 'docs', 'mkdocs.yml'
doc-1     | INFO     -  [09:19:00] Serving on http://0.0.0.0:8000/
keycloak-1  | Updating the configuration and installing your custom providers, if any. Please wait.
keycloak-1  | 2024-04-08 09:19:13,111 INFO  [org.keycloak.common.Profile] (main) Preview features enabled: scripts
keycloak-1  | 2024-04-08 09:19:25,377 INFO  [io.quarkus.deployment.QuarkusAugmentor] (main) Quarkus augmentation completed in 12889ms
keycloak-1  | 2024-04-08 09:19:26,944 INFO  [org.keycloak.common.Profile] (main) Preview features enabled: scripts
keycloak-1  | 2024-04-08 09:19:27,904 INFO  [org.keycloak.quarkus.runtime.hostname.DefaultHostnameProvider] (main) Hostname settings: Base URL: <unset>, Hostname: <request>, Strict HTTPS: false, Path: <request>, Strict BackChannel: false, Admin URL: <unset>, Admin: <request>, Port: -1, Proxied: true
keycloak-1  | 2024-04-08 09:19:30,172 WARN  [io.quarkus.agroal.runtime.DataSources] (main) Datasource <default> enables XA but transaction recovery is not enabled. Please enable transaction recovery by setting quarkus.transaction-manager.enable-recovery=true, otherwise data may be lost if the application is terminated abruptly
keycloak-1  | 2024-04-08 09:19:31,254 INFO  [org.infinispan.SERVER] (keycloak-cache-init) ISPN005054: Native IOUring transport not available, using NIO instead: io.netty.incubator.channel.uring.IOUring
keycloak-1  | 2024-04-08 09:19:31,327 WARN  [org.infinispan.PERSISTENCE] (keycloak-cache-init) ISPN000554: jboss-marshalling is deprecated and planned for removal
keycloak-1  | 2024-04-08 09:19:31,581 WARN  [org.infinispan.CONFIG] (keycloak-cache-init) ISPN000569: Unable to persist Infinispan internal caches as no global state enabled
keycloak-1  | 2024-04-08 09:19:31,610 INFO  [org.infinispan.CONTAINER] (keycloak-cache-init) ISPN000556: Starting user marshaller 'org.infinispan.jboss.marshalling.core.JBossUserMarshaller'
keycloak-1  | 2024-04-08 09:19:31,937 WARN  [io.quarkus.vertx.http.runtime.VertxHttpRecorder] (main) The X-Forwarded-* and Forwarded headers will be considered when determining the proxy address. This configuration can cause a security issue as clients can forge requests and send a forwarded header that is not overwritten by the proxy. Please consider use one of these headers just to forward the proxy address in requests.
keycloak-1  | 2024-04-08 09:19:35,710 INFO  [org.keycloak.quarkus.runtime.storage.legacy.liquibase.QuarkusJpaUpdaterProvider] (main) Initializing database schema. Using changelog META-INF/jpa-changelog-master.xml
keycloak-1  | 2024-04-08 09:19:38,588 INFO  [org.keycloak.connections.infinispan.DefaultInfinispanConnectionProviderFactory] (main) Node name: node_750940, Site name: null
keycloak-1  | 2024-04-08 09:19:38,796 INFO  [org.keycloak.broker.provider.AbstractIdentityProviderMapper] (main) Registering class org.keycloak.broker.provider.mappersync.ConfigSyncEventListener
keycloak-1  | 2024-04-08 09:19:38,851 INFO  [org.keycloak.services] (main) KC-SERVICES0050: Initializing master realm
keycloak-1  | 2024-04-08 09:19:41,201 INFO  [org.keycloak.exportimport.singlefile.SingleFileImportProvider] (main) Full importing from file /opt/keycloak/bin/../data/import/realm.json
keycloak-1  | 2024-04-08 09:19:43,902 INFO  [org.keycloak.exportimport.util.ImportUtils] (main) Realm 'test-ecosecrets' imported
keycloak-1  | 2024-04-08 09:19:43,942 INFO  [org.keycloak.exportimport.dir.DirImportProvider] (main) Importing from directory /opt/keycloak/bin/../data/import
keycloak-1  | 2024-04-08 09:19:43,942 INFO  [org.keycloak.services] (main) KC-SERVICES0030: Full model import requested. Strategy: IGNORE_EXISTING
keycloak-1  | 2024-04-08 09:19:43,942 INFO  [org.keycloak.services] (main) KC-SERVICES0032: Import finished successfully
keycloak-1  | 2024-04-08 09:19:44,100 INFO  [io.quarkus] (main) Keycloak 21.1.0 on JVM (powered by Quarkus 2.13.7.Final) started in 18.516s. Listening on: http://0.0.0.0:8080
keycloak-1  | 2024-04-08 09:19:44,101 INFO  [io.quarkus] (main) Profile dev activated. 
keycloak-1  | 2024-04-08 09:19:44,101 INFO  [io.quarkus] (main) Installed features: [agroal, cdi, hibernate-orm, jdbc-h2, jdbc-mariadb, jdbc-mssql, jdbc-mysql, jdbc-oracle, jdbc-postgresql, keycloak, logging-gelf, micrometer, narayana-jta, reactive-routes, resteasy, resteasy-jackson, smallrye-context-propagation, smallrye-health, vertx]
keycloak-1  | 2024-04-08 09:19:44,205 INFO  [org.keycloak.services] (main) KC-SERVICES0009: Added user 'ecosecrets' to realm 'master'
keycloak-1  | 2024-04-08 09:19:44,207 WARN  [org.keycloak.quarkus.runtime.KeycloakMain] (main) Running the server in development mode. DO NOT use this configuration in production.
api-1       | INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
api-1       | INFO  [alembic.runtime.migration] Will assume transactional DDL.
api-1       | INFO  [alembic.runtime.migration] Running upgrade  -> 58e367bdda57, init
api-1       | INFO  [alembic.runtime.migration] Running upgrade 58e367bdda57 -> 4cf2ba8715d2, sequence-deployments-projects
api-1       | INFO:     Will watch for changes in these directories: ['/home/app']
api-1       | INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
api-1       | INFO:     Started reloader process [1] using StatReload
api-1       | Process SpawnProcess-1:
api-1       | Traceback (most recent call last):
api-1       |   File "/usr/local/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
api-1       |     self.run()
api-1       |   File "/usr/local/lib/python3.8/multiprocessing/process.py", line 108, in run
api-1       |     self._target(*self._args, **self._kwargs)
api-1       |   File "/.venv/lib/python3.8/site-packages/uvicorn/_subprocess.py", line 76, in subprocess_started
api-1       |     target(sockets=sockets)
api-1       |   File "/.venv/lib/python3.8/site-packages/uvicorn/server.py", line 61, in run
api-1       |     return asyncio.run(self.serve(sockets=sockets))
api-1       |   File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
api-1       |     return loop.run_until_complete(main)
api-1       |   File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
api-1       |     return future.result()
api-1       |   File "/.venv/lib/python3.8/site-packages/uvicorn/server.py", line 68, in serve
api-1       |     config.load()
api-1       |   File "/.venv/lib/python3.8/site-packages/uvicorn/config.py", line 467, in load
api-1       |     self.loaded_app = import_from_string(self.app)
api-1       |   File "/.venv/lib/python3.8/site-packages/uvicorn/importer.py", line 21, in import_from_string
api-1       |     module = importlib.import_module(module_str)
api-1       |   File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
api-1       |     return _bootstrap._gcd_import(name[level:], package, level)
api-1       |   File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
api-1       |   File "<frozen importlib._bootstrap>", line 991, in _find_and_load
api-1       |   File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
api-1       |   File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
api-1       |   File "<frozen importlib._bootstrap_external>", line 843, in exec_module
api-1       |   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
api-1       |   File "/home/app/src/main.py", line 9, in <module>
api-1       |     from src.keycloak.idp import idp
api-1       |   File "/home/app/src/keycloak/idp.py", line 4, in <module>
api-1       |     idp = Keycloak(
api-1       |   File "/home/app/src/keycloak/base.py", line 10, in __init__
api-1       |     super().__init__(**kwargs)
api-1       |   File "/.venv/lib/python3.8/site-packages/fastapi_keycloak/api.py", line 162, in __init__
api-1       |     self._get_admin_token()  # Requests an admin access token on startup
api-1       |   File "/.venv/lib/python3.8/site-packages/fastapi_keycloak/api.py", line 346, in _get_admin_token
api-1       |     response = requests.post(url=self.token_uri, headers=headers, data=data, timeout=self.timeout)
api-1       |   File "/.venv/lib/python3.8/site-packages/requests/api.py", line 115, in post
api-1       |     return request("post", url, data=data, json=json, **kwargs)
api-1       |   File "/.venv/lib/python3.8/site-packages/requests/api.py", line 59, in request
api-1       |     return session.request(method=method, url=url, **kwargs)
api-1       |   File "/.venv/lib/python3.8/site-packages/requests/sessions.py", line 575, in request
api-1       |     prep = self.prepare_request(req)
api-1       |   File "/.venv/lib/python3.8/site-packages/requests/sessions.py", line 486, in prepare_request
api-1       |     p.prepare(
api-1       |   File "/.venv/lib/python3.8/site-packages/requests/models.py", line 368, in prepare
api-1       |     self.prepare_url(url, params)
api-1       |   File "/.venv/lib/python3.8/site-packages/requests/models.py", line 439, in prepare_url
api-1       |     raise MissingSchema(
api-1       | requests.exceptions.MissingSchema: Invalid URL 'None': No scheme supplied. Perhaps you meant https://None?

Ca ne me parle pas suffisament pour identifier le problème ... Auriez-vous de pistes ?

geobrun commented 7 months ago

Bonjour,

Est-ce que tu pourrais donner des précisions sur ton installation et ton fichier de configuration docker-compose.yml stp ? La dernière ligne pourrait laisser penser qu'il y a un souci la configuration d'une URL dans ce fichier, notamment au niveau de Keycloak (je ne sais plus si le dernier souci à ce sujet avait été résolu).

PNPyrenees commented 7 months ago

Voici le contenu du fichier docker-compose.yml :

version: "3.9"

x-restart-policy: &restart_policy
  restart: unless-stopped

x-project-defaults: &project_defaults
  <<: *restart_policy
  networks:
    - annotation_nw
  env_file: .env

services:
  traefik:
    <<: *project_defaults
    image: ${TRAEFIK_IMAGE:-traefik:v2.6}
    depends_on:
      frontend:
        condition: service_healthy
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro

  api:
    <<: *project_defaults
    image: ${API_IMAGE:-registry.gitlab.com/natural-solutions/geonature/annotation:api-dev}
    build:
      context: ../api
      dockerfile: Dockerfile
      target: development
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/openapi.json"]
      interval: 10s
      timeout: 5s
      retries: 3
      start_period: 20s
    depends_on:
      db:
        condition: service_healthy
      minio:
        condition: service_healthy
      keycloak:
        condition: service_healthy
    labels:
      - traefik.enable=true
      - traefik.http.routers.api.rule=${TRAEFIK_ROUTER_RULE_API:-PathPrefix(`/api/v1`)}
      - traefik.http.routers.api.entrypoints=web
      - traefik.http.routers.api.middlewares=api-stripprefix
      - traefik.http.middlewares.api-stripprefix.stripprefix.prefixes=${API_ROOT_PATH:-/api/v1}
    environment:
      - KEYCLOAK_ADMIN_CLIENT_SECRET=${KEYCLOAK_ADMIN_CLIENT_SECRET}
      - KEYCLOAK_CALLBACK_URI=${KEYCLOAK_CALLBACK_URI:-http://api:8000/callback}
      - KEYCLOAK_CLIENT_ID=${KEYCLOAK_CLIENT_ID:-geonature-annotation}
      - KEYCLOAK_CLIENT_SECRET=${KEYCLOAK_CLIENT_SECRET}
      - KEYCLOAK_REALM=${KEYCLOAK_REALM:-geonature-annotation}
      - KEYCLOAK_SERVER_URL=${KEYCLOAK_SERVER_URL:-http://keycloak:8080/auth}
      - KEYCLOAK_TOKEN_URI=${KEYCLOAK_TOKEN_URI}

  taxapi:
    <<: *project_defaults
    image: ${TAXAPI_IMAGE:-registry.gitlab.com/natural-solutions/geonature/taxapi:taxapi-latest}
    labels:
      - traefik.enable=true
      - traefik.http.routers.taxapi.rule=${TRAEFIK_ROUTER_RULE_TAXAPI:-PathPrefix(`/taxapi`)}
      - traefik.http.routers.taxapi.entrypoints=web
    environment:
      - TAXREF_FILE=${TAXAPI_TAXREF_FILE:-TAXREF_v16_2022.zip}
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:5666/taxapi/V1/healthcheck"]
      interval: 10s
      timeout: 30s
      retries: 5
      start_period: 10s

  frontend:
    <<: *project_defaults
    image: ${FRONTEND_IMAGE:-registry.gitlab.com/natural-solutions/geonature/annotation:frontend-dev}
    build:
      context: ../frontend
      dockerfile: Dockerfile
    depends_on:
      api:
        condition: service_healthy
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost/"]
      interval: 10s
      timeout: 5s
      retries: 3
      start_period: 60s
    labels:
      - traefik.enable=true
      - traefik.http.routers.frontend.rule=${TRAEFIK_ROUTER_RULE_FRONTEND:-PathPrefix(`/`)}
      - traefik.http.routers.frontend.entrypoints=web
    environment:
      - REACT_APP_KEYCLOAK_CLIENT_URL=${KEYCLOAK_CLIENT_URL:-http://localhost:8888/auth}
      - REACT_APP_KEYCLOAK_CLIENT_REALM=${KEYCLOAK_REALM_ID:-geonature-annotation}
      - REACT_APP_KEYCLOAK_CLIENT_CLIENT_ID=${KEYCLOAK_CLIENT_CLIENT_ID:-frontend}

  db:
    <<: *project_defaults
    image: ${DB_IMAGE:-postgres:14-alpine}
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "pg_isready -U ${DB_USER-dbuser} -d ${DB_NAME-annotation} -h 127.0.0.1",
        ]
      interval: 10s
      timeout: 5s
      retries: 3
      start_period: 60s
    volumes:
      - db_data:/var/lib/postgresql/data
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD:-dbpassword}
      POSTGRES_DB: ${DB_NAME:-annotation}
      POSTGRES_USER: ${DB_USER:-dbuser}

  keycloak:
    <<: *project_defaults
    image: ${KEYCLOAK_IMAGE:-registry.gitlab.com/natural-solutions/geonature/annotation:keycloak-dev}
    build:
      context: ../keycloak
    command: start-dev --import-realm
    healthcheck:
      # https://github.com/keycloak/keycloak/issues/17273#issuecomment-1693549331
      test: cat /proc/net/tcp | grep '00000000:1F90 00000000:0000' || exit 1
      interval: 10s
      timeout: 5s
      retries: 10
      start_period: 80s
    depends_on:
      db:
        condition: service_healthy
    labels:
      - traefik.enable=true
      - traefik.http.routers.keycloak.rule=${TRAEFIK_ROUTER_RULE_KEYCLOAK:-PathPrefix(`/auth`)}
      - traefik.http.routers.keycloak.entrypoints=web

  minio:
    <<: *project_defaults
    image: ${MINIO_IMAGE:-minio/minio:RELEASE.2023-08-23T10-07-06Z}
    volumes:
      - minio_data:/data
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3
    command: server --console-address :9001 /data
    labels:
      - traefik.enable=true
      - traefik.http.routers.minio.rule=${TRAEFIK_ROUTER_RULE_MINIO:-PathPrefix(`/miniobucket`)}

volumes:
  db_data:
  minio_data:

networks:
  annotation_nw:
    external: false
MathildeNS commented 7 months ago

Bonjour,

Pourriez-vous vérifier votre fichier .env. Il doit y avoir un problème au niveau d'une des variables probablement celle du domain, l'avez-vous bien renseignée ?

PNPyrenees commented 7 months ago

Etant sur une VM locale, je vous partage le .env pour analyse :

# To activate if production mode has to be activated
#ENV=production

# To activate if demo instance
#DEMO_INSTANCE=True

PROJECT_NAME=test-ecosecrets
DOMAIN=10.0.0.218
PROTOCOL=https  # either http or https
HTTP_HTTPS_PORT=443  # either 80 or 443
CANONICAL_URL=${PROTOCOL}://${DOMAIN}:${HTTP_HTTPS_PORT}

# Login/password to log in into GeoCam
APP_USER=ecosecrets
APP_PASSWORD=ecosecrets

# Database credentials
DB_IMAGE=postgres:14-alpine
DB_USER=ecosecrets
DB_PASSWORD=ecosecrets
DB_NAME=ecosecrets
DB_ADDR=db
DB_URL=postgresql://${DB_USER}:${DB_PASSWORD}@${DB_ADDR}/${DB_NAME}

MINIO_ROOT_USER=miniouser
MINIO_ROOT_PASSWORD=miniopassword
MINIO_ENTRYPOINT_URL=${CANONICAL_URL}
MINIO_BUCKET_NAME=miniobucket
MINIO_IMAGE=minio/minio:RELEASE.2023-08-23T10-07-06Z

IMPORT_VARSUBSTITUTION_ENABLED=true

KEYCLOAK_IMAGE=registry.gitlab.com/natural-solutions/geonature/annotation:keycloak-dev
KEYCLOAK_REALM_ID=${PROJECT_NAME}
KEYCLOAK_CLIENT_ID=${PROJECT_NAME}
KEYCLOAK_TOKEN_URI=${CANONICAL_URL}/auth/realms/${KEYCLOAK_REALM_ID}/protocol/openid-connect/token
KEYCLOAK_CLIENT_URL=${CANONICAL_URL}/auth
KEYCLOAK_CLIENT_SECRET=lPcSs3cPCiJRuAd9RZYvkjiGTralT5O4
KEYCLOAK_ADMIN_CLIENT_SECRET=7SkNd9NuRf0bDIVmxcJVKtVcjyTSKyxB
KEYCLOAK_CLIENT_CLIENT_ID=frontend
KEYCLOAK_ADMIN=${APP_USER}
KEYCLOAK_ADMIN_PASSWORD=${APP_PASSWORD}
KC_HTTP_RELATIVE_PATH=/auth
KC_PROXY=edge
KC_FEATURES=scripts
KC_DB=postgres
KC_DB_URL_HOST=${DB_ADDR}
KC_DB_URL_DATABASE=${DB_NAME}
KC_DB_USERNAME=${DB_USER}
KC_DB_PASSWORD=${DB_PASSWORD}
KEYCLOAK_FRONTEND_CLIENT_ROOT_URL=${CANONICAL_URL}

API_IMAGE=registry.gitlab.com/natural-solutions/geonature/annotation:api-dev
API_ROOT_PATH=/api/v1

REACT_APP_API_PATH=${CANONICAL_URL}${API_ROOT_PATH}
FRONTEND_IMAGE=registry.gitlab.com/natural-solutions/geonature/annotation:frontend-dev

TAXAPI_IMAGE=registry.gitlab.com/natural-solutions/geonature/taxapi:taxapi-latest
TAXAPI_TAXREF_FILE="TAXREF_v16_2022.zip"
TAXAPI_ROOT_PATH=/taxapi

TRAEFIK_IMAGE=traefik:v2.6
TRAEFIK_PORT=${HTTP_HTTPS_PORT}
TRAEFIK_DASHBOARD_PORT=8890
TRAEFIK_ROUTER_RULE_API=HOST(`${DOMAIN}`) && PathPrefix(`${API_ROOT_PATH}`)
TRAEFIK_ROUTER_RULE_KEYCLOAK=HOST(`${DOMAIN}`) && PathPrefix(`${KC_HTTP_RELATIVE_PATH}`)
TRAEFIK_ROUTER_RULE_FRONTEND=HOST(`${DOMAIN}`) && PathPrefix(`/`)
TRAEFIK_ROUTER_RULE_MINIO=Host(`${DOMAIN}`) && PathPrefix(`/${MINIO_BUCKET_NAME}`)
TRAEFIK_ROUTER_RULE_TAXAPI=HOST(`${DOMAIN}`) && PathPrefix(`${TAXAPI_ROOT_PATH}`)
geobrun commented 7 months ago

Effectivement @MathildeNS, j'ai envoyé mon message un peu trop vite : c'est bien sûr du .env dont je voulais parler.

@PNPyrenees, tu peux déjà changer :

Si je comprends bien, tu n'as pas la même configuration que celle testée l'année dernière, mais le même message d'erreur : Invalid URL 'None': No scheme supplied. Perhaps you meant https://None? ?

PNPyrenees commented 7 months ago

J'ai appliqué les modifications recommandées bien que l'IP est valide mais le comportement reste le même.

Pour info, avant d'appliquer les modif, je fais un $ sudo ./scripts/docker.sh down -v Puis un $ sudo ./scripts/docker.sh up -d

Si je comprends bien, tu n'as pas la même configuration que celle testée l'année dernière, mais le même message d'erreur : Invalid URL 'None': No scheme supplied. Perhaps you meant https://None? ?

En effet @geobrun, comme @MathildeNS me l'avais conseillé, j'ai mis de côté le test d'installation pour le reprendre ces derniers jours sur la base de la dernière version et sur une VM remise à zéro.

MathildeNS commented 1 month ago

Bonjour,

A priori, il y a deux soucis dans votre fichier .env:

  1. Il ne faut pas changer les variables APP_USER=admin et APP_PASSWORD=password, le mot de passe peut être changé dans l'interface de keycloak une fois l'installation terminée comme décrit dans la documentation : Documentation keycloak
  2. Il manque une variable dans notre fichier .env.sample et par conséquent dans votre .env également. Il faut rajouter la variable KEYCLOAK_REALM=${PROJECT_NAME}

Avec ces deux modifications, j'ai pu avoir mon instance de dev en local fonctionnelle. Avec ces modifications vous devriez donc réussir à installer l'application.

Je vous joins votre fichier .env corrigé :

# To activate if production mode has to be activated
#ENV=production

# To activate if demo instance
#DEMO_INSTANCE=True

PROJECT_NAME=test-ecosecrets
DOMAIN=10.0.0.218
PROTOCOL=https  # either http or https
HTTP_HTTPS_PORT=443  # either 80 or 443
CANONICAL_URL=${PROTOCOL}://${DOMAIN}:${HTTP_HTTPS_PORT}

# Login/password to log in into ecoSecrets
APP_USER=admin
APP_PASSWORD=password

# Database credentials
DB_IMAGE=postgres:14-alpine
DB_USER=ecosecrets
DB_PASSWORD=ecosecrets
DB_NAME=ecosecrets
DB_ADDR=db
DB_URL=postgresql://${DB_USER}:${DB_PASSWORD}@${DB_ADDR}/${DB_NAME}

MINIO_ROOT_USER=miniouser
MINIO_ROOT_PASSWORD=miniopassword
MINIO_ENTRYPOINT_URL=${CANONICAL_URL}
MINIO_BUCKET_NAME=miniobucket
MINIO_IMAGE=minio/minio:RELEASE.2023-08-23T10-07-06Z

IMPORT_VARSUBSTITUTION_ENABLED=true

KEYCLOAK_IMAGE=registry.gitlab.com/natural-solutions/geonature/annotation:keycloak-dev
KEYCLOAK_REALM=${PROJECT_NAME}
KEYCLOAK_CALLBACK_URI=${CANONICAL_URL}/*
KEYCLOAK_REALM_ID=${PROJECT_NAME}
KEYCLOAK_CLIENT_ID=${PROJECT_NAME}
KEYCLOAK_TOKEN_URI=${CANONICAL_URL}/auth/realms/${KEYCLOAK_REALM_ID}/protocol/openid-connect/token
KEYCLOAK_CLIENT_URL=${CANONICAL_URL}/auth
KEYCLOAK_CLIENT_SECRET=lPcSs3cPCiJRuAd9RZYvkjiGTralT5O4
KEYCLOAK_ADMIN_CLIENT_SECRET=7SkNd9NuRf0bDIVmxcJVKtVcjyTSKyxB
KEYCLOAK_CLIENT_CLIENT_ID=frontend
KEYCLOAK_ADMIN=${APP_USER}
KEYCLOAK_ADMIN_PASSWORD=${APP_PASSWORD}
KC_HTTP_RELATIVE_PATH=/auth
KC_PROXY=edge
KC_FEATURES=scripts
KC_DB=postgres
KC_DB_URL_HOST=${DB_ADDR}
KC_DB_URL_DATABASE=${DB_NAME}
KC_DB_USERNAME=${DB_USER}
KC_DB_PASSWORD=${DB_PASSWORD}
KEYCLOAK_FRONTEND_CLIENT_ROOT_URL=${CANONICAL_URL}

API_IMAGE=registry.gitlab.com/natural-solutions/geonature/annotation:api-dev
API_ROOT_PATH=/api/v1

REACT_APP_API_PATH=${CANONICAL_URL}${API_ROOT_PATH}
FRONTEND_IMAGE=registry.gitlab.com/natural-solutions/geonature/annotation:frontend-dev

TAXAPI_IMAGE=registry.gitlab.com/natural-solutions/geonature/taxapi:taxapi-latest
TAXAPI_TAXREF_FILE="TAXREF_v16_2022.zip"
TAXAPI_ROOT_PATH=/taxapi

TRAEFIK_IMAGE=traefik:v2.6
TRAEFIK_PORT=${HTTP_HTTPS_PORT}
TRAEFIK_DASHBOARD_PORT=8890
TRAEFIK_ROUTER_RULE_API=HOST(`${DOMAIN}`) && PathPrefix(`${API_ROOT_PATH}`)
TRAEFIK_ROUTER_RULE_KEYCLOAK=HOST(`${DOMAIN}`) && PathPrefix(`${KC_HTTP_RELATIVE_PATH}`)
TRAEFIK_ROUTER_RULE_FRONTEND=HOST(`${DOMAIN}`) && PathPrefix(`/`)
TRAEFIK_ROUTER_RULE_MINIO=Host(`${DOMAIN}`) && PathPrefix(`/${MINIO_BUCKET_NAME}`)
TRAEFIK_ROUTER_RULE_TAXAPI=HOST(`${DOMAIN}`) && PathPrefix(`${TAXAPI_ROOT_PATH}`)

Toutes nos excuses pour le délai de réponse

PNPyrenees commented 1 month ago

Merci pour ce retour, Je regarde et test ça dès que possible pour vous en faire un retour.