Minor issues

[mo-dkrz]

@antarcticrainforest, I've listed all the minor issues I encountered during a Freva deployment experiment. I think that once a PR is submitted and merged, we'll be ready to proceed with upgrading the Freva instance in the RegiKlim project.

1. construction of the binaries needs more consideration before any action imo.

   $ ./deploy-freva 
   [272488] Failed to load Python shared library '/home/k/k202187/doploy/deploy-freva/_internal/libpython3.11.so.1.0':   dlopen: /lib64/libm.so.6: version `GLIBC_2.35' not found (required by /home/k/k202187/doploy/deploy-freva/_internal/libpython3.11.so.1.0)

2. I think we need to replace /opt/conda/bin/freva with {{ core_install_dir | regex_replace('^~', ansible_env.HOME)}}/bin/freva in this line: https://github.com/FREVA-CLINT/freva-deployment/blob/main/assets/share/freva/deployment/playbooks/core-server-playbook.yml#L58
3. the comment described here arises because as an unprivileged user the code-line mentioned in the comment retrieves the data_path, which is mandatory but also defined with a default value. When the install_dir is not explicitly defined in the core configuration (since it's optional) this issue happens. In short, if you don't define the install_dir in the core (it's optional), the data ends up in a pre-defined directory, and the data_path defaults to /opt/freva(it's mandatory and also a default value exist for it, which i couldn't get it!). To avoid this issue, one has to define both data_path and install_dir in the same directory. Otherwise, the user will encounter errors during Freva deployment in vault which explained here.
4. this comment happens when we have no /opt/freva dir while we get the root password of machine from user. My idea is to simply make this dir without raising any failed task, but we have to discuss the pros and cons of this approach.
5. In this comment if we are going to raise any error due to the empty username and pass for mail server, imho it would be better to make it mandatory from the beginning instead of getting failed task
6. The logo can easily cause an internal error in Freva-web, potentially leading to a broken Freva-web container. In my opinion, the solution could be to implement a health check for the logo. This would allow the final web task to fail if there's an issue, rather than the job appearing successful and only discovering the problem later in the Docker logs.
7. Allowed_hosts needs more consideration for adding more than one host. it has been explained completely here

Thanks in advance for consideration

[mo-dkrz]

alright, i just followed the readme - i just used a user

$ uname -m
x86_64
$ wget https://github.com/FREVA-CLINT/freva-deployment/releases/download/v2407.2.2/doploy-freva-v2407.2.2-linux-x64.tar.gz
$ tar -xvf doploy-freva-v2407.2.2-linux-x64.tar.gz
$ cd deploy-freva
$ ./deploy-freva
[153812] Failed to load Python shared library '/home/deploy-freva/_internal/libpython3.11.so.1.0': dlopen: /lib64/libm.so.6: version `GLIBC_2.35' not found (required by /home/deploy-freva/_internal/libpython3.11.so.1.0)

I will send a PR regarding this when i'm back from vacation

[mo-dkrz]

probable issue: hardcoded path located in the core playbook, might make failed job in deployment process in situations with different freva installation configuration.

https://github.com/FREVA-CLINT/freva-deployment/blob/main/assets/share/freva/deployment/playbooks/core-server-playbook.yml#L58

[mo-dkrz]

the following dest dir seems to not be well configured: https://github.com/FREVA-CLINT/freva-deployment/blob/main/assets/share/freva/deployment/playbooks/db-server-playbook.yml#L126

[antarcticrainforest]

probable issue: hardcoded path located in the core playbook, might make failed job in deployment process in situations with different freva installation configuration.

https://github.com/FREVA-CLINT/freva-deployment/blob/main/assets/share/freva/deployment/playbooks/core-server-playbook.yml#L58

Very good catch. I don't know what I was doing there. I'll take care of this tomorrow.

[antarcticrainforest]

the following dest dir seems to not be well configured: https://github.com/FREVA-CLINT/freva-deployment/blob/main/assets/share/freva/deployment/playbooks/db-server-playbook.yml#L126

How so?

[mo-dkrz]

the following dest dir seems to not be well configured: https://github.com/FREVA-CLINT/freva-deployment/blob/main/assets/share/freva/deployment/playbooks/db-server-playbook.yml#L126

How so?

I will write down the issue in detail. It seems it defines /opt/freva/freva/config/vault/ as default and kind of unchangeable. I have to dive deeper, but still I’m dubious that it might be my misconfiguration. I just wrote to not forget to consider after finishing deployment

[antarcticrainforest]

the following dest dir seems to not be well configured: https://github.com/FREVA-CLINT/freva-deployment/blob/main/assets/share/freva/deployment/playbooks/db-server-playbook.yml#L126

How so?

I will write down the issue in detail. It seems it defines /opt/freva/freva/config/vault/ as default and kind of unchangeable. I have to dive deeper, but still I’m dubious that it might be my misconfiguration. I just wrote to not forget to consider after finishing deployment

Right, the vault_data_path is (or should be) the data_path that you've set for setting up the db service.

If you are referring to the double freva that's easy explained.

The general idea is to combine all freva stuff under data_path, which defaults to /opt/freva. Different project services and their config will be located in /. This way you could in theory have different projects hosted on one machine. For example /opt/freva/regiklim, /opt/freva/xces etc. The reasoning is that /opt/freva makes it clear that we are talking about freva and all underneath belongs to freva. I had a lengthy discussion with our system admin about the directories.

I suspect you left the default for data_path at /opt/freva and set project name to freva. Then you'll end up with /opt/freva/freva.

To get away with this you can just set the datapath to /opt. But I am open to better suggestions.

[mo-dkrz]

during the deployment with the default configuration, we face permission denied in freva-vault when it adds secret. As I'm deploying via default configuration, for sure there is a trick here, but we need to find it out ...

TASK [db - Inserting server infrastructure]

curl -X POST http://localhost:5002/vault/data?secret=db.container%3Dfreva-vault%2Cdb.host%3Dlocalhost%2Cdb.port%3D3306%2Cdb.db%3Dfrevadb%2Cdb.passwd%3D******%2Cdb.user%3Droot

output

{"detail":"Permission denied"}

[antarcticrainforest]

Would you be able to post more logs of the container?

As a side note, you don't want to use root as your DB user.

[mo-dkrz]

during the deployment with the default configuration, we face permission denied in freva-vault when it adds secret. As I'm deploying via default configuration, for sure there is a trick here, but we need to find it out ...

TASK [db - Inserting server infrastructure]

curl -X POST http://localhost:5002/vault/data?secret=db.container%3Dfreva-vault%2Cdb.host%3Dlocalhost%2Cdb.port%3D3306%2Cdb.db%3Dfrevadb%2Cdb.passwd%3D******%2Cdb.user%3Droot

output

{"detail":"Permission denied"}

As a complement to the freva-vault, we've identified the main issue as being related to user permissions, particularly when reviewing the container logs. The logs indicate multiple attempts to unseal the Vault, each resulting in permission errors. The core issue is that Vault is unable to persist its keyring due to a lack of permissions to create or write to the necessary directories under /vault/file/core.

$ docker logs freva-vault
/bin/runserver.py:199: DeprecationWarning: `example` has been deprecated, please use `examples` instead
  path: Annotated[str, Path(description="Secret location.", example="test")],
/bin/runserver.py:202: DeprecationWarning: `example` has been deprecated, please use `examples` instead
  Query(
/bin/runserver.py:215: DeprecationWarning: `example` has been deprecated, please use `examples` instead
  Header(
/bin/runserver.py:259: DeprecationWarning: `example` has been deprecated, please use `examples` instead
  Path(
/bin/runserver.py:266: DeprecationWarning: `example` has been deprecated, please use `examples` instead
  Path(
2024-08-12T06:47:33+0000 - secret-reader - [INFO] - Unsealing vault
2024-08-12T06:47:33+0000 - secret-reader - [WARNING] - Vault not ready yet.
==> Vault server configuration:

Administrative Namespace: 
             Api Address: http://127.0.0.1:8200
                     Cgo: disabled
         Cluster Address: https://127.0.0.1:8201
   Environment Variables: HOME, HOSTNAME, NAME, PATH, PWD, ROOT_PW, SHLVL, TERM, VAULT_ADDR, VERSION
              Go Version: go1.21.9
              Listener 1: tcp (addr: "0.0.0.0:8200", cluster address: "0.0.0.0:8201", disable_request_limiter: "false", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
               Log Level: 
                   Mlock: supported: true, enabled: false
           Recovery Mode: false
                 Storage: file
                 Version: Vault v1.16.2, built 2024-04-22T16:25:54Z
             Version Sha: c6e4c2d4dc3b0d57791881b087c026e2f75a87cb

==> Vault server started! Log data will stream in below:

2024-08-12T06:47:33.974Z [INFO]  proxy environment: http_proxy="" https_proxy="" no_proxy=""
2024-08-12T06:47:33.974Z [INFO]  incrementing seal generation: generation=1
2024-08-12T06:47:33.974Z [INFO]  core: Initializing version history cache for core
2024-08-12T06:47:33.974Z [INFO]  events: Starting event system
2024-08-12T06:47:34.926Z [INFO]  core: security barrier not initialized
2024-08-12T06:47:34.927Z [INFO]  core: security barrier not initialized
2024-08-12T06:47:34.928Z [ERROR] core: failed to initialize barrier: error="failed to persist keyring: mkdir /vault/file/core: permission denied"
2024-08-12T06:47:34+0000 - secret-reader - [ERROR] - Failed to unseal vault: failed to initialize barrier: failed to persist keyring: mkdir /vault/file/core: permission denied, on put http://127.0.0.1:8200/v1/sys/init
INFO:     Uvicorn running on http://0.0.0.0:5002 (Press CTRL+C to quit)
INFO:     Started parent process [7]
INFO:     Started server process [32]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Started server process [33]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
2024-08-12T06:47:48+0000 - secret-reader - [INFO] - Unsealing vault
2024-08-12T06:47:48.848Z [INFO]  core: security barrier not initialized
2024-08-12T06:47:48.849Z [INFO]  core: security barrier not initialized
2024-08-12T06:47:48.849Z [ERROR] core: failed to initialize barrier: error="failed to persist keyring: mkdir /vault/file/core: permission denied"
2024-08-12T06:47:48+0000 - secret-reader - [ERROR] - Failed to unseal vault: failed to initialize barrier: failed to persist keyring: mkdir /vault/file/core: permission denied, on put http://127.0.0.1:8200/v1/sys/init
2024-08-12T06:47:48+0000 - secret-reader - [WARNING] - Could not add secrets data to {'db.container': 'freva-vault', 'db.host': 'localhost', 'db.port': '3306', 'db.db': 'frevadb', 'db.passwd': '******', 'db.user': 'root'}
INFO:     127.0.0.1:57146 - "POST /vault/data?secret=db.container%3Dfreva-vault%2Cdb.host%3Dlocalhost%2Cdb.port%3D3306%2Cdb.db%3Dfrevadb%2Cdb.passwd%3D******%2Cdb.user%3Droot HTTP/1.1" 500 Internal Server Error

When I attempted to manually create the required directory as the root user within the container, the issue persisted. Even though the directory /vault/file/core was successfully created, Vault still failed to initialize the security barrier, as it could not open or write to the keyring file due to permission denials.

$ docker exec -u root -it freva-vault mkdir /vault/file/core
$ docker logs freva-vault
2024-08-12T13:37:52+0000 - secret-reader - [INFO] - Unsealing vault
2024-08-12T13:37:52.863Z [INFO]  core: security barrier not initialized
2024-08-12T13:37:52.865Z [INFO]  core: security barrier not initialized
2024-08-12T13:37:52.865Z [ERROR] core: failed to initialize barrier: error="failed to persist keyring: open /vault/file/core/_keyring2087876981: permission denied"
2024-08-12T13:37:52+0000 - secret-reader - [ERROR] - Failed to unseal vault: failed to initialize barrier: failed to persist keyring: open /vault/file/core/_keyring2087876981: permission denied, on put http://127.0.0.1:8200/v1/sys/init
2024-08-12T13:37:52+0000 - secret-reader - [WARNING] - Could not add secrets data to {'db.container': 'freva-vault', 'db.host': ...

[mo-dkrz]

Another issue arises when we set the datapath in the db to /opt. Even when defined as the root, this configuration leads to errors. The solution is creating the dir manually and we can pass this ...

TASK [db - Copying public key file to target machine] ***********************************************************************************************************************
fatal: [***.***.***.***]: FAILED! => changed=false 
  checksum: ede8007b4d0d5ca09e59c5e17dd8bac3a84d855d
  msg: Destination directory /opt/freva/vault_service/config does not exist

[antarcticrainforest]

Another issue arises when we set the datapath in the db to /opt. Even when defined as the root, this configuration leads to errors. The solution is creating the dir manually and we can pass this ...

TASK [db - Copying public key file to target machine] ***********************************************************************************************************************
fatal: [***.***.***.***]: FAILED! => changed=false 
  checksum: ede8007b4d0d5ca09e59c5e17dd8bac3a84d855d
  msg: Destination directory /opt/freva/vault_service/config does not exist

The directory should have been created before. Can you please confirm that db_data_path = vault_data_path = /opt.

I am not sure why it wasn't created.

[antarcticrainforest]

About the vault error. I believe there is a user right mismtach. Could post the config you are using?

[antarcticrainforest]

Following on from that. Are you using SELinux on the host machine? What's the context of the paths for the docker volume?

[mo-dkrz]

yeah it uses SELinux, and yes I think by changing to status 0, it has to work. but i'm not happy to make it permissive just for deploying. I'm looking for an approach to make it work in every single situation. Or at least if we couldn't find a solution we have to write in https://github.com/FREVA-CLINT/freva-deployment?tab=readme-ov-file#known-issues

freva-db-compose.yml

    volumes:
      - /opt/freva/vault_service/config:/data:z
      - /opt/freva/vault_service/files:/vault/file:z

in my machine:

$ tree /opt/freva/vault_service/
/opt/freva/vault_service/
├── config
│   └── freva.crt
└── files
    └── core

in freva-vault container:

$ tree /vault/
/vault/
├── config
├── file
│   └── core
└── logs

$ tree /data/
/data/
└── freva.crt

Do you have any job in actions which the vm uses SELinux and if yes what was your approach?

[mo-dkrz]

during the deployment with the default configuration, we face permission denied in freva-vault when it adds secret. As I'm deploying via default configuration, for sure there is a trick here, but we need to find it out ... TASK [db - Inserting server infrastructure]

curl -X POST http://localhost:5002/vault/data?secret=db.container%3Dfreva-vault%2Cdb.host%3Dlocalhost%2Cdb.port%3D3306%2Cdb.db%3Dfrevadb%2Cdb.passwd%3D******%2Cdb.user%3Droot

output

{"detail":"Permission denied"}

As a complement to the freva-vault, we've identified the main issue as being related to user permissions, particularly when reviewing the container logs. The logs indicate multiple attempts to unseal the Vault, each resulting in permission errors. The core issue is that Vault is unable to persist its keyring due to a lack of permissions to create or write to the necessary directories under /vault/file/core.

$ docker logs freva-vault
/bin/runserver.py:199: DeprecationWarning: `example` has been deprecated, please use `examples` instead
  path: Annotated[str, Path(description="Secret location.", example="test")],
/bin/runserver.py:202: DeprecationWarning: `example` has been deprecated, please use `examples` instead
  Query(
/bin/runserver.py:215: DeprecationWarning: `example` has been deprecated, please use `examples` instead
  Header(
/bin/runserver.py:259: DeprecationWarning: `example` has been deprecated, please use `examples` instead
  Path(
/bin/runserver.py:266: DeprecationWarning: `example` has been deprecated, please use `examples` instead
  Path(
2024-08-12T06:47:33+0000 - secret-reader - [INFO] - Unsealing vault
2024-08-12T06:47:33+0000 - secret-reader - [WARNING] - Vault not ready yet.
==> Vault server configuration:

Administrative Namespace: 
             Api Address: http://127.0.0.1:8200
                     Cgo: disabled
         Cluster Address: https://127.0.0.1:8201
   Environment Variables: HOME, HOSTNAME, NAME, PATH, PWD, ROOT_PW, SHLVL, TERM, VAULT_ADDR, VERSION
              Go Version: go1.21.9
              Listener 1: tcp (addr: "0.0.0.0:8200", cluster address: "0.0.0.0:8201", disable_request_limiter: "false", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
               Log Level: 
                   Mlock: supported: true, enabled: false
           Recovery Mode: false
                 Storage: file
                 Version: Vault v1.16.2, built 2024-04-22T16:25:54Z
             Version Sha: c6e4c2d4dc3b0d57791881b087c026e2f75a87cb

==> Vault server started! Log data will stream in below:

2024-08-12T06:47:33.974Z [INFO]  proxy environment: http_proxy="" https_proxy="" no_proxy=""
2024-08-12T06:47:33.974Z [INFO]  incrementing seal generation: generation=1
2024-08-12T06:47:33.974Z [INFO]  core: Initializing version history cache for core
2024-08-12T06:47:33.974Z [INFO]  events: Starting event system
2024-08-12T06:47:34.926Z [INFO]  core: security barrier not initialized
2024-08-12T06:47:34.927Z [INFO]  core: security barrier not initialized
2024-08-12T06:47:34.928Z [ERROR] core: failed to initialize barrier: error="failed to persist keyring: mkdir /vault/file/core: permission denied"
2024-08-12T06:47:34+0000 - secret-reader - [ERROR] - Failed to unseal vault: failed to initialize barrier: failed to persist keyring: mkdir /vault/file/core: permission denied, on put http://127.0.0.1:8200/v1/sys/init
INFO:     Uvicorn running on http://0.0.0.0:5002 (Press CTRL+C to quit)
INFO:     Started parent process [7]
INFO:     Started server process [32]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Started server process [33]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
2024-08-12T06:47:48+0000 - secret-reader - [INFO] - Unsealing vault
2024-08-12T06:47:48.848Z [INFO]  core: security barrier not initialized
2024-08-12T06:47:48.849Z [INFO]  core: security barrier not initialized
2024-08-12T06:47:48.849Z [ERROR] core: failed to initialize barrier: error="failed to persist keyring: mkdir /vault/file/core: permission denied"
2024-08-12T06:47:48+0000 - secret-reader - [ERROR] - Failed to unseal vault: failed to initialize barrier: failed to persist keyring: mkdir /vault/file/core: permission denied, on put http://127.0.0.1:8200/v1/sys/init
2024-08-12T06:47:48+0000 - secret-reader - [WARNING] - Could not add secrets data to {'db.container': 'freva-vault', 'db.host': 'localhost', 'db.port': '3306', 'db.db': 'frevadb', 'db.passwd': '******', 'db.user': 'root'}
INFO:     127.0.0.1:57146 - "POST /vault/data?secret=db.container%3Dfreva-vault%2Cdb.host%3Dlocalhost%2Cdb.port%3D3306%2Cdb.db%3Dfrevadb%2Cdb.passwd%3D******%2Cdb.user%3Droot HTTP/1.1" 500 Internal Server Error

When I attempted to manually create the required directory as the root user within the container, the issue persisted. Even though the directory /vault/file/core was successfully created, Vault still failed to initialize the security barrier, as it could not open or write to the keyring file due to permission denials.

$ docker exec -u root -it freva-vault mkdir /vault/file/core
$ docker logs freva-vault
2024-08-12T13:37:52+0000 - secret-reader - [INFO] - Unsealing vault
2024-08-12T13:37:52.863Z [INFO]  core: security barrier not initialized
2024-08-12T13:37:52.865Z [INFO]  core: security barrier not initialized
2024-08-12T13:37:52.865Z [ERROR] core: failed to initialize barrier: error="failed to persist keyring: open /vault/file/core/_keyring2087876981: permission denied"
2024-08-12T13:37:52+0000 - secret-reader - [ERROR] - Failed to unseal vault: failed to initialize barrier: failed to persist keyring: open /vault/file/core/_keyring2087876981: permission denied, on put http://127.0.0.1:8200/v1/sys/init
2024-08-12T13:37:52+0000 - secret-reader - [WARNING] - Could not add secrets data to {'db.container': 'freva-vault', 'db.host': ...

new update:

in the following lines in playbook, we changed the vault_path to data_path and it could pass the mentioned step:

https://github.com/FREVA-CLINT/freva-deployment/blob/main/assets/share/freva/deployment/playbooks/db-server-playbook.yml#L12-L13

changing {{vault_data_path|regex_replace("^~", ansible_env.HOME)}} to {{db_data_path|regex_replace("^~", ansible_env.HOME)}}

[mo-dkrz]

If you leave username and password empty in the following step, which technically it's optional,

The web will need login credentials to connect to the mail server that has been set up.
You should now enter your login credentials.
Note:These credentials will be securely stored in an encrypted vault

Username for mail server: 
Password for mail server:

you will get the following error

TASK [vault - Inserting email secrets] **************************************************************************************************************************************
fatal: [localhost]: FAILED! => changed=true 
  cmd: |-
    /tmp/docker-or-podman exec freva-vault add-vault-secret email -s username  -s password ''
  delta: '0:00:00.257393'
  end: '2024-08-12 17:40:27.637433'
  msg: non-zero return code
  rc: 1
  start: '2024-08-12 17:40:27.380040'
  stderr: |-
    usage: /bin/add-vault-secret [-h] [--secret SECRET SECRET] path
    /bin/add-vault-secret: error: argument --secret/-s: expected 2 arguments
  stderr_lines: <omitted>
  stdout: ''
  stdout_lines: <omitted>

[antarcticrainforest]

Hmm, I think vault_data_path and db_data_path are interchangeable. But I'll look into it.

Just a shot into the dark. Did you set the right context for the volume paths? https://freva-deployment.readthedocs.io/en/latest/deployment/Configure.html#insufficient-permissions-when-running-in-rootless-mode-selinux

The problem should not just be for rootless containers.

Could you output the ls -lZ /opt/freva/

[mo-dkrz]

Alright, it says deployment is successfully finished with a number of ok, changed, and skipped and without any failed, but I don't have the freva-web around in the list. Now it needs more consideration why I didn't get any failed and successfully finished the job while have no freva-web. I'm a bit skeptical to 26 skipped ones. let's jump there to find out

PLAY RECAP ******************************************************************************************************************************************************************
****.****.***.****             : ok=106  changed=56   unreachable=0    failed=0    skipped=26   rescued=0    ignored=0   
localhost                  : ok=4    changed=3    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

$ docker ps
CONTAINER ID   IMAGE                                         COMMAND                  CREATED          STATUS              PORTS     NAMES
3f5b45b4264e   httpd:latest                                  "/usr/local/bin/prep…"   55 seconds ago   Up 54 seconds                 freva-httpd
b369775bd3b9   redis:latest                                  "docker-entrypoint.s…"   55 seconds ago   Up 54 seconds                 freva-redis
4699f00c9b91   ghcr.io/freva-clint/freva-rest-api:2403.0.3   "python3 -m freva_re…"   2 minutes ago    Up About a minute             freva-freva_rest
5c5972db8f85   solr:9.6.0                                    "docker-entrypoint.s…"   2 minutes ago    Up 2 minutes                  freva-solr
db27310de68e   mongo:latest                                  "docker-entrypoint.s…"   2 minutes ago    Up 2 minutes                  freva-mongo
3e7ac054f957   mariadb:11.1                                  "docker-entrypoint.s…"   2 minutes ago    Up 2 minutes                  freva-db
836f61d5dd47   ghcr.io/freva-clint/freva-vault:2405.1.0      "docker-entrypoint.s…"   2 minutes ago    Up 2 minutes                  freva-vault

[antarcticrainforest]

Where did you deploy this? Why is the server set to localhost?

[mo-dkrz]

ls -lZ /opt/freva/

it's very good we have this documented here

but no i didn't set any right context

$ ls -lZ /opt/freva/
total 0
drwxr-xr-x. 4 root root unconfined_u:object_r:usr_t:s0 33 Aug 12 14:53 vault_service

[mo-dkrz]

Where did you deploy this? Why is the server set to localhost?

pretty timely thanks

on dkrz cloud. I'm getting a test to make it ready for RegiKlim upgrade

[mo-dkrz]

The logo path must be revised; otherwise, the freva-web container will stop. One suggestion is to set a default logo for the institution_logo or implement a logo file health checker before starting the deployment. or at least it needs to failed a task in palybook

$ docker logs freva-web 
/opt/freva_web/django_evaluation/settings/base.py:244: SyntaxWarning: invalid escape sequence '\d'
  USERNAME_FILTER = "[a-z]\d{6,6}"
Traceback (most recent call last):
  File "/opt/freva_web/manage.py", line 12, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/condaenv/lib/python3.12/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/opt/condaenv/lib/python3.12/site-packages/django/core/management/__init__.py", line 382, in execute
    settings.INSTALLED_APPS
  File "/opt/condaenv/lib/python3.12/site-packages/django/conf/__init__.py", line 102, in __getattr__
    self._setup(name)
  File "/opt/condaenv/lib/python3.12/site-packages/django/conf/__init__.py", line 89, in _setup
    self._wrapped = Settings(settings_module)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/django/conf/__init__.py", line 217, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/opt/freva_web/django_evaluation/settings/__init__.py", line 4, in <module>
    from .local import *
  File "/opt/freva_web/django_evaluation/settings/local.py", line 140, in <module>
    INSTITUTION_LOGO = _get_logo(
                       ^^^^^^^^^^
  File "/opt/freva_web/django_evaluation/settings/local.py", line 94, in _get_logo
    shutil.copy(logo_file, new_file)
  File "/opt/condaenv/lib/python3.12/shutil.py", line 435, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/opt/condaenv/lib/python3.12/shutil.py", line 260, in copyfile
    with open(src, 'rb') as fsrc:
         ^^^^^^^^^^^^^^^
IsADirectoryError: [Errno 21] Is a directory: '/home'
Traceback (most recent call last):
  File "/opt/freva_web/manage.py", line 12, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/condaenv/lib/python3.12/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/opt/condaenv/lib/python3.12/site-packages/django/core/management/__init__.py", line 382, in execute
    settings.INSTALLED_APPS
  File "/opt/condaenv/lib/python3.12/site-packages/django/conf/__init__.py", line 102, in __getattr__
    self._setup(name)
  File "/opt/condaenv/lib/python3.12/site-packages/django/conf/__init__.py", line 89, in _setup
    self._wrapped = Settings(settings_module)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/django/conf/__init__.py", line 217, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/opt/freva_web/django_evaluation/settings/__init__.py", line 4, in <module>
    from .local import *
  File "/opt/freva_web/django_evaluation/settings/local.py", line 140, in <module>
    INSTITUTION_LOGO = _get_logo(
                       ^^^^^^^^^^
  File "/opt/freva_web/django_evaluation/settings/local.py", line 94, in _get_logo
    shutil.copy(logo_file, new_file)
  File "/opt/condaenv/lib/python3.12/shutil.py", line 435, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/opt/condaenv/lib/python3.12/shutil.py", line 260, in copyfile
    with open(src, 'rb') as fsrc:
         ^^^^^^^^^^^^^^^
IsADirectoryError: [Errno 21] Is a directory: '/home'
Traceback (most recent call last):
  File "/opt/freva_web/manage.py", line 12, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/condaenv/lib/python3.12/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/opt/condaenv/lib/python3.12/site-packages/django/core/management/__init__.py", line 382, in execute
    settings.INSTALLED_APPS
  File "/opt/condaenv/lib/python3.12/site-packages/django/conf/__init__.py", line 102, in __getattr__
    self._setup(name)
  File "/opt/condaenv/lib/python3.12/site-packages/django/conf/__init__.py", line 89, in _setup
    self._wrapped = Settings(settings_module)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/django/conf/__init__.py", line 217, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/opt/freva_web/django_evaluation/settings/__init__.py", line 4, in <module>
    from .local import *
  File "/opt/freva_web/django_evaluation/settings/local.py", line 140, in <module>
    INSTITUTION_LOGO = _get_logo(
                       ^^^^^^^^^^
  File "/opt/freva_web/django_evaluation/settings/local.py", line 94, in _get_logo
    shutil.copy(logo_file, new_file)
  File "/opt/condaenv/lib/python3.12/shutil.py", line 435, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/opt/condaenv/lib/python3.12/shutil.py", line 260, in copyfile
    with open(src, 'rb') as fsrc:
         ^^^^^^^^^^^^^^^
IsADirectoryError: [Errno 21] Is a directory: '/home'
Traceback (most recent call last):
  File "/opt/freva_web/manage.py", line 12, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/condaenv/lib/python3.12/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/opt/condaenv/lib/python3.12/site-packages/django/core/management/__init__.py", line 382, in execute
    settings.INSTALLED_APPS
  File "/opt/condaenv/lib/python3.12/site-packages/django/conf/__init__.py", line 102, in __getattr__
    self._setup(name)
  File "/opt/condaenv/lib/python3.12/site-packages/django/conf/__init__.py", line 89, in _setup
    self._wrapped = Settings(settings_module)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/django/conf/__init__.py", line 217, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/opt/freva_web/django_evaluation/settings/__init__.py", line 4, in <module>
    from .local import *
  File "/opt/freva_web/django_evaluation/settings/local.py", line 140, in <module>
    INSTITUTION_LOGO = _get_logo(
                       ^^^^^^^^^^
  File "/opt/freva_web/django_evaluation/settings/local.py", line 94, in _get_logo
    shutil.copy(logo_file, new_file)
  File "/opt/condaenv/lib/python3.12/shutil.py", line 435, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/opt/condaenv/lib/python3.12/shutil.py", line 260, in copyfile
    with open(src, 'rb') as fsrc:
         ^^^^^^^^^^^^^^^
IsADirectoryError: [Errno 21] Is a directory: '/home'
Traceback (most recent call last):
  File "/opt/freva_web/manage.py", line 12, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/condaenv/lib/python3.12/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/opt/condaenv/lib/python3.12/site-packages/django/core/management/__init__.py", line 382, in execute
    settings.INSTALLED_APPS
  File "/opt/condaenv/lib/python3.12/site-packages/django/conf/__init__.py", line 102, in __getattr__
    self._setup(name)
  File "/opt/condaenv/lib/python3.12/site-packages/django/conf/__init__.py", line 89, in _setup
    self._wrapped = Settings(settings_module)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/django/conf/__init__.py", line 217, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/opt/freva_web/django_evaluation/settings/__init__.py", line 4, in <module>
    from .local import *
  File "/opt/freva_web/django_evaluation/settings/local.py", line 140, in <module>
    INSTITUTION_LOGO = _get_logo(
                       ^^^^^^^^^^
  File "/opt/freva_web/django_evaluation/settings/local.py", line 94, in _get_logo
    shutil.copy(logo_file, new_file)
  File "/opt/condaenv/lib/python3.12/shutil.py", line 435, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/opt/condaenv/lib/python3.12/shutil.py", line 260, in copyfile
    with open(src, 'rb') as fsrc:
         ^^^^^^^^^^^^^^^
IsADirectoryError: [Errno 21] Is a directory: '/home'
0
Traceback (most recent call last):
  File "/opt/freva_web/manage.py", line 12, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/condaenv/lib/python3.12/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/opt/condaenv/lib/python3.12/site-packages/django/core/management/__init__.py", line 382, in execute
    settings.INSTALLED_APPS
  File "/opt/condaenv/lib/python3.12/site-packages/django/conf/__init__.py", line 102, in __getattr__
    self._setup(name)
  File "/opt/condaenv/lib/python3.12/site-packages/django/conf/__init__.py", line 89, in _setup
    self._wrapped = Settings(settings_module)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/django/conf/__init__.py", line 217, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/opt/freva_web/django_evaluation/settings/__init__.py", line 4, in <module>
    from .local import *
  File "/opt/freva_web/django_evaluation/settings/local.py", line 140, in <module>
    INSTITUTION_LOGO = _get_logo(
                       ^^^^^^^^^^
  File "/opt/freva_web/django_evaluation/settings/local.py", line 94, in _get_logo
    shutil.copy(logo_file, new_file)
  File "/opt/condaenv/lib/python3.12/shutil.py", line 435, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/opt/condaenv/lib/python3.12/shutil.py", line 260, in copyfile
    with open(src, 'rb') as fsrc:
         ^^^^^^^^^^^^^^^
IsADirectoryError: [Errno 21] Is a directory: '/home'
0
[2024-08-12 16:37:53 +0000] [15] [INFO] Starting gunicorn 22.0.0
[2024-08-12 16:37:53 +0000] [15] [INFO] Listening at: http://[::]:8000 (15)
[2024-08-12 16:37:53 +0000] [15] [INFO] Using worker: sync
[2024-08-12 16:37:53 +0000] [16] [INFO] Booting worker with pid: 16
[2024-08-12 16:37:53 +0000] [16] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/opt/condaenv/lib/python3.12/site-packages/gunicorn/arbiter.py", line 609, in spawn_worker
    worker.init_process()
  File "/opt/condaenv/lib/python3.12/site-packages/gunicorn/workers/base.py", line 134, in init_process
    self.load_wsgi()
  File "/opt/condaenv/lib/python3.12/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi
    self.wsgi = self.app.wsgi()
                ^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
                    ^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
    return self.load_wsgiapp()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
    return util.import_app(self.app_uri)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/gunicorn/util.py", line 371, in import_app
    mod = importlib.import_module(module)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/opt/freva_web/django_evaluation/wsgi.py", line 52, in <module>
    from django_evaluation.ldaptools import get_ldap_object
  File "/opt/freva_web/django_evaluation/ldaptools.py", line 14, in <module>
    from django_evaluation import settings
  File "/opt/freva_web/django_evaluation/settings/__init__.py", line 4, in <module>
    from .local import *
  File "/opt/freva_web/django_evaluation/settings/local.py", line 140, in <module>
    INSTITUTION_LOGO = _get_logo(
                       ^^^^^^^^^^
  File "/opt/freva_web/django_evaluation/settings/local.py", line 94, in _get_logo
    shutil.copy(logo_file, new_file)
  File "/opt/condaenv/lib/python3.12/shutil.py", line 435, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/opt/condaenv/lib/python3.12/shutil.py", line 260, in copyfile
    with open(src, 'rb') as fsrc:
         ^^^^^^^^^^^^^^^
IsADirectoryError: [Errno 21] Is a directory: '/home'
[2024-08-12 16:37:53 +0000] [16] [INFO] Worker exiting (pid: 16)
[2024-08-12 16:37:53 +0000] [15] [ERROR] Worker (pid:16) exited with code 3
[2024-08-12 16:37:53 +0000] [15] [ERROR] Shutting down: Master
[2024-08-12 16:37:53 +0000] [15] [ERROR] Reason: Worker failed to boot.

[mo-dkrz]

when we add more than one allowed_hosts, all goes under 'localhost, localhost1, third' which has to be 'localhost', 'localhost1', 'third'

https://github.com/FREVA-CLINT/freva-deployment/blob/main/assets/share/freva/deployment/config/inventory.toml#L344

new update: interesting, actually via changing or adding allowed_hosts in inverntory.toml at the end nothing changes in the ALLOWED_HOSTS in freva-web container. Just my current IP will be added there ...

[mo-dkrz]

strange error. I have to dive to freva-db and freva-web to find why it's inserting my username again and i get CommandError: Error: That username is already taken.

$ docker logs freva-web 
/opt/condaenv/lib/python3.12/site-packages/evaluation_system/model/solr_models/models.py:12: PendingDeprecationWarning: The evaluation_system.model.solr_models module will be removed from v2304.0.0
  warnings.warn(
No changes detected in app 'base'
/opt/condaenv/lib/python3.12/site-packages/evaluation_system/model/solr_models/models.py:12: PendingDeprecationWarning: The evaluation_system.model.solr_models module will be removed from v2304.0.0
  warnings.warn(
Operations to perform:
  Apply all migrations: admin, auth, base, contenttypes, flatpages, sessions, sites
Running migrations:
  No migrations to apply.
/opt/condaenv/lib/python3.12/site-packages/evaluation_system/model/solr_models/models.py:12: PendingDeprecationWarning: The evaluation_system.model.solr_models module will be removed from v2304.0.0
  warnings.warn(
Operations to perform:
  Apply all migrations: contenttypes
Running migrations:
  No migrations to apply.
/opt/condaenv/lib/python3.12/site-packages/evaluation_system/model/solr_models/models.py:12: PendingDeprecationWarning: The evaluation_system.model.solr_models module will be removed from v2304.0.0
  warnings.warn(

0 static files copied to '/opt/freva_web/static', 319 unmodified.
/opt/condaenv/lib/python3.12/site-packages/evaluation_system/model/solr_models/models.py:12: PendingDeprecationWarning: The evaluation_system.model.solr_models module will be removed from v2304.0.0
  warnings.warn(
CommandError: Error: That username is already taken.
0
/opt/condaenv/lib/python3.12/site-packages/evaluation_system/model/solr_models/models.py:12: PendingDeprecationWarning: The evaluation_system.model.solr_models module will be removed from v2304.0.0
  warnings.warn(
Traceback (most recent call last):
  File "/opt/condaenv/lib/python3.12/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/django/db/backends/mysql/base.py", line 75, in execute
    return self.cursor.execute(query, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/MySQLdb/cursors.py", line 179, in execute
    res = self._query(mogrified_query)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/MySQLdb/cursors.py", line 331, in _query
    self._do_get_result(db)
  File "/opt/condaenv/lib/python3.12/site-packages/MySQLdb/cursors.py", line 136, in _do_get_result
    self._result = result = self._get_result()
                            ^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/MySQLdb/cursors.py", line 363, in _get_result
    return self._get_db().store_result()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
MySQLdb.IntegrityError: (1062, "Duplicate entry '1' for key 'PRIMARY'")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/freva_web/manage.py", line 12, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/condaenv/lib/python3.12/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/opt/condaenv/lib/python3.12/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/condaenv/lib/python3.12/site-packages/django/core/management/base.py", line 412, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/opt/condaenv/lib/python3.12/site-packages/django/core/management/base.py", line 458, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/django/core/management/commands/shell.py", line 117, in handle
    exec(options["command"], globals())
  File "<string>", line 1, in <module>
  File "/opt/condaenv/lib/python3.12/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/django/db/models/query.py", line 658, in create
    obj.save(force_insert=True, using=self.db)
  File "/opt/condaenv/lib/python3.12/site-packages/django/db/models/base.py", line 814, in save
    self.save_base(
  File "/opt/condaenv/lib/python3.12/site-packages/django/db/models/base.py", line 877, in save_base
    updated = self._save_table(
              ^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/django/db/models/base.py", line 1020, in _save_table
    results = self._do_insert(
              ^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/django/db/models/base.py", line 1061, in _do_insert
    return manager._insert(
           ^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/django/db/models/query.py", line 1805, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/django/db/models/sql/compiler.py", line 1822, in execute_sql
    cursor.execute(sql, params)
  File "/opt/condaenv/lib/python3.12/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/django/db/backends/utils.py", line 84, in _execute
    with self.db.wrap_database_errors:
  File "/opt/condaenv/lib/python3.12/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/opt/condaenv/lib/python3.12/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/django/db/backends/mysql/base.py", line 75, in execute
    return self.cursor.execute(query, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/MySQLdb/cursors.py", line 179, in execute
    res = self._query(mogrified_query)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/MySQLdb/cursors.py", line 331, in _query
    self._do_get_result(db)
  File "/opt/condaenv/lib/python3.12/site-packages/MySQLdb/cursors.py", line 136, in _do_get_result
    self._result = result = self._get_result()
                            ^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/MySQLdb/cursors.py", line 363, in _get_result
    return self._get_db().store_result()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
django.db.utils.IntegrityError: (1062, "Duplicate entry '1' for key 'PRIMARY'")
0
[2024-08-13 09:14:25 +0000] [51] [INFO] Starting gunicorn 22.0.0
[2024-08-13 09:14:25 +0000] [51] [INFO] Listening at: http://[::]:8000 (51)
[2024-08-13 09:14:25 +0000] [51] [INFO] Using worker: sync
[2024-08-13 09:14:25 +0000] [52] [INFO] Booting worker with pid: 52
Exception in thread Thread-1 (run_ldap_cacheing_daemon):
Traceback (most recent call last):
  File "/opt/condaenv/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/opt/condaenv/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/freva_web/django_evaluation/ldaptools.py", line 112, in run_ldap_cacheing_daemon
    self._cache_ldap_users()
  File "/opt/freva_web/django_evaluation/ldaptools.py", line 120, in _cache_ldap_users
    self.load_from_ldap()
  File "/opt/freva_web/django_evaluation/ldaptools.py", line 257, in load_from_ldap
    self.connection.search_s(
    ^^^^^^^^^^^^^^^
  File "/opt/freva_web/django_evaluation/ldaptools.py", line 150, in connection
    return self._connect_to_ldap()
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/freva_web/django_evaluation/ldaptools.py", line 63, in _connect_to_ldap
    con = self._establish_ldap_connection()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/freva_web/django_evaluation/ldaptools.py", line 50, in _establish_ldap_connection
    raise error
  File "/opt/freva_web/django_evaluation/ldaptools.py", line 40, in _establish_ldap_connection
    con = ldap.initialize(SERVER)
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/ldap/functions.py", line 91, in initialize
    return LDAPObject(
           ^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/ldap/ldapobject.py", line 88, in __init__
    self._l = ldap.functions._ldap_function_call(ldap._ldap_module_lock,_ldap.initialize,uri)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/condaenv/lib/python3.12/site-packages/ldap/functions.py", line 52, in _ldap_function_call
    result = func(*args,**kwargs)
             ^^^^^^^^^^^^^^^^^^^^
ldap.LDAPError: (11, 'Resource temporarily unavailable')
/opt/condaenv/lib/python3.12/site-packages/evaluation_system/model/solr_models/models.py:12: PendingDeprecationWarning: The evaluation_system.model.solr_models module will be removed from v2304.0.0
  warnings.warn(

$ docker logs freva-db 
2024-08-13 09:08:44+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:11.1.6+maria~ubu2204 started.
2024-08-13 09:08:44+00:00 [Note] [Entrypoint]: MariaDB upgrade not required
2024-08-13  9:08:44 0 [Note] Starting MariaDB 11.1.6-MariaDB-ubu2204 source revision 80abd847daf736cf8e57e83241c2e6fed74b4ed3 server_uid iuDyH+u6mEWt9ZoWYIeLR0wIW68= as process 1
2024-08-13  9:08:44 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2024-08-13  9:08:44 0 [Note] InnoDB: Number of transaction pools: 1
2024-08-13  9:08:44 0 [Note] InnoDB: Using AVX512 instructions
2024-08-13  9:08:44 0 [Note] mariadbd: O_TMPFILE is not supported on /tmp (disabling future attempts)
2024-08-13  9:08:44 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB
2024-08-13  9:08:44 0 [Note] InnoDB: Completed initialization of buffer pool
2024-08-13  9:08:44 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes)
2024-08-13  9:08:44 0 [Note] InnoDB: End of log at LSN=261056
2024-08-13  9:08:44 0 [Note] InnoDB: Opened 3 undo tablespaces
2024-08-13  9:08:44 0 [Note] InnoDB: 128 rollback segments in 3 undo tablespaces are active.
2024-08-13  9:08:44 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
2024-08-13  9:08:44 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
2024-08-13  9:08:44 0 [Note] InnoDB: log sequence number 261056; transaction id 424
2024-08-13  9:08:44 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2024-08-13  9:08:44 0 [Note] Plugin 'FEEDBACK' is disabled.
2024-08-13  9:08:44 0 [Note] Plugin 'wsrep-provider' is disabled.
2024-08-13  9:08:44 0 [Note] Server socket created on IP: '0.0.0.0'.
2024-08-13  9:08:44 0 [Note] Server socket created on IP: '::'.
2024-08-13  9:08:44 0 [Note] mariadbd: Event Scheduler: Loaded 0 events
2024-08-13  9:08:44 0 [Note] mariadbd: ready for connections.
Version: '11.1.6-MariaDB-ubu2204'  socket: '/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
2024-08-13  9:08:44 0 [Note] InnoDB: Buffer pool(s) load completed at 240813  9:08:44
2024-08-13  9:10:25 8 [Warning] Aborted connection 8 to db: 'frevadb' user: 'root' host: '***.***.***.***' (Got an error reading communication packets)

[antarcticrainforest]

when we add more than one allowed_hosts, all goes under 'localhost, localhost1, third' which has to be 'localhost', 'localhost1', 'third'

https://github.com/FREVA-CLINT/freva-deployment/blob/main/assets/share/freva/deployment/config/inventory.toml#L344

new update: interesting, actually via changing or adding allowed_hosts in inverntory.toml at the end nothing changes in the ALLOWED_HOSTS in freva-web container. Just my current IP will be added there ...

I guess that's a bug it should be string , separating the hosts.

This needs to be a string, because it gets passed as an env variable to the web container.

[antarcticrainforest]

1. Could you post your config file here?
2. What deploy host you are using?

As you can see from the web error message. The existing error message is not the problem it's because you can establish a connection to the ldap server.

[antarcticrainforest]

So if I see that correctly so far you've found one bug. The allowed host one?

The rest is due to config issues or permissions?

[mo-dkrz]

@antarcticrainforest, as you are going to send another PR here to remove ldap, I will write all of my suggested changes in the description of this issue

[antarcticrainforest]

@antarcticrainforest, as you are going to send another PR here to remove ldap, I will write all of my suggested changes in the description of this issue

Yes after the web PR has been merged.

[antarcticrainforest]

So as far as I can see, was the only remaining issue the hardcoded /opt/conda/freva/bin which has been fixed.

[mo-dkrz]

So as far as I can see, was the only remaining issue the hardcoded /opt/conda/freva/bin which has been fixed.

@antarcticrainforest the description is updated. Please have a look. Thanks

[antarcticrainforest]

I've allowed myself to update the first comments so that I can address them better by numbers:

1. You seem to run the binary from levante. Aside from the fact that I don't fully understand why you are trying to run this from levante the pre-build binary only supports version GLIBC version 2.31+, levante currently runs on 2.28 from 2018. If you want to use this software on levante pip install it.
2. That should be fixed, feel free to correct me if I am wrong.
3. I don't quite follow. data_path shouldn't be part of the core setup. And I assume you're referring to the core setup? Could you please clarify what this issue is?
4. I think this path should have been created. db_data_path and vault_data_path should be interchangeable. Can you confirm this?
5. This should be fixed, by removing the username/password section.
6. From the error message I figure you've set the logo to a directory. To be honest I think that's a fair point of failure.
7. This should be a string as it is passed as an env-variable to the docker container.

[antarcticrainforest]

Ok and update on issue 3.

I think I got what you meant. If you leave the install_dir or data_path blank and use the cmd subcommand you'll get erroneous results.

The TUI should catch the blank install_dir and data_path variables. The cmd subcommand on the other hand should be used if you already have a working config and want to avoid going through the TUI. If on the other hand you don't have a working config, then you should use the TUI. The cmd option can be seen as an expert option - that doesn't pester you with warnings that certain things have to be set etc. And I would like it to keep it that way.

[mo-dkrz]

Thanks for taking time and answering all concerns @antarcticrainforest As you fixed the most important issues and on the other hand, answers are convincing to me, so I think we can close this issue