search

umap-project / umap

uMap lets you create maps with OpenStreetMap layers in a minute and embed them in your site.
https://umap-project.org
Other
1.16k stars 220 forks source link

Docker container crashed with version 2.4.x #1947

Closed geimist closed 2 weeks ago

geimist commented 3 months ago

Describe the bug If I use the Docker image >= 2.4, the container does not start successfully. With the 2.3.1 image, the container starts as usual.

Have I overlooked a necessary adjustment?

container log (Lines in the log are sorted from new to old) ``` > RuntimeError: Max post-process passes exceeded. > raise processed > File \"/venv/lib/python3.11/site-packages/umap/storage.py\", line 46, in post_process > for original_path, processed_path, processed in processor: > File \"/venv/lib/python3.11/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py\", line 148, in collect > ^^^^^^^^^^^^^^ > collected = self.collect() > File \"/venv/lib/python3.11/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py\", line 209, in handle > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > output = self.handle(*args, **options) > File \"/venv/lib/python3.11/site-packages/django/core/management/base.py\", line 459, in execute > self.execute(*args, **cmd_options) > File \"/venv/lib/python3.11/site-packages/django/core/management/base.py\", line 413, in run_from_argv > self.fetch_command(subcommand).run_from_argv(self.argv) > File \"/venv/lib/python3.11/site-packages/django/core/management/__init__.py\", line 436, in execute > utility.execute() > File \"/venv/lib/python3.11/site-packages/django/core/management/__init__.py\", line 442, in execute_from_command_line > management.execute_from_command_line() > File \"/venv/lib/python3.11/site-packages/umap/bin/__init__.py\", line 9, in main > ^^^^^^ > sys.exit(main()) > File \"/venv/bin/umap\", line 8, in > Traceback (most recent call last): > Error with file All > Loaded local config from /etc/umap/umap.conf ```

Additional context Installed at Synology DSM 7.2 (Container Manager)

RichardHausgartner commented 2 months ago

I just want to conform @geimist bug issue. I run 2.3.1 and the docker update fails to: -> 2.4.0 -> 2.4.2

(2.4.1 untested, but it seems the same problem)

yohanboniface commented 2 months ago

I'm not exactly sure why this happens, as we deployed with Docker on our ANCT instance without issue, and I can run it locally with docker compose, but the error message seems the same as the one we hit recently, and fixed in master:

https://github.com/umap-project/umap/commit/891f04656f556f8873565ddb003fe626ea084799

We'll deploy a new version including this fix this week, let's see if it fixes your issue to!

geimist commented 2 months ago

Unfortunately, the problem still exists with version 2.5.1: 🙁

2024/07/15 14:42:15,stdout,Loaded local config from /etc/umap/umap.conf
2024/07/15 14:42:27,stdout,Error with file All
2024/07/15 14:42:27,stderr,Traceback (most recent call last):
2024/07/15 14:42:27,stderr,"  File \"/venv/bin/umap\", line 8, in <module>
2024/07/15 14:42:27,stderr,    sys.exit(main())
2024/07/15 14:42:27,stderr,             ^^^^^^
2024/07/15 14:42:27,stderr,"  File \"/venv/lib/python3.11/site-packages/umap/bin/__init__.py\", line 9, in main
2024/07/15 14:42:27,stderr,    management.execute_from_command_line()
2024/07/15 14:42:27,stderr,"  File \"/venv/lib/python3.11/site-packages/django/core/management/__init__.py\", line 442, in execute_from_command_line
2024/07/15 14:42:27,stderr,    utility.execute()
2024/07/15 14:42:27,stderr,"  File \"/venv/lib/python3.11/site-packages/django/core/management/__init__.py\", line 436, in execute
2024/07/15 14:42:27,stderr,    self.fetch_command(subcommand).run_from_argv(self.argv)
2024/07/15 14:42:27,stderr,"  File \"/venv/lib/python3.11/site-packages/django/core/management/base.py\", line 413, in run_from_argv
2024/07/15 14:42:27,stderr,"    self.execute(*args, **cmd_options)
2024/07/15 14:42:27,stderr,"  File \"/venv/lib/python3.11/site-packages/django/core/management/base.py\", line 459, in execute
2024/07/15 14:42:27,stderr,"    output = self.handle(*args, **options)
2024/07/15 14:42:27,stderr,             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024/07/15 14:42:27,stderr,"  File \"/venv/lib/python3.11/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py\", line 209, in handle
2024/07/15 14:42:27,stderr,    collected = self.collect()
2024/07/15 14:42:27,stderr,                ^^^^^^^^^^^^^^
2024/07/15 14:42:27,stderr,"  File \"/venv/lib/python3.11/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py\", line 148, in collect
2024/07/15 14:42:27,stderr,"    for original_path, processed_path, processed in processor:
2024/07/15 14:42:27,stderr,"  File \"/venv/lib/python3.11/site-packages/umap/storage.py\", line 50, in post_process
2024/07/15 14:42:27,stderr,    raise processed
2024/07/15 14:42:27,stderr,RuntimeError: Max post-process passes exceeded.
date,stream,content
umap-app-1
davidbgk commented 2 months ago

@geimist 😞

It appears that you're hitting this limit which raises that issue.

Do you have a way to bump that limit here within your container prior to (re)run the collectstatic command? Let's say max_post_process_passes = 50 to see if it fixes the deployment and then we'll refine the value.

geimist commented 2 months ago

I think I need support for this. Since the container cannot be started, I would probably have to adapt the image. Or can I adjust this in the configuration or as an environment parameter?

davidbgk commented 2 months ago

Indeed that's not trivial and I probably have to release a dedicated image to test.

By the way, do you have/set custom statics for your instance?

geimist commented 2 months ago

By the way, do you have/set custom statics for your instance?

No, not to my knowledge. In the django-settings.py it says: STATICFILES_DIRS = [] # May be extended when using UMAP_CUSTOM_STATICS

yohanboniface commented 2 months ago

Can you confirm this is fixed with 2.5.1 image ?

geimist commented 2 months ago

I had already written about this above:

Unfortunately, the problem still exists with version 2.5.1: 🙁

yohanboniface commented 2 months ago

Oops, sorry! Back from holidays, I've missed some notifications :p

2024/07/15 14:42:27,stdout,Error with file All

This "All" file (which comes from our logs) is still a mystery to me.

Let's maybe make entrypoint.sh's collectstatic verbose in next release ?

anphex commented 1 month ago

I am chipping in. I am still having the exact same issue. Here is my compose:

version: '3'
services:
  db:
    # check https://hub.docker.com/r/postgis/postgis to see available versions
    image: postgis/postgis:16-master
    environment:
      - POSTGRES_HOST_AUTH_METHOD=trust
    volumes:
      - umap_db:/var/lib/postgresql/data

  app:
    # Check https://hub.docker.com/r/umap/umap/tags to find the latest version
    image: umap/umap:2.5.1
    ports:
      # modify the external port (8001, on the left) if desired, but make sure it matches SITE_URL, below
      - "8055:8000"
    environment:
      - DATABASE_URL=postgis://postgres@db/postgres
      - SITE_URL=https://umap.XXXX.de:8055/
      - STATIC_ROOT=/srv/umap/static
      - MEDIA_ROOT=/srv/umap/uploads
    volumes:
      - umap_userdata:/srv/umap/uploads
      # FIX the path on the left, below, to your location 
      # OPTIONAL, you can comment the line below out for default
      # values to apply
      - /home/umap/umap.conf:/etc/umap/umap.conf
    restart: always
    depends_on:
      - db

volumes:
  umap_userdata:
  umap_db:

Both containers start up fine, but the app container then keeps throwing: 

 Loaded local config from /etc/umap/umap.conf
Error with file All
Traceback (most recent call last):
  File "/venv/bin/umap", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/venv/lib/python3.11/site-packages/umap/bin/__init__.py", line 9, in main
    management.execute_from_command_line()
  File "/venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/venv/lib/python3.11/site-packages/django/core/management/base.py", line 413, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/venv/lib/python3.11/site-packages/django/core/management/base.py", line 459, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 209, in handle
    collected = self.collect()
                ^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 148, in collect
    for original_path, processed_path, processed in processor:
  File "/venv/lib/python3.11/site-packages/umap/storage.py", line 50, in post_process
    raise processed
RuntimeError: Max post-process passes exceeded.

My "hunch" is that the umap.conf file I created empty needs to contain real default settings at init? But I couldn't find any reference to this, so I just kept it empty with #placeholder as comment and started it up.

yohanboniface commented 2 weeks ago

New Docker image released today, with a few fixes related to Docker, could have a try with 2.6.1 ? :crossed_fingers:

geimist commented 2 weeks ago

Great, it works perfectly. Thank you very much for your excellent job!

RichardHausgartner commented 2 weeks ago

New Docker image released today, with a few fixes related to Docker, could have a try with 2.6.1 ? 🤞

I just want confirm, that the Docker image upgrade 2.3.1 -> 2.6.3 works! Thank you very much!

davidbgk commented 2 weeks ago

Yay!