Fix Docker image with new version requiring JS

[ketsapiwiq]

Fixes #45

• upgraded base image to bookworm: we could also install nodejs 18 on bullseye with something along the lines of

  curl https://nodejs.org/dist/v18.17.1/node-v18.17.1-linux-x64.tar.xz
  tar -xJvf node-v18.17.1-linux-x64.tar.xz -C /usr/local/lib/nodejs 

• maybe in the long term we want to devise a way to cleverly use multi-stage builds to have a more readable Dockerfile. But I ended up getting frustrated on thinking about which files need to be copied from where. Ideally I'd just "squash" some build steps of the image. See also this clever trick with RUN --mount: https://github.com/moby/moby/issues/34565#issuecomment-778223790

[saschafoerster]

I get this error with this Dockerfile:

pretalx  | 2023-09-12 19:59:55,503 INFO success: pretalxtask entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
pretalx  | 2023-09-12 20:00:03,926 WARN exited: pretalxtask (exit status 1; not expected)
pretalx  | 2023-09-12 20:00:04,930 INFO spawned: 'pretalxtask' with pid 61

Logs:

TypeError: UnixDomainSocketConnection.__init__() got an unexpected keyword argument 'username'
ERROR 2023-09-12 20:00:08,870 django.dispatch dispatcher Error calling refetch_gravatars in Signal.send_robust() (UnixDomainSocketConnection.__init__() got an unexpected keyword argument 'username')
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/kombu/utils/functional.py", line 32, in __call__
    return self.__value__
AttributeError: 'ChannelPromise' object has no attribute '__value__'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/kombu/transport/virtual/base.py", line 951, in create_channel
    return self._avail_channels.pop()
IndexError: pop from empty list

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/redis/connection.py", line 1048, in get_connection
    connection = self._available_connections.pop()
IndexError: pop from empty list

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/django/dispatch/dispatcher.py", line 211, in send_robust
    response = receiver(signal=self, sender=sender, **named)
  File "/pretalx/src/pretalx/person/tasks.py", line 60, in refetch_gravatars
    gravatar_cache.apply_async(args=(user.pk,))
  File "/usr/local/lib/python3.10/site-packages/celery/app/task.py", line 594, in apply_async
    return app.send_task(
  File "/usr/local/lib/python3.10/site-packages/celery/app/base.py", line 798, in send_task
    amqp.send_task_message(P, name, message, **options)
  File "/usr/local/lib/python3.10/site-packages/celery/app/amqp.py", line 517, in send_task_message
    ret = producer.publish(
  File "/usr/local/lib/python3.10/site-packages/kombu/messaging.py", line 186, in publish
    return _publish(
  File "/usr/local/lib/python3.10/site-packages/kombu/connection.py", line 556, in _ensured
    return fun(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/kombu/messaging.py", line 195, in _publish
    channel = self.channel
  File "/usr/local/lib/python3.10/site-packages/kombu/messaging.py", line 218, in _get_channel
    channel = self._channel = channel()
  File "/usr/local/lib/python3.10/site-packages/kombu/utils/functional.py", line 34, in __call__
    value = self.__value__ = self.__contract__()
  File "/usr/local/lib/python3.10/site-packages/kombu/messaging.py", line 234, in <lambda>
    channel = ChannelPromise(lambda: connection.default_channel)
  File "/usr/local/lib/python3.10/site-packages/kombu/connection.py", line 953, in default_channel
    self._ensure_connection(**conn_opts)
  File "/usr/local/lib/python3.10/site-packages/kombu/connection.py", line 459, in _ensure_connection
    return retry_over_time(
  File "/usr/local/lib/python3.10/site-packages/kombu/utils/functional.py", line 318, in retry_over_time
    return fun(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/kombu/connection.py", line 934, in _connection_factory
    self._connection = self._establish_connection()
  File "/usr/local/lib/python3.10/site-packages/kombu/connection.py", line 860, in _establish_connection
    conn = self.transport.establish_connection()
  File "/usr/local/lib/python3.10/site-packages/kombu/transport/virtual/base.py", line 975, in establish_connection
    self._avail_channels.append(self.create_channel(self))
  File "/usr/local/lib/python3.10/site-packages/kombu/transport/virtual/base.py", line 953, in create_channel
    channel = self.Channel(connection)
  File "/usr/local/lib/python3.10/site-packages/kombu/transport/redis.py", line 741, in __init__
    self.client.ping()
  File "/usr/local/lib/python3.10/site-packages/redis/client.py", line 1106, in ping
    return self.execute_command('PING')
  File "/usr/local/lib/python3.10/site-packages/redis/client.py", line 836, in execute_command
    conn = self.connection or pool.get_connection(command_name, **options)
  File "/usr/local/lib/python3.10/site-packages/redis/connection.py", line 1050, in get_connection
    connection = self.make_connection()
  File "/usr/local/lib/python3.10/site-packages/redis/connection.py", line 1088, in make_connection
    return self.connection_class(**self.connection_kwargs)
TypeError: UnixDomainSocketConnection.__init__() got an unexpected keyword argument 'username'

[rixx]

Actually, could you check out the errors @saschafoerster mentioned, before I merge this PR?

[ketsapiwiq]

Hi! Ok I'll admit I just checked if the Docker would build and didn't check if the docker compose would work well.

I'm a bit lost on this error.

It seems it has something to do with refetch_gravatars or maybe with Redis and "Kombu"?

I'll try to check if it has something to do with it being in Debian 11 instead of Debian 10.

[rixx]

That would be good, thank you!

Should I merge this in the meantime to have a working image at all, or should I wait for the compose setup to work? Either way is fine for me (and I'm also happy to add anybody willing as contibutor).

[ketsapiwiq]

I guess it's useless without a working docker compose, so let's keep it that way, except if we're really stuck on this error and then we might open an issue on the main pretalx repo to see if we can get help :)

[rixx]

I'm rather confused by the whole Dockerfile tbh – but that may be because I don't know Docker at all (this is why I'll never commit to maintaining this repo / pretalx-docker in general): It installs both mysql and postgres libraries, it pins redis to a specific version (rather than using pretalx[postgres], pretalx[redis], pretalx[mysql]). As this problem is with the connection to redis, it may be sufficient to just remove this pinned version and use pretalx[redis] instead.

[ketsapiwiq]

I think you're right, this pinned redis version is puzzling. I added a commit removing it, but after a very quick test (just re-upping the docker compose [1]) I still can't have a reachable pretalx instance after that, so I fear more work might be needed.

As for the rationale behind installing both mysql and postgres libraries I can help: the goal of a Docker image is to be versatile as long as it doesn't make the Docker image too heavy.

I didn't have time to test it on Bullseye rather than Bookworm yet. To me that's the next logical step.

[1]: note: sometimes you have to docker compose restart pretalx because it tried reaching the db before it's up, a nice add to this repo would be a little waiting loop in deployment/docker/pretalx.bash

[rixx]

I still feel like the correct method would be to install pretalx[redis]. For example, pretalx used to pull in django-redis, which is still installed here, but it doesn't anymore (as Django has built-in redis support now). There's no reason for the Docker repo to go out of sync with production requirements like that, when it could just install the pretalx-specified dependencies. Same for MySQL and postgres: pretalx also provides pretalx[mysql] and pretalx[postgres].

[ketsapiwiq]

Oh ok sorry it's me that didn't get that this was not a notation but real packages (pretalx[mysql]), I don't know enough python to package all this haha! Will try soon.

[rixx]

Yeah, no worries at all! It's a way in Python to specify variants to packages that include optional dependencies. You can combine them with commas, like pip install pretalx[mysql,postgres,redis].

[ketsapiwiq]

In the end the issue was the binding to a localhost interface: in a docker container that doesn't allow the app to be reached out (no security issue here, 0.0.0.0 is just how you abstract the network when creating a Docker image). It should work now. For the pretalx[mysql] and pretalx[postgres] I tried pip3 install -e /pretalx/[mysql,postgres,redis] and I'm surprised it's working! The Docker image still installs memcached python libs, this should be an optional dependency or just dropped, right?

[rixx]

Memcached can be used rather than redis for caching, so installing it seems in line with installing all the database support etc.

Also: nice work, thank you very much. Just to confirm, I can merge this PR now?

[ketsapiwiq]

Yes, LGTM! :)

Message ID: @.***>

[rixx]

Automatic push seems to have worked, no bit rot there!

[saschafoerster]

Upgrading with v2023.1.3 worked.