MediaCMS on Kubernetes

[xico42]

Describe the issue I am trying to deploy MediaCMS with Kubernetes. My first attempt is using a "single server" approach. So both WEB and celery workers are enabled in a single container.

I have managed to test the web application and uploaded media. However, the encoding never finishes.

Taking a look at supervisor services, the celery long and short workers always exit early:

supervisor> status
celery_beat                      RUNNING   pid 421, uptime 0:01:26
celery_long                      EXITED    Feb 14 06:34 PM
celery_short                     EXITED    Feb 14 06:22 PM
nginx                            RUNNING   pid 51, uptime 1:25:39
uwsgi                            RUNNING   pid 50, uptime 1:25:39

I haven't found any useful logs at logs/celery_long.log and logs/celery_short.log but will place the contents here:

logs/celery_long.log:

[2022-02-14 17:11:30,793: INFO/MainProcess] Connected to redis://redis-master:6379/1
[2022-02-14 17:11:31,328: INFO/MainProcess] mingle: searching for neighbors
[2022-02-14 18:34:53,283: INFO/MainProcess] Connected to redis://redis-master:6379/1
[2022-02-14 18:34:53,332: INFO/MainProcess] mingle: searching for neighbors

The redis instance is up, and there are keys in the connected DB (1):

127.0.0.1:6379[1]> select 1
OK
127.0.0.1:6379[1]> keys *
 1) "_kombu.binding.short_tasks"
 2) "long_tasks"
 3) "_kombu.binding.long_tasks"
 4) "_kombu.binding.celery.pidbox"
 5) "_kombu.binding.celeryev"
 6) "_kombu.binding.celery"
 7) ":1:django.contrib.sessions.cachenvvg8380gd0uuf79yu9il14emk2fccwy"
 8) "long_tasks\x06\x169"
 9) "short_tasks"
10) "_kombu.binding.reply.celery.pidbox"
127.0.0.1:6379[1]> 

Any idea on how to debug this?

[xico42]

I've also tried to execute the celery worker manually, but it stills exits early:

/home/mediacms.io/mediacms# /home/mediacms.io/bin/celery multi start long1 --pidfile=/var/run/mediacms/42.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/celery_long.log -Ofair --prefetch-multiplier=1 -Q long_tasks
celery multi v4.4.7 (cliffs)
> Starting nodes...
    > long1@mediacms-web-6cd64f5b86-k66v4: OK

[mgogoulos]

Hi @fcoedno , can you try out the Docker compose version rather than the single version? https://github.com/mediacms-io/mediacms/blob/main/docs/admins_docs.md#3-docker-installation Would be interesting if you can include your methodology. Also @swiftugandan could provide useful feedback here.

Thanks

[mgogoulos]

Changed the title, as I don't think it's that much related with Celery rather than how this is tested against Kubernetes environment.

[swiftugandan]

@fcoedno @mgogoulos looks like we have a kubernetes deployment here - https://youtu.be/MnvO2OMs5e4

[mgogoulos]

@fcoedno @mgogoulos looks like we have a kubernetes deployment here - https://youtu.be/MnvO2OMs5e4

Thanks for the link @swiftugandan ! I saw the video, and have placed a comment if the configuration is available.