worker: make celery beat only run scheduler

[slint]

• The previous parameters spawned an actual worker which also run beat/scheduler. Current call only spawns the celery beat service.

Basically changing from celery worker --beat ... to celery beat .... See also https://docs.celeryq.dev/en/stable/userguide/periodic-tasks.html#starting-the-scheduler

[ppanero]

Would this mean that it would consume less resources? right now, with less than 2GB of memory it would crash... If we can run it with 1GB we would have one less worker (true) but in terms of architecture it would be clearer where the components doing a task are.

[ntarocco]

Would this mean that it would consume less resources? right now, with less than 2GB of memory it would crash... If we can run it with 1GB we would have one less worker (true) but in terms of architecture it would be clearer where the components doing a task are.

Just tested, worker with beat:

Only beat:

We can greatly decrease resources if we go with this. At the moment, the defaults are still very high.

[ppanero]

Thanks a lot for testing that!!!! While I do think we would be "wasting a pod", since we have more than enough to do a rollout, if we can save resources... Do you see any benefits of having worker + beat?

[ntarocco]

I am totally ok to with a POD with only beat, it makes a lot of sense. @slint if you can simply maybe update the resources for the workerBeat values, then it is perfect... we could try to request 200Mb of mem, limit 500?

[slint]

Done, memory limits for worker-beat updated to 200/500 Mb

[ntarocco]

Done, memory limits for worker-beat updated to 200/500 Mb

Shall we change also the default values in the invenio/values.yml?

[slint]

Added, sorry had no idea this existed :)