Free software: GNU Affero General Public License v3+
poetry add celery-aio-pool
pip
& PyPI.orgpip install celery-aio-pool
pip
& GitHubpip install git+https://github.com/the-wondersmith/celery-aio-pool.git
pip
& A Local Copy Of The Repogit clone https://github.com/the-wondersmith/celery-aio-pool.git
cd celery-aio-pool
pip install -e "$(pwd)"
celery-aio-pool
's Provided Patcher (non-preferred)celery_aio_pool
in the same module where your Celery "app" is definedpatch_celery_tracer
utility is called before any other
Celery code is called"""My super awesome Celery app."""
# ...
from celery import Celery
# add the following import
import celery_aio_pool as aio_pool
# ensure the patcher is called *before*
# your Celery app is defined
assert aio_pool.patch_celery_tracer() is True
app = Celery(
"my-super-awesome-celery-app",
broker="amqp://guest@localhost//",
# add the following keyword argument
worker_pool=aio_pool.pool.AsyncIOPool,
)
At the time of writing, Celery does not have
built-in support for out-of-tree pools like celery-aio-pool
, but support should
be included starting with the first non-beta release of Celery 5.3. (note: PR #7880 was merged on 2022-11-15
).
The official release of Celery 5.3 enables the configuration of custom worker pool classes thusly:
Set the environment variable CELERY_CUSTOM_WORKER_POOL
to the name of
your desired worker pool implementation implementation.
package:class
% export CELERY_CUSTOM_WORKER_POOL='celery_aio_pool.pool:AsyncIOPool'
Tell Celery to use your desired pool by specifying --pool=custom
when running your worker instance(s)
% celery worker --pool=custom --loglevel=INFO --logfile="$(pwd)/worker.log"
To verify the pool implementation, examine the output of the celery inspect stats
command:
% celery --app=your_celery_project inspect stats
-> celery@freenas: OK
{
...
"pool": {
...
"implementation": "celery_aio_pool.pool:AsyncIOPool",
...
NOTE: Our preferred packaging and dependency manager is Poetry. Installation instructions can be found here.
Clone the repo and install the dependencies
$ git clone https://github.com/the-wondersmith/celery-aio-pool.git \
&& cd celery-aio-pool \
&& poetry install --sync
Optionally, if you do not have or prefer not to use Poetry, celery-aio-pool
is
fully PEP-517 compliant and can be installed directly by any PEP-517-compliant package
manager.
$ cd celery-aio-pool \
&& pip install -e "$(pwd)"
TODO: Coming Soon™
To run the test suite:
$ poetry run pytest tests/
TODO: Coming Soon™