bkellenb
commented
2 years ago Hello!
Thank you for opening the issue. I believe I have been able to replicate the error; it lies in the way Celery is invoked in the systemd service: /bin/sh -c '${CELERY_BIN} … cannot work because ${CELERY_BIN} is a Python file and not a shell script. I checked my logs and got the same error, but did not find them before, because curiously enough the Celery daemon worked nonetheless in my tests.
The quickest solution for you would be to do the following:
- Log in as root:
sudo -s - Execute these lines, one by one:
sed -i "s/\/bin\/sh -c '/\/home\/azureuser\/anaconda3\/envs\/aide\/bin\/python /g" /etc/systemd/system/aide-worker.service sed -i "s/'$//g" /etc/systemd/system/aide-worker.service systemctl daemon-reload service aide-worker stop service aide-worker start
Sanity check: your systemd service file should now look like this:
[Unit]
Description=Celery Service for AIDE AIWorker
After=network.target
After=rabbitmq-server.service
After=redis.service
After=postgresql.service
[Service]
Type=forking
User=aide_celery
Group=aide
EnvironmentFile=/etc/default/celeryd_aide
WorkingDirectory=/home/azureuser/aerial_wildlife_detection
ExecStart=/home/azureuser/anaconda3/envs/aide/bin/python ${CELERY_BIN} -A $CELERY_APP multi start $CELERYD_NODES --pidfile=${CELERYD_PID_FILE} --logfile=${CELERYD_LOG_FILE} --loglevel="${CELERYD_LOG_LEVEL}" $CELERYD_OPTS
ExecStop=/home/azureuser/anaconda3/envs/aide/bin/python ${CELERY_BIN} multi stopwait $CELERYD_NODES --pidfile= --loglevel="${CELERYD_LOG_LEVEL}"
ExecReload=/home/azureuser/anaconda3/envs/aide/bin/python ${CELERY_BIN} -A $CELERY_APP multi restart $CELERYD_NODES --pidfile=${CELERYD_PID_FILE} --logfile=${CELERYD_LOG_FILE} --loglevel="${CELERYD_LOG_LEVEL}" $CELERYD_OPTS
Environment=AIDE_CONFIG_PATH=/home/azureuser/aerial_wildlife_detection/config/settings.ini
Environment=AIDE_MODULES=LabelUI,AIController,AIWorker,FileServer
Environment=PYTHONPATH=/home/azureuser/aerial_wildlife_detection
Restart=always
[Install]
WantedBy=multi-user.targetEssentially, the /bin/sh -c should be replaced with the path to the correct Python executable and all the single quotes removed.
Also, an invoke of journalctl -xe should indicate that there is no more "is a directory" errors after restarting the worker daemon process.
The rest looks fine (although note that environment variable CELERYBEAT_PID_FILE is specified twice in your environment file).
Please let me know if this helped. I will update the installer script with the next commit.
ryan-ntt
Hi all,
I'd appreciate some assistance if possible configuring the Celery worker service. I'm experiencing an issue where Celery is passing the project working directory as a variable into a pidfile checking method in multi.py, which is subsequently raising an error. The full traceback is below:
Our environment file /etc/default/celeryd_aide is as follows:
The systemd service file is as follows:
I've done some searching for this issue to see if it is a bug with Celery itself but haven't found anything decisive. My main experience with Celery is through Django and I often have very simple Celery app definitions. My OS is Ubuntu 20.04.3 and I'm using commit 087aa40bf4938346b4bb5c98038d4027e70c8b53.
Package versions:
Any pointers would be much appreciated