Open calcutec opened 6 years ago
Figured out how to inform the crontab of environment variables. Pointed it to the .bash_profile where they are set.
CRONJOBS = [ (' * source $HOME/.bash_profile;', 'publicity.cron.my_scheduled_job') ]
Works great
Have a look at the README.There's an example for the setting CRONTAB_COMMAND_PREFIX
. You should either define your needed values there or place them manually on the top of crontab via crontab -e
.
I glossed over that. Thanks.
Great project!
I solved it editing the crontab, add this at first line:
SHELL=/bin/bash
And change the line created by crontab library with the command:
* * * * * source /Users/billburton/Projects/djangodigital/dpmvenv/bin/activate && /Users/billburton/Projects/djangodigital/manage.py crontab run 79da9099e73f903adeec54d63e2acd67 # django-cronjobs for djangodigital
Yeah, it seems like I have to add that line SHELL=/bin/bash
manually in crontab. Setting it as a prefix doesn't make it work with or without export
.
One solution to wrap the command after specifying the shell in the settings.py
https://unix.stackexchange.com/a/430478/16814
If use in docker or k8s, SHELL=/bin/bash can not work, my solution is this:
export > /opt/env
CRONJOBS = [
('*/5 * * * * (source /opt/env || true) &&', 'app.crontab.crontab')
]
None of the above worked for me, but it pointed me in the right direction.
carestad's answer in this link was what did the trick for me https://stackoverflow.com/questions/2229825/where-can-i-set-environment-variables-that-crontab-will-use
basically, the crontab doesn't have the env variables in scope, so to do that, we can output our env
into the crontab, that'll make them available for the scripts that need to be executed
in my entrypoint.sh:
if [[ "$RUN_CRON" == *"YES"* ]]; then
service rsyslog start
echo "$(env ; crontab -l)" | crontab -
/etc/init.d/cron start
python /code/manage.py crontab add
fi
then in the settings.py
CRONJOBS = [
('0 0 * * *', 'app.management.commands.test_command.test_command', '>> /code/cron_nightly.logs')
]
@kraiz is right, you can use the CRONTAB_COMMAND_PREFIX
in your settings.py as below.
envs = []
for envkey in os.environ.keys():
envs.append(envkey + '=' + os.environ[envkey])
CRONTAB_COMMAND_PREFIX = 'env $(echo ' + '\n'.join(envs) + ' | xargs)'
This is working, given the environment variables were not modified/added during runtime.
If you are using Ubuntu >=18.0, try with following changes. I am assuming you have successfully integrated django-crontab with your django application. Now, access to your server via ssh and type "crontab -e", You will see your cronjobs are defined here. Add Your Environment Variables just above your cronjob and save the file using CTRL+o & CTRL+x. See the following piece of lines-
DB_NAME = "**" DB_USER = "**" DB_PASSWORD = "" SECRET_KEY = "***"
/1 * /home/sms/sms/env/bin/python /home/sms/sms/sms-backend/website/manage.py crontab run 446aeeebac8c8f82d9e01bb662dc9b21 >> /tmp/scheduled_job.log # django>
Nota Bene: For me, I have 4 variables, Add all your environment variables as it is mentioned and replace all asterisk alpahbet with your information.
For docker/k8s case:
printenv > /app_folder/.env
pip install python-dotenv
from pathlib import Path
import os
from dotenv import dotenv_values
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(file))) ENV_FILE=str(Path(BASE_DIR)) + '/.env' myvars=dotenv_values(ENV_FILE)
4. Reuse .env values across all Django project:
```python
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': myvars['POSTGRES_NAME'],
'USER': myvars['POSTGRES_USER'],
'PASSWORD': myvars['POSTGRES_PASSWORD'],
'HOST': myvars['POSTGRES_HOST'],
'PORT': 5432,
}
}
Had a similar issue with a project but of course i figured it out just about the environment variables you just need to add the CRONTAB_COMMAND_PREFIX before the list of your crontabs. OK basically that tells crontab where to find the environment it needs to work with ie:
CRONTAB_COMMAND_PREFIX = (
'STAGE={whether production or development}'
)
CRONJOBS = [
('{time for cron}', '{your crontab view}'),
]
Solved. Thank you very much for your reply.
On Tue, 20 Sep 2022, 7:13 pm Ayume Francis, @.***> wrote:
Had a similar issue with a project but of course i figured it out just about the environment variables you just need to add the CRONTAB_COMMAND_PREFIX before the list of your crontabs. OK basically that tells crontab where to find the environment it needs to work with ie:
`CRONTAB_COMMAND_PREFIX = ( 'STAGE={whether production or development}' )
CRONJOBS = [ (' *', '{your crontab view}'),
]`
— Reply to this email directly, view it on GitHub https://github.com/kraiz/django-crontab/issues/88#issuecomment-1252336107, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKECZZLT7HWOBDVVQ5YEVMTV7GZ7XANCNFSM4FPCXTDA . You are receiving this because you commented.Message ID: @.***>
For docker/k8s case:
- Export all env set previously in docker cmd in entrypoint.sh or any start script to .env file:
printenv > /app_folder/.env
- Install additional plugin python-dotenv==0.20.0 and add it to requirements.txt
pip install python-dotenv
- Import python-dotenv in settings.py and import values:
from pathlib import Path import os from dotenv import dotenv_values BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) ENV_FILE=str(Path(BASE_DIR)) + '/.env' myvars=dotenv_values(ENV_FILE)
- Reuse .env values across all Django project:
DATABASES = { 'default': { 'ENGINE': 'django.db.backends.postgresql', 'NAME': myvars['POSTGRES_NAME'], 'USER': myvars['POSTGRES_USER'], 'PASSWORD': myvars['POSTGRES_PASSWORD'], 'HOST': myvars['POSTGRES_HOST'], 'PORT': 5432, } }
- No changes required in CRONJOBS
This a good approach and working solution for more complex variables. In my case, I don't use docker but I needed to pass the database configurations, which happens to work unexpectedly using the default approach of CRONTAB_COMMAND_PREFIX. This approach worked for me
echo "$(env ; crontab -l)" | crontab -
I used this command in the docker console, and it really saved my day.
Environment & Versions
Settings
django-crontab
settings:Details
crontab -l
after runningpython manage.py crontab add
.crontab run
command and I copy this command to my terminal the job works: yes/nono
The traceback is below. The issue is that crontab doesn't recognize the os.environ variables i use in settings.py for db connections, aws secret keys, etc. I don't know how to make these available to a cron job and hoped that django-crontab would have a baked-in solution.
The same function works fine when I am running it from the django project itself.
Thanks!
Message 18: From billburton@Jimmys-iMac.local Fri Aug 10 15:05:00 2018 X-Original-To: billburton Delivered-To: billburton@Jimmys-iMac.local From: billburton@Jimmys-iMac.local (Cron Daemon) To: billburton@Jimmys-iMac.local Subject: Cron billburton@Jimmys-iMac /Users/billburton/Projects/djangodigital/dpmvenv/bin/python /Users/billburton/Projects/djangodigital/manage.py crontab run 79da9099e73f903adeec54d63e2acd67 # django-cronjobs for djangodigital X-Cron-Env:
X-Cron-Env:
X-Cron-Env:
X-Cron-Env:
X-Cron-Env:
Date: Fri, 10 Aug 2018 15:05:00 -0400 (EDT)
Traceback (most recent call last): File "/Users/billburton/Projects/djangodigital/manage.py", line 22, in
execute_from_command_line(sys.argv)
File "/Users/billburton/Projects/djangodigital/dpmvenv/lib/python3.6/site-packages/django/core/management/init.py", line 371, in execute_from_command_line
utility.execute()
File "/Users/billburton/Projects/djangodigital/dpmvenv/lib/python3.6/site-packages/django/core/management/init.py", line 317, in execute
settings.INSTALLED_APPS
File "/Users/billburton/Projects/djangodigital/dpmvenv/lib/python3.6/site-packages/django/conf/init.py", line 56, in getattr
self._setup(name)
File "/Users/billburton/Projects/djangodigital/dpmvenv/lib/python3.6/site-packages/django/conf/init.py", line 43, in _setup
self._wrapped = Settings(settings_module)
File "/Users/billburton/Projects/djangodigital/dpmvenv/lib/python3.6/site-packages/django/conf/init.py", line 106, in init
mod = importlib.import_module(self.SETTINGS_MODULE)
File "/Users/billburton/Projects/djangodigital/dpmvenv/lib/python3.6/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 994, in _gcd_import
File "", line 971, in _find_and_load
File "", line 955, in _find_and_load_unlocked
File "", line 665, in _load_unlocked
File "", line 678, in exec_module
File "", line 219, in _call_with_frames_removed
File "/Users/billburton/Projects/djangodigital/djangodigital/settings.py", line 94, in
'HOST': os.environ['RDS_DPM_MAIN_HOSTNAME'],
File "/Users/billburton/Projects/djangodigital/dpmvenv/bin/../lib/python3.6/os.py", line 669, in getitem
raise KeyError(key) from None
KeyError: 'RDS_DPM_MAIN_HOSTNAME'