MazamaScience / PWFSLSmoke

Utilities for working with air quality monitoring data (e.g. federal regulatory monitors)
http://mazamascience.github.io/PWFSLSmoke/
GNU General Public License v3.0
19 stars 5 forks source link

practice with cron #120

Open jonathancallahan opened 3 years ago

jonathancallahan commented 3 years ago

cron and crontabs are a fundamental part of any UINIX system and allow you to schedule commands to be run.

This task is just to become familiar with cron and practice editing a crontab.

Here is a stupidly simple crontab entry that works on my computer. You will have to change the absolute path:

# Simplest example
* * * * * echo "Hello world" >> /Users/jonathancallahan/hello_world.txt

After running for 4 minutes I see:

% cat /Users/jonathancallahan/hello_world.txt
Hello world
Hello world
Hello world
Hello world

Note that the cron daemon doesn't necessarily have the same environment variables as you do, especially PATH. So it's always good to use absolute paths.

Also note that cron entries cannot use a line continuation character.

After you get some simple cron jobs to run, you can upgrade to something a little more complicated. Here are a few lines from a crontab that include an extended line running an _exec.R script inside a docker container.

# AirNow 'daily' runs once daily in the wee hours ______________________________

29 02 * * *    cp -f /data/monitoring/latest/RData/airnow_PM2.5_latest45.RData /data/monitoring/AirNow/RData/latest
30 02 * * *    docker run --rm -v /home/monitoring/Projects/monitoring-data-ingest-v4:/monitoring/v4 -v /data/monitoring:/monitoring/v4/data -w /monitoring/v4 monitoring-data-ingest-v4:latest-2018 /monitoring/v4/airnow_createDailyMonitorObjects_exec.R --outputDir=/monitoring/v4/data/latest/RData --logDir=/monitoring/v4/data/latest/logs

When running docker, chunks like -v /data/monitoring:/monitoring/v4/data mean "mount /data/monitoring/ on the host machine as /monitoring/v4/data/ inside the docker container."

Outside of a docker container, that last crontab entry would look like this:

30 02 * * *    /home/monitoring/Projects/monitoring-data-ingest-v4/airnow_createDailyMonitorObjects_exec.R --outputDir=/data/monitoring/latest/RData --logDir=/data/monitoring/latest/logs