Documentation for jobs system

[GoogleCodeExporter]

If there is any then I can't find it!

Original issue reported on code.google.com by andybak on 9 Oct 2008 at 5:52

[GoogleCodeExporter]

Original comment by v.oostv...@gmail.com on 15 Oct 2008 at 8:35

• Changed state: Accepted

[GoogleCodeExporter]

Original comment by v.oostv...@gmail.com on 15 Oct 2008 at 10:04

• Added labels: Type-Documentation
• Removed labels: Type-Defect

[GoogleCodeExporter]

Short answer, there is none at the moment :(

A little longer answer:

Use create_jobs to make a 'jobs' directory inside of an application.
Then you can create jobs in much the same way that management commands work.

Some simple examples are provided by the django_extensions.jobs package.

Basically a job is a python script with a mandatory Job class which extends from
HourlyJob, DailyJob, WeeklyJob or MonthlyJob. It has one method that must be
implemented called 'execute', which is called when the job is ran.

The following commands are related to jobs:

 - create_jobs, created the directory structure for jobs
 - runjob, runs a single job
 - runjobs, run all hourly/daily/weekly/monthly jobs

Use "runjob(s) -l" to list all jobs recognized.

Original comment by v.oostv...@gmail.com on 18 Oct 2008 at 5:20

[GoogleCodeExporter]

Here's the bit I didn't understand. Do jobs run automatically via some cron-like
mechanism or do I have to issue regular 'runjobs' command to ensure scheduled 
jobs
are run.

I would have assumed the former (it seems a fairly minimal requirement for a
scheduled jobs system to, you know, er, run scheduled jobs ;) ) but then what 
does
are the runjobs commands for?

I'll admit I've made no effort to look at the code to answer this :(  

Original comment by andybak on 18 Oct 2008 at 9:10

[GoogleCodeExporter]

no jobs do not run "automatically via some cron-like mechanism"
you have to issue "runjobs [hourly][daily][weekly][monthly]" command through 
cron to
make sure that jobs are executed.

Original comment by v.oostv...@gmail.com on 14 Nov 2008 at 7:10

[GoogleCodeExporter]

In that case - why don't I just run the jobs themselves through cron? I don't 
quite
see what value the jobs system actually adds! 

Original comment by andybak on 17 Nov 2008 at 3:36

[GoogleCodeExporter]

Because i hate (micro)management. Running multiple custom Django sites on 
multiple
machines there are a lot of site specific little cronjobs. Ranging from filling 
the
database with feeds like twitter/rss to custom database aggregation, 
calculations,
invalidation caches and so on.

Now onces so often sites transfer from one server to another, or a server goes 
down
and the sites have to be reinstalled on a different server. At that point one 
of the
server-admins has to install all those cronjobs back into the crontab. This is 
either
forgotten or not done properly since it's not always clear which jobs have to go
where and in which time segments.

Thus having one system which is the same for every deployments helps to ease 
thing a
lot. It's easier for developers and easier for administrators. It forces 
developers
to clean up there mess and disable old scripts and sysadmins never forgot that
./manage.py cleanup line which floods the database with sessions (or any other 
lose
cronjob)

Now this was my personal interest but surely other people have other needs and 
a more
improved/generalized and more flexible jobs framework is still on the table. 

I wish we could have some discussion on the mailinglist and/or wiki to 
streamline
what people need/want and where something like django-jobs (if we split it of 
from
django-extensions) should go in the future.

Original comment by v.oostv...@gmail.com on 18 Nov 2008 at 6:15

[GoogleCodeExporter]

OK. That makes sense. My next question is the significance of runjobs 'hourly'
'monthly' etc. If I set a cronjob 'runjobs monthly' but I set the cronjob to run
hourly - how often will the jobs run. Hourly or monthly!?

Original comment by andybak on 19 Nov 2008 at 3:40

[GoogleCodeExporter]

Hourly, those names are like labels and do not enforce time-to-run by themselfs.
Normally i have:

@hourly  /var/www/site/app/manage.py runjobs hourly
@daily   /var/www/site/app/manage.py runjobs daily
@weekly  /var/www/site/app/manage.py runjobs weekly
@monthly /var/www/site/app/manage.py runjobs monthly

So that those labels make sense. Having a system where the labels actually 
enforce
the time when jobs are run imposes a lot of extra questions about how to handle 
things.

Though a daemon, a djangocron so to speak, would not be very difficult to write.
./manage.py runjobs --daemon which creates a processes that watches time and 
once in
a while executes jobs. This could also allow for fine-grained scheduling and 
even
something like 'at' scheduling. (Where you could have a trigger, let's say a 
signal,
that schedules some job to run 5 minutes from now.)

Anyways these are the kind-of things i would like to see some discussion about, 
so i
know what people need and would like in future versions.

Original comment by v.oostv...@gmail.com on 20 Nov 2008 at 7:04

[GoogleCodeExporter]

Have you looked at django-cron? There appears to be some overlap and their 
cron-less
approach could be useful for people on shared hosts.

Original comment by andybak on 20 Nov 2008 at 8:36

[GoogleCodeExporter]

The jobs part should really be broken out of django-extensions.
As it is a complete app in it's own right.

There are several things that I don't like in the django-cron 
approach/code-base, but
it does have some good points.

I don't know how many people are actually using the jobs in django-extensions 
and
important a feature like that is to users/developers.

If there is enough interest from the community I might be willing to find some 
time
and work on a (stand-alone) (production quality) successor.

Original comment by v.oostv...@gmail.com on 16 Apr 2009 at 11:05

[GoogleCodeExporter]

Is anybody still interested in continuing this further ?

Original comment by v.oostv...@gmail.com on 11 May 2009 at 8:41