Low Priority Queue

[GoogleCodeExporter]

We should have a low priority que that we can use to run large non time 
sensative jobs in a way that doesn't interfere with time sensitive jobs.

This could be implemented as a sperate channel that jobs can submit tasks to 
and from which nodes can grab tasks from if no tasks come through on the high 
priority channel before a time out occurs.

In the future we may wish to have nodes kill low priority tasks that are taking 
a long time if a high priority job comes in. 

Original issue reported on code.google.com by ryanbres...@gmail.com on 26 Jul 2012 at 8:52

[GoogleCodeExporter]

Original comment by hrov...@systemsbiology.org on 27 Jul 2012 at 9:42

• Changed title: Low Priority Queue

[GoogleCodeExporter]

I think we should support ~2 discrete levels of priority (background and 
production) with distinct behaviors.

Production jobs get a fair share of the available resources as soon as they 
become available.

Background jobs are slowly ramped up as the cluster lies idle. Their rate of 
submission may be related to how long it takes them to finish or their may be a 
maximum time before they are killed or we may even consider kill -STOP and kill 
-CONT int them as high priority jobs come in.

Original comment by ryanbres...@gmail.com on 28 Jul 2012 at 12:06

• Added labels: Priority-High
• Removed labels: Priority-Medium

[GoogleCodeExporter]

+Hector

Original comment by ryanbres...@gmail.com on 28 Jul 2012 at 12:07