Rethink the way we monitor external builds and synchronise builds in Jenkins/PBS

[kinow]

Currently we are tying jobs submitted from within Jenkins. But if jobs in Jenkins fail or are restarted, the job in PBS Torque may still be running.

We could copy the behaviour of the workflow, durable task or monitor external builds. Some links that may help.

• http://developer-blog.cloudbees.com/2014/03/support-for-long-running-builds-in.html
• https://wiki.jenkins-ci.org/display/JENKINS/Monitoring+external+jobs
• http://docs.huihoo.com/javadoc/jenkins/1.512/hudson/model/class-use/Queue.FlyweightTask.html
• https://github.com/jenkinsci/async-job-plugin/blob/6d3640045aeec169e8de8092f23a93f6f5fd15ab/src/test/java/org/jenkinsci/plugins/asyncjob/TestAsyncJob.java
• https://groups.google.com/forum/#!msg/jenkinsci-dev/nLOBkO8ttVM/ZUytb1qQxT4J