PBS Torque scheduler support requested

[somalee]

Currently the software doesn't support PBS Torque scheduler.

[kotoroshinoto]

This isn't really the fault of Hugeseq itself, but rather hpc-sjm and its not THAT hard to build in libdrmaa support for torque if you know how to program in C/C++ (I've done it once, I'll see if I can scrounge up the code as we switched to SGE and i'm not using that version now).

Even if I don't have the code on hand I could probably do it again, but that'll have to wait until I have some time. An important distinction is to have a class for Torque[libDRMAA] and Torque[libtorque] because if you have torque with libDRMAA hooks, that interface is much easier to program for than libtorque, and until we have a working libtorque class, the versions of torque that have been mixed & matched with other software in commercial cluster managers may not have been built with libdrmaa support, and will require libtorque support.

It only requires adding in one class, editing the files used to generate the configure script and makefile and editing the code to use your class when the configure script is called with your options. This SOUNDS like a lot but only amounts to a few lines outside of the class that you implemented.

If your instance of torque doesnt have libdrmaa built in you're going to have a harder time, because the libtorque interface uses a poorly documented linked-list structure to store its submission options (poorly documented in the sense that the documentation doesnt tell you the minimum required options, the expected order of options, and the defaults that should be applied when no values are given, it DOES tell you quite clearly what member variables are in the stucture.)

edit-- I do have the code, though it does appear to be a tad sloppy, and it only worked with torque that supports libdrmaa, which the clusters i have access to no longer use (thus I can't test it)

[somalee]

Dear kotoroshinoto: Thank you for your feedback. Yes, we need the underlying SJM to support PBS Torque. Beside our current resource issues, we don't have PBS Torque on our cluster. Setting up a VM instance is on the to-do list but low in priority. One of our industry partners have expressed interest in adding support. Another of our NIH collaborators have offered to provide testing environment. Once there is a concrete proposal, I will update.

[kotoroshinoto]

I can send you a copy of the source that I had produced that did appear to function (in pbs torque that was built with libdrmaa library support at least).

It will at least give them a starting point. The C code itself was fairly sound but the configure.ac and makefile.am files could have used some cleanup

[somalee]

That would be great.

[dmwebber]

@kotoroshinoto We are installing the HugeSeq pipeline on a cluster with a Torque - Maui scheduler. If it is not too much trouble, could we also borrow a copy of the code that you used to run the pipeline on your cluster? This would be a huge help.

[kotoroshinoto]

First, you have to determine if the torque binarires on your cluster were compiled with libdrmaa support.

If they weren't we're going to have to either ask your IT team to re-compile them WITH that support turned on or learn how to use libtorque's linked-list style option-storing data structure. The rest of the SJM setup wasn't that hard, but I had a very difficult time with that, since they didn't document what the server expected to recieve as a minimum transaction, nor whether it expected a specific order to the elements. (logic dictates that it shouldn't , but implementations often break logic)

I initially tried llibtorque implementation, but since it proved troublesome, and the option was available, I went with libdrmaa support (as it was mostly the same as the SGE implementation.)

M. Gooch On 5/2/2013 12:31 PM, dmwebber wrote:

@kotoroshinoto https://github.com/kotoroshinoto We are installing the HugeSeq pipeline on a cluster with a Torque - Maui scheduler. If it is not too much trouble, could we also borrow a copy of the code that you used to run the pipeline on your cluster? This would be a huge help.

— Reply to this email directly or view it on GitHub https://github.com/StanfordBioinformatics/HugeSeq/issues/1#issuecomment-17348896.

[dmwebber]

This is perfect! Thanks so much for the guidance. This will start me in the right direction as I work with the systems administrator to setup the pipeline. Thanks

From: kotoroshinoto [notifications@github.com] Sent: Friday, May 03, 2013 1:11 PM To: StanfordBioinformatics/HugeSeq Subject: Re: [HugeSeq] PBS Torque scheduler support requested (#1)

First, you have to determine if the torque binarires on your cluster were compiled with libdrmaa support.

If they weren't we're going to have to either ask your IT team to re-compile them WITH that support turned on or learn how to use libtorque's linked-list style option-storing data structure. The rest of the SJM setup wasn't that hard, but I had a very difficult time with that, since they didn't document what the server expected to recieve as a minimum transaction, nor whether it expected a specific order to the elements. (logic dictates that it shouldn't , but implementations often break logic)

I initially tried llibtorque implementation, but since it proved troublesome, and the option was available, I went with libdrmaa support (as it was mostly the same as the SGE implementation.)

M. Gooch

On 5/2/2013 12:31 PM, dmwebber wrote:

@kotoroshinoto https://github.com/kotoroshinoto We are installing the HugeSeq pipeline on a cluster with a Torque - Maui scheduler. If it is not too much trouble, could we also borrow a copy of the code that you used to run the pipeline on your cluster? This would be a huge help.

[kotoroshinoto]

I don't honestly remember if I ever followed up on this. Apologies if I didn't. (its been 2 years.) I don't know if I still have the code for the libdrmaa functionality or not.

Have you fellows @ hugeseq made progress toward use of libtorque itself?