[interhelp #25958] What cluster does it support? Only LSF?

[GoogleCodeExporter]

I have a PBS cluster, and i hope it will support PBS soon

Original issue reported on code.google.com by sf.to...@gmail.com on 31 Mar 2013 at 1:36

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Dear I5 user,

LSF is the only cluster InterProScan 5 fully supports. This is because LSF is 
the only cluster we can test in house.
However InterProScan 5 is an open source project and the way it is written it 
should be possible to run it on any cluster job scheduler. We have another user 
who is working towards an SGE interface. Therefore if you want to contribute to 
adjust InterProScan 5 to support PBS as well, feel free to contact us again and 
we will find a solution how we can collaborate.

Kind Regards,
Maxim

Original comment by Maxim.Sc...@gmail.com on 2 Apr 2013 at 11:27

[GoogleCodeExporter]

I was thinking about what changes need to be done to run InterProScan on a PBS 
cluster. In theory the only file which needs adjustments is the 
'interproscan.properties' file.
It contains the following 4 properties, unfortunately misleading in their 
naming as they are indicating LSF:
#The following 2 PBS commands are used by workers to spawn normal or high 
memory workers
grid.lsf.worker.command=
grid.lsf.worker.high.memory.command=
#following 2 PBS commands are used by the master to spawn normal or high memory 
workers
grid.lsf.master.command=
grid.lsf.master.high.memory.command=

We would really appreciate if you could work out the submission commands for 
PBS and give it a try on your cluster.

We do have documentations about how LSF and SGE are configured here:
SGE:
http://code.google.com/p/interproscan/wiki/i5RC6ClusterModeSGE
LSF:
http://code.google.com/p/interproscan/wiki/i5RC5_Appendix_A_Cluster_Mode

Many Thanks,
Maxim

Original comment by Maxim.Sc...@gmail.com on 8 Apr 2013 at 8:37

[GoogleCodeExporter]

Thank you for your kindly answer. My PBS cluster is in the process of 
debugging.  I will test the InterProScan5 as soon as it finishes.

Best wishes,

Fei

Original comment by sf.to...@gmail.com on 8 Apr 2013 at 8:50

[GoogleCodeExporter]

Dear Fei,
That sounds great. I am looking forward to see how InterProScan behaves in 
combination with PBS. Don't hesitate to contact us again. I am pretty sure we 
can help you to set up the right submission commands.

Many Thanks,
Maxim

Original comment by Maxim.Sc...@gmail.com on 8 Apr 2013 at 2:24

[GoogleCodeExporter]

Has there been any further movement on this issue? We are looking into 
installing InterProScan at the Pittsburgh Supercomputing Center.  Our machines 
run Torque.

I assume that information related to running older versions of InterProScan 
with Torque/PBS is no longer relevant, e.g.:

http://freelancingscience.com/2008/07/10/configuring-torque-and-interproscan/

https://sites.google.com/a/brown.edu/bioinformatics-in-biomed/iprscan-by-qsub

Thanks.

Philip Blood
Pittsburgh Supercomputing Center
blood@psc.edu

Original comment by bl...@psc.edu on 18 Jun 2013 at 5:34

[GoogleCodeExporter]

Dear Philip,

InterProScan 5 (I5) is a complete re-written piece of software and therefore 
any information of an older version (for instance 4.8) doesn't apply anymore. 
However in theory InterProScan 5 is written flexible enough to run on any 
cluster platform and not only on LSF and SGE. But LSF and SGE are the only 
platforms we can test here at the EBI.

FYI, we've got feedback from at least one user who successfully configured and 
run I5 on a PBS cluster. So in theory all you have to do is, follow the 
instruction on the following page and change the submission commands to your 
cluster platform:
http://code.google.com/p/interproscan/wiki/InterProScan5ClusterMode

Could we kindly ask you to provide us the final configuration section within 
your interproscan.properties file when you successfully managed to run I5 on 
your cluster platform including the cluster version, so we can share that 
information with our users. Many Thanks in advance.

Kind Regards,
Maxim

Original comment by Maxim.Sc...@gmail.com on 24 Jun 2013 at 2:41

• Changed state: Started

[GoogleCodeExporter]

I'll handle this one.

Original comment by nuka....@gmail.com on 6 Aug 2013 at 4:07

• Changed state: Accepted

[GoogleCodeExporter]

Original comment by Maxim.Sc...@gmail.com on 15 Aug 2013 at 11:13

• Added labels: Subject-ClusterMode

[GoogleCodeExporter]

Original comment by Maxim.Sc...@gmail.com on 5 Nov 2013 at 12:20

• Changed state: Done

[GoogleCodeExporter]

Hello,

Is there any progress on deployment IPS5 under other queue systems except LSF 
and SGE? My HPC is running PBS Torque and it would be great to have some 
instruction for installing IPS5 with cluster mode.

Best, Yuri.

Original comment by yura...@gmail.com on 19 Nov 2013 at 7:20

[GoogleCodeExporter]

Dear Yuri,

I am sorry, but you won't get an advise from us on how to set up InterProScan 5 
(I5) for PBS Torque. We can just support want we got on site.
In theory what you have to do is, you would have to work out the submission 
command for PBS Torque and then you would need to set these in your 
'interproscan.properties' file like for SGE and LSF 
(https://code.google.com/p/interproscan/wiki/ClusterMode).

I have contacted a couple of people who said they would try to set up I5 on PBS 
Torque and asked them if they would like to share their knowledge with the 
community. I will let you know, as soon as I hear anything back from them.
But also I think setups might differ depending on versions and how environments 
are set-uped.

Kind Regards,
Maxim

Original comment by Maxim.Sc...@gmail.com on 20 Nov 2013 at 11:08

• Changed state: Started

[GoogleCodeExporter]

Dear Maxim,

Thank you for explanation. I'll contact my HPC admin in this respect. Anyway, 
suggestions from people who have IPS5 working under PBS are appreciated.

Best, Yuri.

Original comment by yura...@gmail.com on 21 Nov 2013 at 6:35

[GoogleCodeExporter]

Original comment by Maxim.Sc...@gmail.com on 16 Jul 2014 at 1:33

• Changed state: Done

[GoogleCodeExporter]

Hi,  I have InterProScan running on a cluster with a PBS scheduler.  To do this 
I had to write a perl wrapper script for the 'qsub' command.  I can share this 
script but I wanted to make sure I didn't do something silly and miss 
something... So here's the issues I encountered

With InterProScan 5.6 it tried to append the java command to the end of the 
qsub statement (i.e. the value of grid.master.submit.command in the properties 
file).  But the PBS version I'm using doesn't allow this.  It requires a 
submission script.

The PBS version I'm using will allow for a command to be provided on the 
command-line if it's done in this way:  echo "[command]" > qsub [options].  
When I set the grid.name to "other" in the interproscan.properties file it 
would create an "echo" command but would still append it to whatever was in the 
grid.master.submit.command. I could not figure out how to make it work.  Am I 
missing some details?

To get around this problem, I wrote a Perl script that received the arguments 
as if it was a qsub command for sge and then generated the proper PBS 
submission script and submit it.  That seemed to solve the problem.  I can 
share the script if you like, but I want to make sure I didn't miss something 
that would have allowed me to make it work without it.

Finally, I am still having one issue.  I'm not sure if this is related to the 
PBS job environment settings, but when the interproscan jobs run, it creates 
log files in [interproscan]/logs/[clusterunid] directory and it creates some 
files in [interproscan]/temp/[some id] and also here [project dir]/temp/[some 
id].  This caused path problems when the binaries were run as they couldn't 
find all of the temp files. To fix this I passed the -T option to 
interproscan.sh, and set it to [interproscan]/temp/[some id] but this isn't a 
very pretty solution.  On our SGE setup we don't have this issue at all.  Any 
thoughts?

Thanks,
Stephen

Original comment by spfick...@gmail.com on 20 Aug 2014 at 1:25

[GoogleCodeExporter]

So on my last comment, in regards to the temp file problem. It turns out we 
were having the same problem on our SGE system but a fix was applied that I 
didn't know of.  So, perhaps that's not related to PBS but a different problem.

Original comment by spfick...@gmail.com on 20 Aug 2014 at 6:14

[GoogleCodeExporter]

Original comment by Maxim.Sc...@gmail.com on 1 Sep 2014 at 10:04

• Changed title: [interhelp #25958] What cluster does it support? Only LSF?

[GoogleCodeExporter]

Hi Stephen,
Thanks for getting in touch with us.

We need to discuss your case internally and will come back to you with another 
message as soon as possible.

Thanks for your patience.

Kind Regards,
Maxim

Original comment by Maxim.Sc...@gmail.com on 1 Sep 2014 at 10:53

• Changed state: Accepted

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

The temp file problem is a known issue (Issue 46) and needs fixing.

List of known issues: 
https://code.google.com/p/interproscan/wiki/i5_7_48_KnownIssues

Original comment by Maxim.Sc...@gmail.com on 1 Sep 2014 at 11:13

[GoogleCodeExporter]

Is there any progress on supporting PBSPro?

Original comment by Phh...@gmail.com on 6 Oct 2014 at 11:03

[GoogleCodeExporter]

Hi,

Thank you for the message. Currently at EBI we have only used LSF and SGE, so 
there is no recent progress, comment #8 (onwards) is still the latest status. 
But if you are having problems getting something in particular to work then 
feel free add a new question and we can see if we or some other PBS users in 
the community have managed to solve it!

Thanks,

Matthew

Original comment by Mr.Matth...@gmail.com on 6 Oct 2014 at 2:52

[GoogleCodeExporter]

Hi Stephen,

We have now fixed the problem with temp directories and should be working fine 
in our latest release. 

May you share with us your script you use to run InterProScan on PBS scheduler.

Regards,
Gift

Original comment by nuka....@gmail.com on 16 Feb 2015 at 10:04