Some problem about RazerS 3.1

[WeijianYe]

Hi, When I installed the OptiType, I found that the website about Razer3 could not be accessed and it seems that there is no other way to download the Razer3. Could you mind to help?

[b-schubert]

Hi,

Unfortunately, we do not maintain RazerS. You can send David Weese (the main author of RazerS) an email. The address is david.weese[at]fu-berlin.de .

[WeijianYe]

Thank you very much for your help. Now the website of RazerS is acceptable and it seems that I have installed the OptiType successfully( all the additional tools have been down ) But when I run the OptiType with the example data, there was an error below

0:22:06.86 Keeping only the minimal number of required alleles (62,)

0:22:06.86 Creating compact model...

0:22:06.96 Initializing OptiType model... WARNING: No construction rule or expression specified for constraint 'c' Invalid option '-s'; try /faststorage/home/siyang/Bin/glpk-4.55/bin/glpsol --help ERROR: "[base]/site-packages/coopr/opt/base/solvers.py", 448, solve Solver (asl) returned non-zero return code (1) ERROR: "[base]/site-packages/coopr/opt/base/solvers.py", 451, solve See the solver log above for diagnostic information. WARNING: Solver does not support multi-threading. Please change the config file accordingly! Fall back to single-threading. Traceback (most recent call last): File "OptiTypePipeline.py", line 309, in result = op.solve(args.enumerate) File "/faststorage/home/siyang/BACKUP/Bin/OptiType-master/model.py", line 154, in solve del self.__solver.options["threads"] KeyError: 'threads'

The problem may be caused by GLPK. SO could you mind to tell me

1. which version of GLPK that can be supported by OptiType?
2. In the conf file of OptiType, I changed the "SOLVER=glpk" into "SOLVER=glpsol" because I only find the executable file named "glpsol" instead of "glpk" OR there maybe other reasons for this error, could you help?

[b-schubert]

Hi,

Did OptiType produce some output, or did it stopp right there?

This is a configuration issue, as stated in the warning. GLPK does not support multithreading. Just set the number of threads used for the solver to one in the config file. If you need a multithreaded ILP solver have a look at BCB (https://projects.coin-or.org/Cbc - Open Source) or CPLEX (http://www-01.ibm.com/support/docview.wss?uid=swg21419058 - free for academics)

[b-schubert]

Which version of Coopr are you using? It seems as if they changed something in their API, thats why the error occurred.

[WeijianYe]

Thank you for your response. Yes, it stopped after two alignment sam files had been finished. My Coopr is version 3.5.8787 I changed the threads(set 1) and rerun the OptiType but the same error still occurred, here I paste the whole log

0:00:01.54 Mapping NA11995_SRR766010_1_fished.fastq to GEN reference...

0:02:24.53 Mapping NA11995_SRR766010_2_fished.fastq to GEN reference...

0:04:40.04 Generating binary hit matrix. 0:04:40.04 Loading alleles and read IDs from ./test/exome/2014_12_18_08_34_39/2014_12_18_08_34_39_0.sam... 0:04:41.86 11179 alleles and 1909 reads found. 0:04:41.86 Initializing mapping matrix... 0:04:41.86 1909x11179 mapping matrix initialized. Populating 1369066 hits from SAM file... 10% completed 20% completed 30% completed 40% completed 50% completed 60% completed 70% completed 80% completed 90% completed 100% completed 0:09:25.26 1369066 elements filled. Matrix sparsity: 1 in 15.59 0:09:44.05 Loading alleles and read IDs from ./test/exome/2014_12_18_08_34_39/2014_12_18_08_34_39_1.sam... 0:09:46.05 11179 alleles and 1876 reads found. 0:09:46.05 Initializing mapping matrix... 0:09:46.06 1876x11179 mapping matrix initialized. Populating 1305761 hits from SAM file... 10% completed 20% completed 30% completed 40% completed 50% completed 60% completed 70% completed 80% completed 90% completed 100% completed 0:14:33.40 1305761 elements filled. Matrix sparsity: 1 in 16.06

0:15:06.32 temporary pruning of identical rows and columns

0:15:06.40 Size of mtx with unique rows and columns: (496, 778) 0:15:06.40 determining minimal set of non-overshadowed alleles

0:15:07.50 Keeping only the minimal number of required alleles (62,)

0:15:07.50 Creating compact model...

0:15:07.59 Initializing OptiType model... WARNING: No construction rule or expression specified for constraint 'c' Invalid option '-s'; try /faststorage/home/siyang/Bin/glpk-4.55/bin/glpsol --help ERROR: "[base]/site-packages/coopr/opt/base/solvers.py", 448, solve Solver (asl) returned non-zero return code (1) ERROR: "[base]/site-packages/coopr/opt/base/solvers.py", 451, solve See the solver log above for diagnostic information. WARNING: Solver does not support multi-threading. Please change the config file accordingly! Fall back to single-threading. Traceback (most recent call last): File "OptiTypePipeline.py", line 309, in result = op.solve(args.enumerate) File "/faststorage/home/siyang/BACKUP/Bin/OptiType-master/model.py", line 154, in solve del self.__solver.options["threads"] KeyError: 'threads'

[b-schubert]

Hm so with my older configurations its working fine. Im currently setting up a test environment with your Coopr and GLPK version to see whats going on there.

Thanks a lot and sorry for the inconvenience.

[WeijianYe]

Thank you very much! As well I paste my conf file here. Looking forward to your news

[MAPPING]

please specify the razerS3 binary path

RAZERS3=/faststorage/home/siyang/Bin/razers3-3.1.1-Linux-x86_64/bin/razers3 THREADS=1

[LIBRARIES] RNA_REF=/faststorage/home/siyang/Bin/OptiType-master/data/hla_reference_rna.fasta DNA_REF=/faststorage/home/siyang/Bin/OptiType-master/data/hla_reference_dna.fasta ALLELES=/faststorage/home/siyang/Bin/OptiType-master/data/alleles.h5

[OPTIMIZATION]

the solver has to be supported by Coopr

SOLVER=glpsol THREADS=1

[b-schubert]

Unfortunately, I cant reproduce the issue. I have tested it on the following setups:

Old: Coopr 3.5.8787 GLPK 4.50

New (Yours): Coopr 3.5.8787 GLPK 4.55

But based on the output from you I guess your initial suspicion was correct; It defiantly has to do with GLPK. Have you tried to run glpk from terminal on a test problem?

As a (hopefully) quick fix try to recompile GLPK again or maybe switch to CBC (https://projects.coin-or.org/Cbc). CBC seems to work fine for me out of the box.

If you have non-proprietary data you could also use our (not yet official) web service hosting OptiType (http://www.epitoolkit.de/). We are still working on it, so the service might be offline from time to time.

[yeemanchan]

I have the same problem too. I think the issue is glpsol doesn't support "-s" option.

I want to try CBC too but unfortunately their site is down as of now...

[b-schubert]

The project site is up. I changed the Installation guide to CBC now. I couldn't find any mentions of that bug in coopr related forums.

[yeemanchan]

CBC works for me.

But it will be great if it also supports glpsol from glpk-4.55. Or tell us which version of glpk it supports.

[b-schubert]

As I said, I couldn't reproduce the problem. I tried several GLPK versions (from 4.50 to 4.55) and all worked fine for me. I contacted the Coopr developers asking for help. But so far haven't got an answer.

I'll keep trying to solve this issue.

Do you guys have any problems calling glpk from command line with a toy problem?

[yeemanchan]

So who is passing the "-s" option to glpsol? Is it OptiType or Coorp?

[b-schubert]

So I got an answer.

Could you please name the solver 'glpk' instead 'glpsol' in the config file. This should fix your problems hopefully.

Here is the quote from Gabriel Hackebeil:

I can reproduce this error when I use GLPK through the NL file interface, which GLPK does not support. However, it is very likely that this is unintentional and coming about by your software (or users) choosing “glpsol” as the solver name rather than “glpk”. We have a registered solver plugins that recognizes the solver name “glpk”, which launches the necessary interface to the “glpsol” executable. When no registered solver plugins match a given name (e.g., “glpsol”), we fall back on using the NL file interface because it is a generic interface to a large number of different solvers.

Short answer:

(incorrect) --solver=glpsol (correct) --solver=glpk

(incorrect) SolverFactory('glpsol’) (correct) SolverFactory(‘glpk’)

Gabe

[yeemanchan]

I renamed the executable and also changed config.ini but it seems like it is assuming glpsol.

---

user@HPC:/tank/optitype$ time python OptiTypePipeline.py -i SRR1558164_fished_1.fastq SRR1558164_fished_2.fastq --dna -o ./SRR1558164/ /usr/local/lib/python2.7/dist-packages/pandas/util/decorators.py:13: FutureWarning: diff is deprecated. Use difference instead FutureWarning) WARNING: No construction rule or expression specified for constraint 'c' WARNING: "[base]/dist-packages/coopr/solvers/plugins/solvers/GLPK_old.py", 69, executable Could not locate the 'glpsol' executable, which is required for solver 'glpk' WARNING: Solver does not support multi-threading. Please change the config file accordingly! Fall back to single-threading. Traceback (most recent call last): File "OptiTypePipeline.py", line 309, in result = op.solve(args.enumerate) File "/tank/optitype/model.py", line 154, in solve del self.__solver.options["threads"] KeyError: 'threads'

[b-schubert]

you dont need to rename the executable, just rename the parameter in the config file from:

solver=glpsol

to

solver=glpk

that should invoke cooprs GLPKs interfaces instead of a generic interface (which happens when naming it glpsol)

Also, you should check whether glpk is still globally executable by typing the following in your command line: glpsol --help

If the help pops up, everything should be fine and OptiType should be working.

[yeemanchan]

Finally, I make glpk works. You have to set the solver in config.ini to "glpk" even if the executable is called "glpsol"

Thanks for your follow up.