maxquant_mqpar Thermo RAW crashes when configuring

[eschen42]

I opened an issue for this with MaxQuant MaxQuant-450 but wonder whether you can help.

In brief, I am trying to run MaxQuant with Thermo RAW files. For the record, I have copied the exact files from the mulled Conda environment onto an MS Windows that has no other instance of the Thermo Raw File Reader software installed and I have shown that MaxQuant (running from that copy) runs successfully. By contrast, in Galaxy (or on Linux outside Galaxy, using either a copy of the environment or an identically-created environment), MaxQuantCmd.exe dies during the "Configuring" step, terminating with a "file not found" message that gives no hint as to which file is not found. strace shows that a "futex" times out but gives no way to tie it back to the file that would not open.

I don't really know where to turn for help; so far, I have had no response from the MaxQuant team.

@npinter Thank you for your work with this tool.

[npinter]

Hi @eschen42,

are you using the latest bioconda container of MQ (uses dotnet instead of mono, please check the "maxquant" file)?

The problem with maxquant (only 1.6.17.0 and higher, not 1.6.43.10) + dotnet + Linux + Galaxy is, that Thermo RAW files cannot be accessed if they are symlinked. We changed the wrapper to copy RAW files instead of using symbolic links, which fixed the the bug you described (File not found).

Are you using symbolic links?

Best Niko

[eschen42]

@npinter Thank you for your reply. I am using Galaxy tool 1.6.17.0, specifically:

toolshed.g2.bx.psu.edu/repos/galaxyp/maxquant/maxquant_mqpar/1.6.17.0+galaxy0

Here's the composition of my mulled environment, which has installed dotnet and not mono:

(mulled-v1-f38ced47f3c08812a1489b064836677f8c18842b30c8a63d2f37feb967ac8b81) $  conda env export | egrep "pyyaml|tar|r-ptxqc|python|maxquant|dotnet|mono|mulled"
name: mulled-v1-f38ced47f3c08812a1489b064836677f8c18842b30c8a63d2f37feb967ac8b81
  - dotnet=2.1.818=ha770c72_0
  - dotnet-aspnetcore=2.1.30=h73ebe80_0
  - dotnet-runtime=2.1.30=h73ebe80_0
  - dotnet-sdk=2.1.818=h73ebe80_0
  - font-ttf-dejavu-sans-mono=2.37=hab24e00_0
  - maxquant=1.6.17.0=py39hdfd78af_4
  - python=3.7.10=hb7a2778_101_cpython
  - python_abi=3.7=2_cp37m
  - pyyaml=5.1.2=py37h516909a_1
  - r-ptxqc=1.0.10=r41hc72bb7e_0
  - tar=1.32=hd4ba37b_0
prefix: /home/art/miniforge3/envs/mulled-v1-f38ced47f3c08812a1489b064836677f8c18842b30c8a63d2f37feb967ac8b81

Still no joy, even without logical links.

(mulled-v1-f38ced47f3c08812a1489b064836677f8c18842b30c8a63d2f37feb967ac8b81) $ ls
2018-05-27_SwissProt_Human_Canonical_Isoform.fasta  jdrake_zsychev_20201223_17947_03_TiO2_S_33rd.raw.thermo.raw  jdrake_zsychev_20201223_17947_06_FeNTA_S_33rd.raw.thermo.raw  limbo
crap_withMycoplasma_2016_Bielow.fasta               jdrake_zsychev_20201223_17947_04_TiO2_N_33rd.raw.thermo.raw  jdrake_zsychev_20201223_17947_07_FeNTA_N_33rd.raw.thermo.raw  mqpar.xml
(mulled-v1-f38ced47f3c08812a1489b064836677f8c18842b30c8a63d2f37feb967ac8b81) $ egrep "thermo.raw|[.]fasta" mqpar.xml
                        <fastaFilePath>2018-05-27_SwissProt_Human_Canonical_Isoform.fasta</fastaFilePath>
                        <fastaFilePath>crap_withMycoplasma_2016_Bielow.fasta</fastaFilePath>
                <string>/home/art/tmp/thermoraw01/jdrake_zsychev_20201223_17947_03_TiO2_S_33rd.raw.thermo.raw</string>
                <string>/home/art/tmp/thermoraw01/jdrake_zsychev_20201223_17947_04_TiO2_N_33rd.raw.thermo.raw</string>
                <string>/home/art/tmp/thermoraw01/jdrake_zsychev_20201223_17947_06_FeNTA_S_33rd.raw.thermo.raw</string>
                <string>/home/art/tmp/thermoraw01/jdrake_zsychev_20201223_17947_07_FeNTA_N_33rd.raw.thermo.raw</string>
(mulled-v1-f38ced47f3c08812a1489b064836677f8c18842b30c8a63d2f37feb967ac8b81) $ maxquant mqpar.xml
dotnet /home/art/miniforge3/envs/mulled-v1-f38ced47f3c08812a1489b064836677f8c18842b30c8a63d2f37feb967ac8b81/bin/MaxQuantCmd.exe mqpar.xml
Configuring

Unhandled Exception: System.ComponentModel.Win32Exception: No such file or directory
   at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo)
   at System.Diagnostics.Process.Start()
   at QueueingSystem.WorkDispatcher.ProcessSingleRunExternalProcess(Int32 taskIndex, Int32 threadIndex)
   at QueueingSystem.WorkDispatcher.DoWork(Int32 taskIndex, Int32 threadIndex)
   at QueueingSystem.WorkDispatcher.Work(Object threadIndex)
   at System.Threading.Thread.ThreadMain_ParameterizedThreadStart(Object parameter)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
/home/art/miniforge3/envs/mulled-v1-f38ced47f3c08812a1489b064836677f8c18842b30c8a63d2f37feb967ac8b81/bin/maxquant: line 5: 32064 Aborted                 dotnet $DIR/MaxQuantCmd.exe $@

[npinter]

Thanks, please take a look inside the combined/proc/ folder (should be in the directory where your raw files are located -> /home/art/tmp/thermoraw01/). There must be one file ending with .error.txt which consists a more informative error message than the one above.

It's definitely not the same error message we got with symbolic links + thermo.raw files.

Probably something:

                        <fastaFilePath>2018-05-27_SwissProt_Human_Canonical_Isoform.fasta</fastaFilePath>
                        <fastaFilePath>crap_withMycoplasma_2016_Bielow.fasta</fastaFilePath>

Can you try to use the absolut path for those two files?

[eschen42]

@npinter Yes, it would be very convenient if there were a .error.txt file in combined/proc/ but sadly there isn't. Perhaps the exception is thrown even before the Configuring task is fully initiialized.

Prompts shortened from

(mulled-v1-f38ced47f3c08812a1489b064836677f8c18842b30c8a63d2f37feb967ac8b81) art@DESKTOP-00J5NN6:~/tmp/thermoraw01$

for clarity:

$ egrep "[.]thermo[.]raw|[.]fasta" /home/art/tmp/thermoraw01/mqpar.xml | sed -e 's/^[ \t]*//'
<fastaFilePath>/home/art/tmp/thermoraw01/2018-05-27_SwissProt_Human_Canonical_Isoform.fasta</fastaFilePath>
<fastaFilePath>/home/art/tmp/thermoraw01/crap_withMycoplasma_2016_Bielow.fasta</fastaFilePath>
<string>/home/art/tmp/thermoraw01/jdrake_zsychev_20201223_17947_03_TiO2_S_33rd.raw.thermo.raw</string>
<string>/home/art/tmp/thermoraw01/jdrake_zsychev_20201223_17947_04_TiO2_N_33rd.raw.thermo.raw</string>
<string>/home/art/tmp/thermoraw01/jdrake_zsychev_20201223_17947_06_FeNTA_S_33rd.raw.thermo.raw</string>
<string>/home/art/tmp/thermoraw01/jdrake_zsychev_20201223_17947_07_FeNTA_N_33rd.raw.thermo.raw</string>
$ ls -l *.thermo.raw *.fasta *.xml
-rw-r--r-- 1 art art   29139981 Oct  5 13:20 2018-05-27_SwissProt_Human_Canonical_Isoform.fasta
-rw-r--r-- 1 art art     943051 Oct  5 13:20 crap_withMycoplasma_2016_Bielow.fasta
-rw-r--r-- 1 art art 1043969746 Oct  5 13:18 jdrake_zsychev_20201223_17947_03_TiO2_S_33rd.raw.thermo.raw
-rw-r--r-- 1 art art  858862173 Oct  5 13:18 jdrake_zsychev_20201223_17947_04_TiO2_N_33rd.raw.thermo.raw
-rw-r--r-- 1 art art 1091848217 Oct  5 13:18 jdrake_zsychev_20201223_17947_06_FeNTA_S_33rd.raw.thermo.raw
-rw-r--r-- 1 art art 1050229272 Oct  5 13:18 jdrake_zsychev_20201223_17947_07_FeNTA_N_33rd.raw.thermo.raw
-rw-r--r-- 1 art art      20119 Oct  7 12:36 mqpar.xml
$ readlink -f .
/home/art/tmp/thermoraw01
$ maxquant -p 1 -e 1 /home/art/tmp/thermoraw01/mqpar.xml
dotnet /home/art/miniforge3/envs/mulled-v1-f38ced47f3c08812a1489b064836677f8c18842b30c8a63d2f37feb967ac8b81/bin/MaxQuantCmd.exe -p 1 -e 1 /home/art/tmp/thermoraw01/mqpar.xml
Configuring

Unhandled Exception: System.ComponentModel.Win32Exception: No such file or directory
   at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo)
   at System.Diagnostics.Process.Start()
   at QueueingSystem.WorkDispatcher.ProcessSingleRunExternalProcess(Int32 taskIndex, Int32 threadIndex)
   at QueueingSystem.WorkDispatcher.DoWork(Int32 taskIndex, Int32 threadIndex)
   at QueueingSystem.WorkDispatcher.Work(Object threadIndex)
   at System.Threading.Thread.ThreadMain_ParameterizedThreadStart(Object parameter)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
/home/art/miniforge3/envs/mulled-v1-f38ced47f3c08812a1489b064836677f8c18842b30c8a63d2f37feb967ac8b81/bin/maxquant: line 5: 15513 Aborted                 dotnet $DIR/MaxQuantCmd.exe $@
$ ls combined/proc/
'#runningTimes.txt'

[npinter]

1. What if you try to call MaxQuantCmd.exe directly (with the mulled conda package/env activated):

dotnet /home/art/miniforge3/envs/mulled-v1-f38ced47f3c08812a1489b064836677f8c18842b30c8a63d2f37feb967ac8b81/bin/MaxQuantCmd.exe -p 1 -e 1 /home/art/tmp/thermoraw01/mqpar.xml

2. Can you provide your mqpar.xml?

3. Have you set useDotNetCore == True?

   <useDotNetCore>True</useDotNetCore>

[eschen42]

1. Direct invocation produces the same failure.
2. I am attaching my mqpar.xml mqpar.zip
3. When I set useDotNetCore == True, it worked (well, it failed at a later step, but that's progress)!

This suggests that perhaps some of the "pre-flight checking" could handle such a case, e.g., something in modify_mqpar.py that would resemble:

mqparam.set_simple_param('useDotNetCore', 'True')

Another nice enhancement might be to cat any combined/proc/*.error.txt files to stderr so that they can be visible from the page that opens by clicking the debugging button.

[eschen42]

I am not sure why it crashed on the next step and am waiting for the re-submission to get more forensics. Here is the Galaxy Error Report that I filed. GalaxyErrorReport.txt

Meanwhile, it is running happily enough outside Galaxy on Linux for me (so I'll have something to compare to the above error) and has not yet failed on our own Galaxy.

In short, this issue may be ready to close, once I tie up these loose ends.

[eschen42]

It worked in our own Galaxy; perhaps I got a bad parameter somewhere on UseGalaxy.eu.

For now, I will close the issue. Thank you @npinter !!!!

[eschen42]

I retitled

MaxQuant issue 450

as

MaxQuantCmd 1.6.5.0 should report when useDotNetCore is set to the wrong value