User's incorrect configuration resulted in not enough memory crashing FragPipe

[Dobrokhotov1989]

Hi there,

I'm trying to run a custom LFQ-offset workflow with Percolator for PSM validation. The problem that FragPipe keep crashing. After the crash workstation got restarted, so I can't save the logfile and I don't know at what step exactly it happens (it takes multiple hours for processing and I'm not sitting all the time in from of the screen). However, after a few attempts there was a error files saved in the bin folder of the FragPipe stating the problem with insufficient memory for the JRE.

I've tried to reduce 'parallelism' in the workflow and I have Python configured for Database splitting but it still keep crashing. Is there anything else I can do to make it work?

P.S. there was no such problem with PeptideProphet or when I trialled the workflow on single MS file.

- Workflow fragpipe.txt - changed extension from .workflow to .txt, otherwise it can't be attached.

- Error files hs_err_pid6504.log hs_err_pid9464.log hs_err_pid20256.log

[fcyu]

Could you change the RAM (GB, 0=auto) to a non-zero value such as 64, and try again?

Thanks,

Fengchao

[Dobrokhotov1989]

I run it only with non-zero value (tried 100 and 128 GB - 128 GB is what we physically have). Should I try auto?

[fcyu]

Your computer may not have 100 GB free memory if the 128 GB is the total size. Could you try 64 and 0?

Thanks,

Fengchao

[Dobrokhotov1989]

Hi @fcyu ,

Sorry for the delay with the response - I'm using a shared workstation and it took a little longer to find a slot to try 64 or 0.

With 64GB, the outcome is exactly the same - FragPipe crashes with the same error file in the bin folder (attached). With auto RAM (0), the problem moved to the earlier steps and frapipe failed to check spectral files (log attached)

JVM started with 1 GB memory
Checking database...
Deisotoping doesn't support low resolution tandem mass spectra. Changing deisotope to 0.
deisotope = 0. Changing deneutralloss to 0.
Checking spectral files...
Failed in checking C:\Users\workstation_user\Desktop\Oleg\raw\PR738_OD4_15_0_Pinch_minus.raw. Will ignore it.
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
...
...
C:\Users\workstation_user\Desktop\Oleg\raw\PR738_OD4_12_4_ILK_minus.raw: Scans = 0; ITMS: false; FTMS: false; Isolation sizes = []
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
    at java.base/java.util.concurrent.FutureTask.report(Unknown Source)
    at java.base/java.util.concurrent.FutureTask.get(Unknown Source)
    at edu.umich.andykong.msfragger.MSFragger.a(Unknown Source)
    at edu.umich.andykong.msfragger.MSFragger.c(Unknown Source)
    at edu.umich.andykong.msfragger.MSFragger.a(Unknown Source)
    at edu.umich.andykong.msfragger.MSFragger.main(Unknown Source)
Caused by: java.lang.OutOfMemoryError: Java heap space
C:\Users\workstation_user\Desktop\Oleg\raw\PR738_OD4_26_4_Parv_plus.raw: Scans = 37794; ITMS: true; FTMS: false; Isolation sizes = [1.6]
Process 'MSFragger' finished, exit code: 1
Process returned non-zero exit code, stopping

~~~~~~~~~~~~~~~~~~~~
Cancelling 303 remaining tasks

hs_err_pid20884.log log_2024-07-13_15-52-10.txt

[fcyu]

I think something was wrong in your command because

JVM started with 1 GB memory

C:\Users\workstation_user\Desktop\Oleg\FragPipe-jre-22.0\fragpipe\jre\bin\java.exe -jar -Dfile.encoding=UTF-8 -Xmx1G C:\Users\workstation_user\Desktop\Oleg\FragPipe-jre-22.0\fragpipe\lib..\tools\MSFragger-4.1\MSFragger-4.1.jar C:\Users\workstation_user\Desktop\Oleg\LFQ_output_27_percolator_try10\fragger.params ....

It seems that you only got 1 GB memory.

Best,

Fengchao

[Dobrokhotov1989]

I'm using a GUI, so I'm not sure what went wrong. I've tried to start the analysis a few more times and each time -Xmx parameter was different (although much higher, e.g., 75, 116, 106 G). With 'Auto' and 116GB assigned, it seems working. Now, I wonder how to make it reproducible if each time it sets -Xmx some "random" numbers.

[fcyu]

Something was messed up. Please double check the FragPipe and system configuration

1. You set the RAM to 0 because there is workflow.ram=0 in your log file.
2. FragPipe detected that there was 1 GB free memory because there is -Xmx1G in your log file.

Best,

Fengchao