search

Nesvilab / PD-Nodes

The implementation of MSFragger and Philosopher (PeptideProphet) as Proteome Discoverer nodes
http://www.nesvilab.org/PD-Nodes/
7 stars 1 forks source link

Memory error raising "System.IO.FileNotFoundException" #34

Closed FriedLabJHU closed 11 months ago

FriedLabJHU commented 1 year ago

System: Xeon E-2146G 12 CPUs @ 3.5 GHz with 32 GB RAM PD: 2.4.0.305 JRE: Adoptium JDK 20.0.1+9 x64, Adoptium JDK 8u372+b04 x64, Java SE 8u371 + Hotspot MSFraggerPD-Node 1.2.0 for PD 2.5 (PD 2.4 not available?) & 1.1.0 for PD 2.4 MSFragger: 4.7 & 4.8 Philosopher: 4.8.1 & 5.0.0 Max No. Processing WF in Parallel Execution: 1 Max No. Consensus WF in Parallel Execution: 1

We have consistent issues with this error when running LFQ jobs. A certain pepXML file is not created during the processing workflow. This error occurred with all the combinations of PD-Node, MSFragger, and Philosopher we have listed above. We completed only one LFQ job without raising the "System.IO.FileNotFoundException" error. I have attached images of the errors we get. Can these same jobs be run in FragPipe instead? Any help would be greatly appreciated!

20230612_msf_error 20230613_mem_err_msf 20230613_pd_err_msf

FriedLabJHU commented 1 year ago

For reference, the successful job used PD 2.4.0.305, Adoptium JDK 20.0.1+9 x64, PD-Node 1.2.0 MSFragger-PD2.5, MSFragger 4.8, Philosopher 5.0.0.

MagellanServer.log

huiyinc commented 1 year ago

Hi,

According to the log file, the error is caused by the lack of computer memory (i.e., out of memory). Can you please first check which version of JRE do you have in your computer? It is suggested to install the 64-bit Java JRE because the maximum memory usage for 32-bit Java JRE is only 4 GB.

Best,

Huiyin

Fried Lab @ Johns Hopkins University @.***> 於 2023年6月14日 上午4:41 寫道：



For reference, the successful job used PD 2.4.0.305, Adoptium JDK 20.0.1+9 x64, PD-Node 1.2.0 MSFragger-PD2.5, MSFragger 4.8, Philosopher 5.0.0.

— Reply to this email directly, view it on GitHub https://github.com/Nesvilab/PD-Nodes/issues/34#issuecomment-1589993525, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALAWWAYZ72QJW7BOPJORAL3XLDF7VANCNFSM6AAAAAAZFMYMPM . You are receiving this because you are subscribed to this thread.Message ID: @.***>

FriedLabJHU commented 1 year ago

We tried both Java JRE 8u371 64-bit & Adoptium JRE 20.0.1 64-bit. I ensured there weren’t any 32-bit versions.

In the processing wf, you will also see a message saying that JVM initialized with 23 gb of memory which should only be possible with a 64-bit version of Java. Do you think this is purely based on system memory? We had other jobs complete with extremely similar setting. The successful run searched for peptides of len 7-20, and the failed run from 21-30. Both searches producing >20M peptides.

FriedLabJHU commented 1 year ago

Here is what I am referring too.

Time | Processing Node | Level | Message -- | -- | -- | -- 11:59 AM | (2): MSFragger | Info | JVM started with 23 GB memory 11:59 AM | (2): MSFragger | Info | Java Info: 20.0.1, OpenJDK 64-Bit Server VM, Eclipse Adoptium 11:59 AM | (2): MSFragger | Info | System OS: Windows 10, Architecture: AMD64

Here is our Processing WF if it helps troubleshoot. image

huiyinc commented 1 year ago

Hi,

Thanks for sharing this information. Can you please try to process the data using FragPipe? If you still get the error message (i.e., out of memory), I would suggest you to run the PD-node in a computer with larger computer memory, or process the data using FragPipe with the "Database splitting" option.

Huiyin

Fried Lab @ Johns Hopkins University @.***> 於 2023年6月14日 週三 下午11:50寫道：

Here is what I am referring too. Time Processing Node Level Message 11:59 AM (2): MSFragger Info JVM started with 23 GB memory 11:59 AM (2): MSFragger Info Java Info: 20.0.1, OpenJDK 64-Bit Server VM, Eclipse Adoptium 11:59 AM (2): MSFragger Info System OS: Windows 10, Architecture: AMD64

— Reply to this email directly, view it on GitHub https://github.com/Nesvilab/PD-Nodes/issues/34#issuecomment-1591520792, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALAWWA2JM26NNCF5R2XXI5TXLHMUJANCNFSM6AAAAAAZFMYMPM . You are receiving this because you commented.Message ID: @.***>

-- Hui-Yin Chang, 張彙音 Assistant Professor Department of Biomedical Sciences and Engineering National Central University, Taiwan

wangrui85 commented 11 months ago

I suffer same problem as above images. I'm sure that's ok using FragPipe.

wangrui85 commented 11 months ago

screening

wangrui85 commented 11 months ago

error

huiyinc commented 11 months ago

Hi,

According to the log file, you are probably using an older version of Java Runtime Environment. It is suggested to install Java 11 Runtime Environment or above to run MSFragger. The reason you have no problem using FragPipe is possibly because you download FragPipe along with the JRE. So the JRE FragPipe uses is not the one installed in your computer.

Best,

Huiyin

wangrui85 @.***> 於 2023年7月31日 下午10:37 寫道：



[image: error] https://user-images.githubusercontent.com/87349846/257255228-05babbd9-beda-4d99-9b2e-22baf3d7b377.jpg

— Reply to this email directly, view it on GitHub https://github.com/Nesvilab/PD-Nodes/issues/34#issuecomment-1658496875, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALAWWA6RZMZIZAL2H5FU3MLXS67IXANCNFSM6AAAAAAZFMYMPM . You are receiving this because you commented.Message ID: @.***>

wangrui85 commented 11 months ago

Hi, chuiyin, Thanks so much. I update new Java to solve my issue. sincerely, Rui

发件人：chuiyin @.> 发送时间：2023年8月1日(星期二) 18:00 收件人：Nesvilab/PD-Nodes @.> 抄　送：王蕊 @.>; Comment @.> 主　题：Re: [Nesvilab/PD-Nodes] Memory error raising "System.IO.FileNotFoundException" (Issue #34) Hi, According to the log file, you are probably using an older version of Java Runtime Environment. It is suggested to install Java 11 Runtime Environment or above to run MSFragger. The reason you have no problem using FragPipe is possibly because you download FragPipe along with the JRE. So the JRE FragPipe uses is not the one installed in your computer. Best, Huiyin wangrui85 @.> 於 2023年7月31日 下午10:37 寫道：  [image: error] https://user-images.githubusercontent.com/87349846/257255228-05babbd9-beda-4d99-9b2e-22baf3d7b377.jpg https://user-images.githubusercontent.com/87349846/257255228-05babbd9-beda-4d99-9b2e-22baf3d7b377.jpg > — Reply to this email directly, view it on GitHub https://github.com/Nesvilab/PD-Nodes/issues/34#issuecomment-1658496875 https://github.com/Nesvilab/PD-Nodes/issues/34#issuecomment-1658496875 >, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALAWWA6RZMZIZAL2H5FU3MLXS67IXANCNFSM6AAAAAAZFMYMPM https://github.com/notifications/unsubscribe-auth/ALAWWA6RZMZIZAL2H5FU3MLXS67IXANCNFSM6AAAAAAZFMYMPM > . You are receiving this because you commented.Message ID: @.> — Reply to this email directly, view it on GitHub <https://github.com/Nesvilab/PD-Nodes/issues/34#issuecomment-1659982783 >, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AU2NUVSOABU6MO5BWSY77LTXTDHSFANCNFSM6AAAAAAZFMYMPM >. You are receiving this because you commented.Message ID: @.***>