MSfragger node freezing with large number of samples

[lyons89]

I started testing out the MSfragger node in PD2.5. It worked perfectly on 3 quality control samples. Then when I went to implement it with a larger unlabeled experiment with 22 files it freezes early on in the deisotoping process. I've been successfully able to search a smaller subset of the data but once I add in all 22 files it freezes after the first 15 samples. I'm thinking this might relate to RAM, which i leave at the default value of -1. I'm wondering if this issue is known and if so what solutions exist?

Best, Scott

[huiyinc]

Hi Scott,

The RAM=-1 indicates that the node will use available memory for database search. How much RAM do you have? Maybe you could first send me the MagellanServer.log file for checking? You may find the log at: C:\ProgramData\Thermo\Proteome Discoverer 2.X\Logs

Thanks.

Huiyin

Scott Lyons @.***> 於 2022年8月9日 上午3:05 寫道：



I started testing out the MSfragger node in PD2.5. It worked perfectly on 3 quality control samples. Then when I went to implement it with a larger unlabeled experiment with 22 files it freezes early on in the deisotoping process. I've been successfully able to search a smaller subset of the data but once I add in all 22 files it freezes after the first 15 samples. I'm thinking this might relate to RAM, which i leave at the default value of -1. I'm wondering if this issue is known and if so what solutions exist?

Best, Scott

[image: image] https://user-images.githubusercontent.com/91705245/183493128-6fde4075-8d9a-49ff-b718-efbf653f9af8.png

— Reply to this email directly, view it on GitHub https://github.com/Nesvilab/PD-Nodes/issues/27, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALAWWA3JSH7JFNW7IIDA43DVYFLATANCNFSM556FNXTQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[lyons89]

Hello Huiyin,

I really appreciate your time and assistance. The computer I'm using has 100GB of RAM. I attached the log file from the current search in which I attempted to fix the issue by splitting the processing step up into 4 separate processing steps. Each step had 9, 5, 4, and 4 raw files. They all successfully finished the processing step except for the one that had 9 raw files. That one is again hung up in the deisotoping process in the beginning of the processing step.

Thanks again, Scott

MagellanServer.log

[huiyinc]

Hi Scott,

No problem. According to the log file, you use three search engines (MS Amanda 2.0, Sequest HT, and MSFragger) to process the raw files. Have you ever tried to only use MSFragger to process the 22 raw files? Also, is it possible that you share the 22 raw and fasta file with me so I can test on my computer? Thanks.

Huiyin

Scott Lyons @.***> 於 2022年8月9日 下午8:32 寫道：



Hello Huiyin,

I really appreciate your time and assistance. The computer I'm using has 100GB of RAM. I attached the log file from the current search in which I attempted to fix the issue by splitting the processing step up into 4 separate processing steps. Each step had 9, 5, 4, and 4 raw files. They all successfully finished the processing step except for the one that had 9 raw files. That one is again hung up in the deisotoping process in the beginning of the processing step.

Thanks again, Scott

MagellanServer.log https://github.com/Nesvilab/PD-Nodes/files/9290318/MagellanServer.log

— Reply to this email directly, view it on GitHub https://github.com/Nesvilab/PD-Nodes/issues/27#issuecomment-1209321551, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALAWWA4LJRSAS6XKAMD2IQ3VYJFUNANCNFSM556FNXTQ . You are receiving this because you commented.Message ID: @.***>

[lyons89]

Hey Huiyin,

Regardless if I use 3 search engines or just MSfragger, the search still stalls early on. I'm pretty sure it doesn't have anything to do with the raw files. I set up another search in PD using 12 control samples (mouse liver) and just using the MSfragger search engine and it froze again (I attached the log file). So MSfragger works fine using a limited number of samples but once you go over that limit it freezes. I'm thinking it has something to do with the RAM setting. I might try to re-search the 12 control samples but define the RAM argument, maybe that will be enough to get it going.

Best, Scott

MagellanServer.log

[huiyinc]

Hi Scott,

Thanks for the effort. I have tested the PD node using 24 files and it works fine. Also, there is no error message shown in the log file, so I currently have no clues what the problem might be. Is it possible that you share the raw and fasta files with me? If it is not convenient for you sharing the files, maybe you could try to process the 22 raw files via FragPipe (link https://github.com/Nesvilab/FragPipe) and see if it works? Thanks.

Huiyin

Scott Lyons @.***> 於 2022年8月12日 上午12:19 寫道：



Hey Huiyin,

Regardless if I use 3 search engines or just MSfragger, the search still stalls early on. I'm pretty sure it doesn't have anything to do with the raw files. I set up another search in PD using 12 control samples (mouse liver) and just using the MSfragger search engine and it froze again (I attached the log file). So MSfragger works fine using a limited number of samples but once you go over that limit it freezes. I'm thinking it has something to do with the RAM setting. I might try to re-search the 12 control samples but define the RAM argument, maybe that will be enough to get it going.

Best, Scott

MagellanServer.log https://github.com/Nesvilab/PD-Nodes/files/9310132/MagellanServer.log

— Reply to this email directly, view it on GitHub https://github.com/Nesvilab/PD-Nodes/issues/27#issuecomment-1212205892, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALAWWA7GFVBUQX6UOQJJLZLVYUR2HANCNFSM556FNXTQ . You are receiving this because you commented.Message ID: @.***>

[lyons89]

I downloaded the test files and ran with on PD2.5 using msfragger and everything worked perfectly fine! So now I have no idea why MSfragger won't run if I have too many samples. Does MSfragger not like it when file names have a dash ("-") in them? Some of my test raw files have have the dash in them, but the files ran fine when there wasn't many of them, but when I had 12 it froze. I'll let you know if I figure it out on my end. If you want I can send you some files, just let me know what you think the best way of sending them is.

Best, Scott

[anesvi]

Dear Scott

Perhaps you can use FragPipe instead. PD is a commercial tool, and we are limited in the support we (Hui-yin) can provide, but I would lave it up to Hui-yin how much she is able to help.

Best, Alexey

From: Scott Lyons @.> Sent: Tuesday, August 16, 2022 11:17 AM To: Nesvilab/PD-Nodes @.> Cc: Subscribed @.***> Subject: Re: [Nesvilab/PD-Nodes] MSfragger node freezing with large number of samples (Issue #27)

External Email - Use Caution

I downloaded the test files and ran with on PD2.5 using msfragger and everything worked perfectly fine! So now I have no idea why MSfragger won't run if I have too many samples. Does MSfragger not like it when file names have a dash ("-") in them? Some of my test raw files have have the dash in them, but the files ran fine when there wasn't many of them, but when I had 12 it froze. I'll let you know if I figure it out on my end. If you want I can send you some files, just let me know what you think the best way of sending them is.

Best, Scott

— Reply to this email directly, view it on GitHubhttps://github.com/Nesvilab/PD-Nodes/issues/27#issuecomment-1216779873, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIIMM6YT32J7M2DGGR4LRYLVZOWGRANCNFSM556FNXTQ. You are receiving this because you are subscribed to this thread.Message ID: @.**@.>>

---

Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

[lyons89]

Hello Alexey,

You're absolutely right, I don't want to take up any more of Hui-yin's time. Especially since there doesn't seem to be any indications as to what my issue is. I have been using MSfragger with the stand-alone fragpipe and that works very well. I just wanted to get it working in PD so that I could easily combine multiple search engines. I think I'll move on to OpenMS and see if I can get it working in there.

Best, Scott

[anesvi]

Ok, thanks

The only thing I would possibly suggest is to use less cores, or if there is a parallelization parameter that can be set to a lower value, in case PD tries to run many MSFragger jobs at the same time, instead of running one files using all available cores.

Best Alexey

From: Scott Lyons @.> Sent: Tuesday, August 16, 2022 1:48 PM To: Nesvilab/PD-Nodes @.> Cc: Nesvizhskii, Alexey @.>; Comment @.> Subject: Re: [Nesvilab/PD-Nodes] MSfragger node freezing with large number of samples (Issue #27)

External Email - Use Caution

Hello Alexey,

You're absolutely right, I don't want to take up any more of Hui-yin's time. Especially since there doesn't seem to be any indications as to what my issue is. I have been using MSfragger with the stand-alone fragpipe and that works very well. I just wanted to get it working in PD so that I could easily combine multiple search engines. I think I'll move on to OpenMS and see if I can get it working in there.

Best, Scott

— Reply to this email directly, view it on GitHubhttps://github.com/Nesvilab/PD-Nodes/issues/27#issuecomment-1216957727, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIIMM663OE5O2PG77BYERT3VZPH3LANCNFSM556FNXTQ. You are receiving this because you commented.Message ID: @.**@.>>

---

Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

[huiyinc]

Hi Scott,

Maybe you can send me an email and we can discuss how to share the files in private. My email address: @.*** Thanks.

Huiyin

Alexey Nesvizhskii @.***> 於 2022年8月17日 上午1:54 寫道：



Ok, thanks

The only thing I would possibly suggest is to use less cores, or if there is a parallelization parameter that can be set to a lower value, in case PD tries to run many MSFragger jobs at the same time, instead of running one files using all available cores.

Best Alexey

From: Scott Lyons @.> Sent: Tuesday, August 16, 2022 1:48 PM To: Nesvilab/PD-Nodes @.> Cc: Nesvizhskii, Alexey @.>; Comment @.> Subject: Re: [Nesvilab/PD-Nodes] MSfragger node freezing with large number of samples (Issue #27)

External Email - Use Caution

Hello Alexey,

You're absolutely right, I don't want to take up any more of Hui-yin's time. Especially since there doesn't seem to be any indications as to what my issue is. I have been using MSfragger with the stand-alone fragpipe and that works very well. I just wanted to get it working in PD so that I could easily combine multiple search engines. I think I'll move on to OpenMS and see if I can get it working in there.

Best, Scott

— Reply to this email directly, view it on GitHub< https://github.com/Nesvilab/PD-Nodes/issues/27#issuecomment-1216957727>, or unsubscribe< https://github.com/notifications/unsubscribe-auth/AIIMM663OE5O2PG77BYERT3VZPH3LANCNFSM556FNXTQ>.

You are receiving this because you commented.Message ID: @.**@.>>

---

Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

— Reply to this email directly, view it on GitHub https://github.com/Nesvilab/PD-Nodes/issues/27#issuecomment-1216964318, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALAWWA2ECRG65ZHD4LJFKMDVZPIUHANCNFSM556FNXTQ . You are receiving this because you commented.Message ID: @.***>