search

compomics / peptide-shaker

Interpretation of proteomics identification results
http://compomics.github.io/projects/peptide-shaker.html
48 stars 18 forks source link

Peptideshaker-Memory issue #283

Closed Yaarub closed 4 years ago

Yaarub commented 7 years ago

Dear Peptideshaker team, I have been trying to import SearchGui output (analysis of 6 MS raw files) files to peptide shaker, however I have been experiencing a memory issue at the step of: "Validating identifications as 1% FDR, quality control of matches". Basically I experiencing same issue mentioned in issue #282. This is a bit surprising as 39G are set as memory for peptide shaker and I am using a workstation with Xeon E5 (3.5GHz) processor. The same data has been processed previously with Peaks, MQ, and Spectronaut and everything went smoothly, however, I am very new to peptideshaker and this is my first trial with the tool. I have attached the bug report for your review.

I would appreciate if anyone could provide me with suggestion that could be helpful in solving this issue.

Best regards, Yaarub

PeptideShaker 1.16.12 log.txt bugReport 31082017.txt

hbarsnes commented 7 years ago

Hi Yaarub,

It seem like we may have introduced a memory issue in a recent version of PeptideShaker. At least we cannot see any obvious reasons for this to occur. The good news is that we are currently refactoring the backend to greatly reduce the memory usage and generally speed up the processing. The bad news is that it will still take some time before this new version becomes available, but we will let you know as soon as a beta version is available for testing.

Best regards, Harald

Yaarub commented 7 years ago

Dear Harald,

Thank you very much for your reply. Looking forward to your notification regarding the newer version of PeptidesShaker.

Best regards, Yaarub

SMSweet commented 7 years ago

I've also seen a number of memory issues, and some other failures, from PeptideShaker. This was while trying to import search results from either 11 or 12 .mgf files, searched using 3 search engines, with pSer and pThr variable mods (in addition to OxMet). 6 GB memory was assigned to PS. This was using the GUI.

Error message below: Mon Sep 25 14:25:02 BST 2017 File import completed. 234122 first hits imported (1513908 secondary) from 96587 spectra. Mon Sep 25 14:25:02 BST 2017 [192759 first hits passed the initial filtering] Mon Sep 25 14:25:02 BST 2017 Computing assumptions probabilities. Mon Sep 25 14:25:02 BST 2017 Saving assumptions probabilities. Mon Sep 25 14:25:08 BST 2017 Selecting best peptide per spectrum. Mon Sep 25 14:26:04 BST 2017 Computing PSM probabilities. Mon Sep 25 14:26:04 BST 2017 Scoring PTMs in PSMs (D-score and PhosphoRS) Mon Sep 25 15:00:57 BST 2017 Resolving peptide inference issues.

Mon Sep 25 15:01:14 BST 2017 Ran out of memory!

Mon Sep 25 15:01:14 BST 2017 Importing Data Canceled!

hbarsnes commented 7 years ago

I've also seen a number of memory issues, and some other failures, from PeptideShaker.

This should all be fixed as soon as the new version of PeptideShaker becomes available. We are now done updating the backend code and have started updating the rest of the code to take advantage of the new and improved backend. Beta versions for testing will hopefully be available soon.

Best regards, Harald

hbarsnes commented 5 years ago

This issue should hopefully be solved in the new beta releases of SearchGUI and PeptideShaker. Please see https://groups.google.com/forum/#!topic/peptide-shaker/1ecY0IyMOBM for more details.

If the issue still exists in the beta releases, please open a new issue at https://github.com/compomics/peptide-shaker-2.0-issue-tracker/issues.

If the issue has been solved, it would also be great of you could let us know by replying to this original issue and letting us know. :)

hbarsnes commented 4 years ago

This issue should hopefully be fixed in the new release that is available now. If not, please let us know by opening a new issue.