anesvi
commented
2 years ago Fengchao: I think other things (like uncheck ‘protein level summary’ box) will not work for him. We need to pass him pre-release of FragPipe with min prob option in Percolator (set to 0.5 or even higher for that much data).
Alex: you would only need to rerun from Validation step, so it should be fast compared to the search step. And we do recommend Percolator with MSFragger-DIA (assuming this is what you run in DIA_SpecLib_Quant)
Best Alexey
From: Alexander Hogrebe @.> Sent: Wednesday, December 29, 2021 6:24 PM To: Nesvilab/FragPipe @.> Cc: Subscribed @.***> Subject: [Nesvilab/FragPipe] Philosopher filter step fatal error: out of memory (Issue #569)
External Email - Use Caution
Hi FragPipe Team,
As pointed out in my Twitter threadhttps://twitter.com/AlexHgO/status/1476273931094921221 that you kindly answered to, I am contacting you because the largest FragPipe analysis I am trying to run so far ran out of memory during the Philosopher filter step. The analysis consists of:
- FragPipe v17.1, everything up-to-date
- generation of a hybrid library consisting of a total of 550 mzML files (DDA, DIA-GPF and DIA files)
- 90min staggered window phospho runs, demultiplexed via MSConvert, resulting in ~2.2GB mzML for the 389 DIA files
- Using DIA_SpecLib_Quant default values + STY in MSFragger & "--var-mod UniMod:21,79.966331,STY --monitor-mod UniMod:21" in DIA-NN (tested with smaller amount of files)
- Analyzed on a AMD Ryzen 9 5950X with 1TB + 2TB SSD and 128GB RAM
- "fatal error: out of memory" at "philosopher.exe filter --picked --prot 0.01 --tag rev_ --pepxml D:\Alex\MSF_output --protxml D:\Alex\MSF_output\combined.prot.xml --razor" after repeated 100% RAM usage and high disk access (but reading only afaik)
It would probably work with better data filtering (e.g. the Percolator min setting pointed out in this threadhttps://github.com/Nesvilab/FragPipe/issues/538, which I do not have access to though). I tried using localization filtering via PTMprophet, but it significantly increased processing time and I think also storage space needed.
I have already successfully analyzed the data with the DDA files for library generation only, but was hoping that using the DIA files as well would give me a boost for IDs and quants. I would be curious for your suggestions, or if this is just a bit too ambitious.
Thanks a lot in advance for your help, very much appreciate this great program! Best, Alex
log_2021-12-29_13-13-17.ziphttps://github.com/Nesvilab/FragPipe/files/7790896/log_2021-12-29_13-13-17.zip
— Reply to this email directly, view it on GitHubhttps://github.com/Nesvilab/FragPipe/issues/569, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIIMM6ZMY32U7VA2C7YJICDUTOKCHANCNFSM5K6YDUZA. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you are subscribed to this thread.Message ID: @.**@.>>
Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues
fcyu
AlexHgO
guoci
prvst
Hi FragPipe Team,
As pointed out in my Twitter thread that you kindly answered to, I am contacting you because the largest FragPipe analysis I am trying to run so far ran out of memory during the Philosopher filter step. The analysis consists of:
It would probably work with better data filtering (e.g. the Percolator min setting pointed out in this thread, which I do not have access to though). I tried using localization filtering via PTMprophet, but it significantly increased processing time and I think also storage space needed.
I have already successfully analyzed the data with the DDA files for library generation only, but was hoping that using the DIA files as well would give me a boost for IDs and quants. I would be curious for your suggestions, or if this is just a bit too ambitious.
Thanks a lot in advance for your help, very much appreciate this great program! Best, Alex
log_2021-12-29_13-13-17.zip