smith-chem-wisc / MetaMorpheus

Proteomics search software with integrated calibration, PTM discovery, bottom-up, top-down and LFQ capabilities
MIT License
90 stars 46 forks source link

Trying to reduce the memory footprint of GPTMD and also speed it up #2302

Closed trishorts closed 1 year ago

trishorts commented 1 year ago

So, gptmd is a dog for big searches. This PR eliminates PEP (which never actually did anything) and it truncates the psm list to those w/ q <0.05 rather than keeping them all around.

Also, the database update gets parallelized with the locks at the protein accession level to prevent collisions.

Results are slightly different but somehow i get different search results between current version and release when using multinotch (separate from gptmd). I don't think that I can decouple the two.

codecov[bot] commented 1 year ago

Codecov Report

Merging #2302 (b2bf36f) into master (d9ee565) will increase coverage by 0.10%. Report is 1 commits behind head on master. The diff coverage is 93.93%.

Additional details and impacted files [![Impacted file tree graph](https://app.codecov.io/gh/smith-chem-wisc/MetaMorpheus/pull/2302/graphs/tree.svg?width=650&height=150&src=pr&token=m0HXyK8tFy&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=smith-chem-wisc)](https://app.codecov.io/gh/smith-chem-wisc/MetaMorpheus/pull/2302?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=smith-chem-wisc) ```diff @@ Coverage Diff @@ ## master #2302 +/- ## ========================================== + Coverage 92.28% 92.39% +0.10% ========================================== Files 135 135 Lines 20616 20723 +107 Branches 2836 2846 +10 ========================================== + Hits 19025 19146 +121 + Misses 1106 1095 -11 + Partials 485 482 -3 ``` | [Files Changed](https://app.codecov.io/gh/smith-chem-wisc/MetaMorpheus/pull/2302?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=smith-chem-wisc) | Coverage Δ | | |---|---|---| | [MetaMorpheus/TaskLayer/GPTMDTask/GPTMDTask.cs](https://app.codecov.io/gh/smith-chem-wisc/MetaMorpheus/pull/2302?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=smith-chem-wisc#diff-TWV0YU1vcnBoZXVzL1Rhc2tMYXllci9HUFRNRFRhc2svR1BUTURUYXNrLmNz) | `77.57% <57.14%> (-1.65%)` | :arrow_down: | | [MetaMorpheus/EngineLayer/Gptmd/GptmdEngine.cs](https://app.codecov.io/gh/smith-chem-wisc/MetaMorpheus/pull/2302?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=smith-chem-wisc#diff-TWV0YU1vcnBoZXVzL0VuZ2luZUxheWVyL0dwdG1kL0dwdG1kRW5naW5lLmNz) | `99.40% <100.00%> (+10.06%)` | :arrow_up: | | [MetaMorpheus/EngineLayer/Gptmd/GptmdResults.cs](https://app.codecov.io/gh/smith-chem-wisc/MetaMorpheus/pull/2302?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=smith-chem-wisc#diff-TWV0YU1vcnBoZXVzL0VuZ2luZUxheWVyL0dwdG1kL0dwdG1kUmVzdWx0cy5jcw==) | `100.00% <100.00%> (ø)` | | ... and [5 files with indirect coverage changes](https://app.codecov.io/gh/smith-chem-wisc/MetaMorpheus/pull/2302/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=smith-chem-wisc)
trishorts commented 1 year ago

inexplicably slower

trishorts commented 1 year ago

not working so far. don't know why