Memory usage of PeptideShaker?

ProtProtocols / IsoProt

Protocol of the analysis of iTRAQ/TMT proteomics data including quantification, statistical analysis and maybe clustering

https://protprotocols.github.io

11 stars 4 forks source link

Memory usage of PeptideShaker? #44

Closed veitveit closed 6 years ago

veitveit commented 6 years ago

It happened already several times that I didn't get to the tsv file. I assume that there is a memory exception from either the data processing or the generation of the tsv file. This happens only sometimes and I usually get the files when redoing the "run my search"

Let's keep an eye on this and maybe reduce memory usage to see whether this caused the error.

veitveit commented 6 years ago

This seems to be a big problem on Windows machines.

veitveit commented 6 years ago

@jgriss

Still occurring including Linux machines (see attached log file). The problem is that the script looks for the currently available free memory and does not just take a share of the total memory on the computer. This often seems to lead to situations where the "-Xmx" option gets assigned an amount well below 1GB.

search_processed.log

veitveit commented 6 years ago

@jgriss

How about:

Test for total and free memory on the machine
Assign a minimum of 2GB and throw a warning when the free memory is below that

But this might not be the end of this story. We tested also fixed 2GB (by setting the value in the script) and PeptideShaker threw an error as well (not sure whether it was the same though).

The free memory that you are testing for in the script is very variable and most computers automatically use the entire RAM but free sufficient resources if they are requested.

jgriss commented 6 years ago

@veitveit I believe that I have found the bug that caused our issues. Now, PeptideShaker gets at least 1GB of memory. Also, we now try to allocate all of the freely available memory.