compomics / searchgui

Highly adaptable common interface for proteomics search and de novo engines
http://compomics.github.io/projects/searchgui.html
38 stars 16 forks source link

MS file exceeds memory mapped reader max buffer size. #356

Closed milnus closed 2 months ago

milnus commented 11 months ago

Hello, I have a set of files n=12 for my full set of spectra and n=6 for one condition (part of the full set). I am working with Searchgui v. 4.2.17 installed via conda (see yml for environment information - searchgui_environment_yml.txt)

As I am running on a linux based machine, I am running searchgui through the command-line interface. I have tried running searchgui with my full set of files, n=12, but get this Error: 'File exceeds memory mapped reader max buffer size.' (full output - full_set_output.txt). When running with only one condition, n=6 files, I get the exact same error (one_condition_error.txt). I have tried to increase the memory allocated to the Java Virtual Machine, however this is fairly finicky in conda and does not seem to solve the problem (if the way I do it work). What is even more stange and raised my eyebrows is the fact that searchgui seem to 'run out of memory' at n-1 files in both instances, both for the full set of files and for one condition.

Any explanation on why this may be or wrongdoing on my part?

Cheers, Magnus

mvaudel commented 11 months ago

Hi Magnus and thanks for the detailed report.

I am afraid you are not doing anything wrong. It seems that spectrum files are too large and our index runs out of capacity. A short-term fix is to split the spectrum files - the easiest for this is to convert them to mgf and then split the files. I am working on a generic fix but that will take a bit more time.

One thing that I was wondering also is whether these are DIA files? SearchGUI is made to handle DDA files, I am not sure how Tide handles DIA.

Hope it helps,

Marc

hbarsnes commented 2 months ago

I've just released SearchGUI version 4.3.8 that should take care of the problem with the buffer size. If this is not the case, please let me know and I'll reopen the issue.