Not enough memory allocated to MSFragger in big database

[thaumP]

First of all thank you for the development of fragpipe.

I am doing research on metaproteome and when I use metagenome as a database (4-8 million protein sequences with decoys & contaminats), fragpipe works well and even if I encounter the problem of not enough memory, I can solve it by splitting the database.

However, in one of my studies, I needed to use the cog database (version 2020 with about 6 million sequences with decoys & contaminats) as a database, but always encountered the problem of insufficient memory and could not solve it by splitting the database. In fact, I did not find the memory usage too high during running the software. Even if I install fragpipe on a server with 1.5T RAM, the problem of insufficient memory still occurs.

My computer CPU: AMD Ryzen 7 5800X 8-Core Processor 3.80 GHz RAM: 64G

---

The case of Not enough memory allocated to MSFragger cannot be solved by splitting the database log_2022-03-08_16-19-00.txt

The case of Not enough memory allocated to MSFragger can be solved by splitting the database log_2022-03-08_16-12-07.txt

[fcyu]

You have DIA data, which does not support split database. Thus, the number of split was one in your analysis.

Best,

Fengchao

[thaumP]

You have DIA data, which does not support split database. Thus, the number of split was one in your analysis.

Best,

Fengchao

Thank you. But when I use only DDA data, this still happens log_2022-03-09_07-16-33.txt

[fcyu]

Your database is too big to perform the first search. Can you change calibrate mass to off?

Best,

Fengchao

[thaumP]

Your database is too big to perform the first search. Can you change calibrate mass to off?

Best,

Fengchao

I turn off the calibrate mass, but the Not enough memory allocated to MSFragger still happens.

Further, I shortened the peptide length range, reduced the number of missed cleavage sites, and other parameters, but it was still doesn't work.

Strangely, the software terminated after taking up only a small portion of memory.

log_2022-03-09_09-08-00.txt

[fcyu]

msfragger.misc.slice-db=1

You are using 1 for split database this time.

One spike in the memory usage, which might not be recorded by the task manager, would crash the program.

Best,

Fengchao

[thaumP]

msfragger.misc.slice-db=1

You are using 1 for split database this time.

One spike in the memory usage, which might not be recorded by the task manager, would crash the program.

Best,

Fengchao

haha，it is work. Thank you！

One more question, is there any difference between the library generated using fragpipe and then quantified by the standalone DIA-NN software and DIA_speclib_quant workflow？

[fcyu]

If you use the same command and the spectral library for DIA-NN, the result should be the same.

Best,

Fengchao

[thaumP]

standalone

thank you