search

DiltheyLab / MetaMaps

Long-read metagenomic analysis
Other
96 stars 23 forks source link

Memory usage??? #42

Closed Thomieh73 closed 3 years ago

Thomieh73 commented 4 years ago

Hi, I am trying to run MetaMaps with the small database ( miniSeq+H) that you provided.

I downloaded the database and unpacked it.

MetaMaps was installed using Conda on our HP cluster and I am using slurm to run an analysis of a dataset.

Now I see that the memory usage is pretty big (> 140Gb), and I am not really understanding why that is. The result is that I need to up my memory, since the first command I run get's killed as some point.

This is the command: metamaps mapDirectly -t 12 --all -r $MM_DB/DB.fa -q ${DATAFILE} -o ${DATAFILE}.classification_results

The last lines of the output before it gets killed:

Added C854|kraken:taxid|x709|NZ_CP016627.1 with length 2173408; est. memory ~150.235 GB
Added C10455|kraken:taxid|1266844|NC_021993.1 with length 9914; est. memory ~150.235 GB
Added C5413|kraken:taxid|x1885|NZ_CP009616.1 with length 4687494; est. memory ~150.264 GB
/var/spool/slurmd/job785761/slurm_script: line 56: 54322 Killed                  metamaps mapDirectly -t 12 --all -r $MM
_DB/DB.fa -q ${DATAFILE} -o ${DATAFILE}.classification_results

Is this just loading the database into memory?

I know I can limit the memory usage, but what is the effect of that on the analysis run?

Thomieh73 commented 3 years ago

I have solved this. closing this issue.