DaehwanKimLab / centrifuge

Classifier for metagenomic sequences
GNU General Public License v3.0
246 stars 73 forks source link

Sharing one index between several processes, over time. #135

Closed Adoni5 closed 5 years ago

Adoni5 commented 6 years ago

Hi, I suppose this is less of issue and more of an open question.

I am trying to run several centrifuge processes over a period of time, but from only one index. So ideally I'd like to load the index and then have all the processes reference that one index during classification. I have limited memory and processing power, so I was looking at the --mm option. However I was a little unclear on how it works.

Does using the --mm flag mean that a centrifuge-class process loads the index into memory and then any subsequent classification process use that pre-loaded index once that process is finished, or do they share it in real time?

Once the classifications are finished I assume the index is removed from memory. Is there a way to leave it loaded?

Also, can someone explain how --mm works if the index doesn't fit into memory? Is the index placed into virtual memory instead?

I know that's a lot of questions, so I appreciate any help!

Thank you

Rory

mourisl commented 6 years ago

If you have root right, maybe you can try ramdisk as mentionedn in #65 .

Adoni5 commented 5 years ago

Coming back to this #95 was the solution I needed. Thank you very much, sorry I never closed this!