MosaikJump: Cannot use more than 40 GB RAM using the -mem parameter

[GoogleCodeExporter]

What steps will reproduce the problem?

1. Run MosaikJump with option -mem 64 set on a machine with 128 GB of RAM

MosaikJump -ia reference/hg18.dat -out reference/hg18 -hs 15 -mhp 100 -mem 64

What is the expected output? What do you see instead?

No matter what value above 40 GB you specify to MosaikJump, it only
allocates 40 GB of RAM. Once it uses all this memory it starts writing to
temporary files under MOSAIK_TMP.

What version of the product are you using? On what operating system?
MosaikJumpESC 1.0.1307
CentOS release 5.3 (Final)
2.6.18-128.1.14.el5

Please provide any additional information below.

Original issue reported on code.google.com by bro...@gmail.com on 10 Nov 2009 at 8:35

[GoogleCodeExporter]

Ah this is a bit of a misnomer. The -mem parameter in MosaikJump (also in 
MosaikMerge
and MosaikSort) specify how much memory to use for temporarily storing and 
sorting
records before writing temporary files to disk.

In most cases, it should be left at the default value unless you want to 
increase it
slightly. At some point however it takes more time to sort n elements in memory
(usually an n log n operation) than writing the chunk of memory to hard disk - 
so
bigger is not always better in this case.

You have a valid point though, I should check for an upper bound when parsing 
the
command line parameters. My guess is that it should fail at 32 GB RAM - so I'll 
make
that the upper bound.

Thanks for bringing this to my attention.

Cheers,

// Michael

Original comment by snowneb...@gmail.com on 16 Jan 2010 at 2:33

• Changed title: MosaikJump: Cannot use more than 40 GB RAM using the -mem parameter
• Changed state: Accepted