Preprocessing memory issues - "0.65"

[philkingston]

I have been attempting to produce a dataset using great_britain.osm.pbf on a system with 8GB. I was receiving a memory exhaustion error during the 'preprocessing' stage of osrm_prepare:

preprocessing .... 10% . 20% . 30% . 40% . 50% . 60% terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
Aborted (core dumped)

Digging into the code, the memory problem occurs in Contractor/Contractor.h around line 246:

while ( numberOfContractedNodes < numberOfNodes ) {
            if(!flushedContractor && (numberOfContractedNodes > (numberOfNodes*0.65) ) ){
               ....

Why 0.65? Reducing the value to 0.5 completes the processing. It would seem to me that the code was written to suit one particular data file with one particular system specification. Would it not be more logical to set this to 0.5? Then, the memory consumption would be split equally and use less overall memory than if this value was set to any other value. Alternatively, it would be possible to do the flushing of the data more than once, so even less memory was used.

Any thoughts?

Thanks, Phil

[DennisOSRM]

The value 0.5 is as arbitrary as 0.65. Although the values may imply some symmetry in reality the memory consumption is only lowered and not halved (or cut by two thirds). But if it helps then please go ahead and change it. Flushing the memory more than once doesn't help too much for memory consumption.