slim mode for MapnikRenderer preprocessor

[GoogleCodeExporter]

Currently, MapnikRenderer preprocessor requires huge amounts of memory, which 
simplify and speed up multiprocess. For huge extensions, 64bit capacity is 
required. 
A slim mode would benefit a lot of "common" users. The main problem could be 
multiprocess in slim mode. But, in most cases, single process could just be 
fine.

Original issue reported on code.google.com by fever...@gmail.com on 17 Dec 2010 at 7:00

[GoogleCodeExporter]

You can adjust how much memory the MapnikRenderer plugin uses:

 - Adjust the amount of threads
 - Adjust the size of meta tiles

Each thread uses 256*256*3*(meta-tile-size+1)*(meta-tile-size+1) bytes.

Does this fix your problem? Or am I misunderstanding your suggestions of a 
"slim mode"?

Original comment by veaac.fd...@gmail.com on 17 Dec 2010 at 9:33

• Added labels: Type-Enhancement
• Removed labels: Type-Defect

[GoogleCodeExporter]

I'm afraid not only the thread memory matters.
For example, the std container used in preprocessing is likely to overflow the 
std::size_t capacity on 32 bit platform for relative large dataset.
I've modified a slim version based on the old preprocessor code(simply disable 
OMP and directly write the index to the disk), which works fine on huge data 
set and should works on most pc.

Original comment by fever...@gmail.com on 17 Dec 2010 at 11:17

[GoogleCodeExporter]

How large are the areas you want to render? And up to which zoom level?

Each index entry should be less than 16 Bytes. That means Germany will take 
about 20MB for zoom level 15, 100MB for zoom level 16. Of course, if you try to 
render the whole of Europe it will take about 1GB, but Europe shouldn't fit on 
most mobile devices anyway ( >35GB data ).

Furthermore, if the index data structure is problematic for you, shouldn't the 
OSMImporter also be a problem for your? Each node's coordinate takes about 8 
Bytes to store the coordinate. At least at one point the importer has to copy 
and reorder these coordinates.

Original comment by veaac.fd...@gmail.com on 17 Dec 2010 at 12:51

• Changed state: Accepted

[GoogleCodeExporter]

The china.osm from official site is very small(62MB bzip). So OSMImporter works 
just fine.
I'd like to render China(almost as large as Europe) to at least zoom 16. 
Although most of the land in osm are empty, the index entry always needs to 
occupy its place.

Original comment by fever...@gmail.com on 17 Dec 2010 at 2:58

[GoogleCodeExporter]

Original comment by veaac.fd...@gmail.com on 18 Apr 2011 at 3:01

• Changed state: WontFix