search

makinacorpus / landez

Landez operates with tiles, arranges them together into images or builds MBTiles files
GNU Lesser General Public License v3.0
135 stars 52 forks source link

Slow to render mbtiles compared to Tilemill #46

Closed PeterSprague closed 9 years ago

PeterSprague commented 9 years ago

I found landez very slow, as in days, compared to Tilemill to render a 6-15z custom OSM baselayer of southern British Columbia, Canada. Tillmill took 3 days, and I shut down landez after 5 while it was still on z15.

Are there any options or methods to increase the rendering speed?

Is there a way to "parallel" process the rendering, aka render Z6 as process, render Z7 as four processes, Z8 as 16 processes, ... , then combine all the tiles at end? Does this even make sense? Could the code be rewritten in Twisted to perform faster? Or maybe the bottlenecks are elsewhere?

Peter

leplatrem commented 9 years ago

Thank you for this feedback !

Indeed, you caught it, Landez is not multiprocess ! Also, Tilemill is smart : when a tile is empty, it won't render the pyramid of tiles in lower zooms. See https://www.mapbox.com/blog/rendering-the-world/

Without going as far as rewriting with Twisted, I believe that using the multiprocessing module in stdlib would be enough to start. Detecting empty tiles (i.e. a uniform color) could be easy using PIL.

If you want to start a proof-of-concept, we would happy to review it as a pull-request!

Otherwise, another option is to render with Tilemill and then just use Landez to merge MBTiles files.

Good luck :)

PeterSprague commented 9 years ago

Thanks for the reply.

Not really a coder but a map developer, so this would be a stretch. May give a go this summer or fall if I can find some time to flesh out the idea and study your code.

I like the idea of having the tools in python so I can try them on some sparc servers.