minetest / minetestmapper

Generates a overview image of a minetest map.
BSD 2-Clause "Simplified" License
112 stars 40 forks source link

Issue with performance #69

Closed scr267 closed 4 years ago

scr267 commented 5 years ago

I had performance issues with this version of minetestmapper running on a Virtual Machine that had 6 Gbs of RAM, 4 CPUS and the VM disk running on an SSD benchmarked at 600Mb/s transfer. When I ran minetestmapper, I couldn't find any I/O or CPU bottleneck. However, here's what I found:

NOTE: Most of these benchmarks were made running with est31's leaftest tiling script. This script calls minetestmapper with specific geometry parameters.

Basically I ran some tests and found that the forked minetestmapper from adrido is much faster, at least in my case. https://github.com/adrido/minetest-mapper-cpp

I benchmarked a single 1024,1024 tile (not called by leaftest) between the original minetest/mintestmapper project vs adrido's fork and ended up with these numbers:

Original: 20 seconds Adrido's: 9 seconds

It's slightly more than 100% faster, but I also found something else that was interesting: When Adrido's fork is used in combination with est31's leaftest script I get even more improvement in speed. I think the reason is due to the way Adrido's handles unexplored portions of map (blank.) It doesn't seem to process them at all and skips through them. A single tile which is sized at around 256 would take approximately 3 ~ 5 seconds to generate with the original mapper. With adrido's mapper, the same took:

0.035s realtime for blank tiles. 0.245s ~ 1.6s realtime for explored tiles with data.

The 1.6s time didn't occur very often, most tiles that had data took in the vicinity of 0.245s ~ 0.5s.

All in all, it took the same VM 30 minutes to generate the map. With the original mapper it took ~4 hours to generate a map half the size.

On a side note that I did run into a problem with the compilation of adrido's version. It seems one of the functions from the experimental 'filesystem' library was not found, this may be an environmental issue as I compiled this on Fedora 29. I realize it has nothing to do with minetest's own minetestmapper repo but it might be good to note.

sfan5 commented 4 years ago

Should be fixed by 7ff22886276667b3211d791c70066501ffadac7f, depending on the options you set (--min-y, --max-y).