tilezen / joerd

Joerd can be used to download, merge and generate tiles from digital elevation data
MIT License
324 stars 50 forks source link

Group similar jobs #70

Closed zerebubuth closed 8 years ago

zerebubuth commented 8 years ago

This adds a new type of job, called "renderbatch", where each job has the same set of sources. This happens frequently with high-zoom tiles and seems to be a good way to enforce re-use. This is done using a GroupingDispatcher which maintains a map of the sources to the list of output tiles which share the same set.

Running this on the dev cluster shows a 14x improvement.

Connects to #67.

@rmarianski could you review, please?

rmarianski commented 8 years ago

:+1: