This adds a new type of job, called "renderbatch", where each job has the same set of sources. This happens frequently with high-zoom tiles and seems to be a good way to enforce re-use. This is done using a GroupingDispatcher which maintains a map of the sources to the list of output tiles which share the same set.
Running this on the dev cluster shows a 14x improvement.
This adds a new type of job, called "renderbatch", where each job has the same set of sources. This happens frequently with high-zoom tiles and seems to be a good way to enforce re-use. This is done using a
GroupingDispatcher
which maintains a map of the sources to the list of output tiles which share the same set.Running this on the dev cluster shows a 14x improvement.
Connects to #67.
@rmarianski could you review, please?