janelia-flyem / dvid

Distributed, Versioned, Image-oriented Dataservice
http://dvid.io
Other
197 stars 33 forks source link

Improve range queries on subvolumes within very large volumes #282

Closed DocSavage closed 6 years ago

DocSavage commented 6 years ago

When getting subvolumes, a single range query is used that moves between the first TKey and the last TKey. If these are encoded ZYX block coordinates, queries that have large Z width will unnecessarily pull in key-value pairs outside the YX subvolume bounds but are within the Z bounds (same for large Y width). Instead, range queries should be broken up into spans only in ZY.

This is true for annotation data type. Also check for uint8blk/labelarray/labelmap. All of this is better handled once (i.e., efficient decomposition of n-d subvolume query into many range queries) and then reused by different data types.