Overlapping, multi-resolution tiles, or lazy-loading layers

[jasondavies]

As part of my quest to build the ultimate zooming/panning scanned manuscript reader, I've found that my original plan of creating multiple po.image() layers doesn't perform well enough for large numbers of layers (>200). I presume this is due to creating a large number of SVG elements all loading the images even if I have set the transform to be off-screen.

Perhaps what I'm really after is a single multi-resolution layer i.e. where each tile can have its own transform for scaling and translation. Alternatively, if layers could be lazy-loaded so that they are only created when they overlap browser viewport, this could be a neater fix, although the non-uniform layer sizes still need to be taken into account.

The way I have set up the scanned documents is to have a separate set of uniform (256x256) tiles for each scanned document. Each document scan has a slightly different size, but I do have the exact resolution information so I can apply the appropriate scaling transform to each set of tiles. The documents in the next column and row need to overlap the previous by different amounts depending on the exact width and height.

Sorry if this is stretching Polymaps a bit too much :-)

[mbostock]

Interesting application. I'm confident you could do this in Polymaps, but getting reasonable performance if 200+ layers are visible simultaneously may be tricky.

Are all the layers visible at the same time? The first thing you could experiment with is toggling the visibility of each layer when the map moves, so that if the layer is not visible with the viewport, you hide the layer and Polymaps won't draw any of the tiles (even though the tiles are invisible with about:blank).

The next thing I would try is to customize the layer implementation (see nypl.image in the "transform" example) to use a blank <svg:g> element rather than an image with the "about:blank" URL. I'm not quite sure how the browser handles displaying an image with the "about:blank" URL, so I'm guessing that an empty group element will be displayed more efficiently than an image.

Lastly, you could write your own layer implementation from scratch, perhaps as a wrapper on top of po.layer to get the lazy-loading you desire. Or perhaps we could figure out a way for the tile.element to be null for sparse tilesets.

[jasondavies]

Thanks, turning off the visibility worked a treat when layers are off-screen, and performance is fine when zooming out and viewing all layers at once. I'll give you a sneak preview once I've uploaded the 5GB of tiles! I've also just committed an optimisation to allow null URLs instead of using about:blank as this seems slightly faster in Firefox: 5aa27172262d4ac6575095918313a64827c52472

Edit: added link to commit.

[mbostock]

Good stuff. I pulled your commit into my personal fork. It'll make the official release in a few weeks.

[jasondavies]

Okay, sneak preview is finally up: http://www.jasondavies.com/voynich/

The source code is on GitHub too: http://github.com/jasondavies/voynich

I originally had code to show everything on one map as discussed above (see the commit history if you're interested). However, I personally prefer using a navigation menu to browse between scans, so this is turned off for now.

[mbostock]

Nice! My thought, if you still wanted to have all the scans visible simultaneously, is to implement them as a single layer. You could use my Hilbert curve code to figure out how to determine which scan to display based on the tile coordinates: http://bl.ocks.org/597287

[mbostock]

You could still use bookmarks for navigation, which would move the map center, rather than replacing the layer.

[jasondavies]

Yes, I'm still planning to put it all in one map. Still need to figure out the Mercator conversion and I should have it working.

[jasondavies]

I'd prefer to have each scan as a separate layer so that I can move flush with each other as they have varying widths. My experiments with this were encouraging.

[mbostock]

You can make them flush in a single layer even if they have varying widths, though that would certainly make the tile coordinate layout more challenging. The only thing you couldn't do, as a single layer, would be to have overlapping scans.

[jasondavies]

Ah, interesting. I may also want to scale layers though as I think I can get the original scan resolutions, so I can have a uniform zoom scale for all scans. I think that would need separate layers?

I know separate layers will work anyway, sounds easier than using a custom tile coordinate layout :-)