MBTiles Support

[GoogleCodeExporter]

This is an open issue for mousebird consulting and Development Seed to discuss 
MBTiles support.

Here's the quick low down.  MBTiles is a nice, simple way of expressing an 
image pyramid.  We'd like to support that in 1.3 and now we just need to hash 
out the specifics.

Original issue reported on code.google.com by s...@mousebirdconsulting.com on 24 Jan 2012 at 4:22

[GoogleCodeExporter]

(pasting parts of an email to Steve in an effort to move the discussion over to 
here)

Since I hadn't dabbled in the actual code to WhirlyGlobe before, I spent a bit 
of time on that today and basically did what (I think) Will got working some 
time back (he's traveling today, but I wanted to do this for myself anyway). 
See attached for a screenshot. The obvious problem here, which I'm pretty sure 
we identified before, is un-projecting the source. But regardless, this is made 
of 256 tile images, each 256px square, for zoom level 4, exported to PNGs on 
disk from an MBTiles via mbutil, popped into the app with a custom plist, and 
with setting the project to not compress PNGs. Here's a link to the PNGs & 
their plist, if you'd like to play around yourself: 
http://dl.dropbox.com/u/575564/geoclass.zip

Now that I've gotten this (minimally) far, the problems I see are this (and 
sorry if this is old news to you): 

1. Creating un-projected images. Our entire stack is centered around Web 
Mercator. This is likely in our court. 
2. Connecting the pinch gesture recognizer, essentially, for the globe view to 
link its heightAboveGlobe to the MBTiles z-level, in order to pull new images 
for different "zooms". This seems trivial (maybe?)
3. Make the GL view performant enough to refresh the textures based on these 
changes. My guess, based on the initial render time now, is that this is 99% of 
the problem space as you've identified it. 

Original comment by jus...@developmentseed.org on 24 Jan 2012 at 5:43

Attachments:

• [Screen Shot 2012-01-23 at 6.29.48 PM.png](https://storage.googleapis.com/google-code-attachments/whirlyglobe/issue-1/comment-1/Screen Shot 2012-01-23 at 6.29.48 PM.png)

[GoogleCodeExporter]

Of course, the final result would read these tiles directly out of MBTiles -- I 
was just working within the unmodified library for now. 

Original comment by jus...@developmentseed.org on 24 Jan 2012 at 5:45

[GoogleCodeExporter]

That sounds like an excellent start.  Putting that together certainly clarifies 
what needs to be done.

In WhirlyGlobe terms, we need a data layer.  By default you were using 
Spherical Earth layer, which is not cognizant of any other textures or of a 
coordinate system other than unrolled lat/lon.

The new data layer (WhirlyGlobeMBTileLayer) would be responsible for the 
following:
- Loading individual tiles of imagery
- Constructing individual tiles of geometry
- Unloading tiles as the viewer moves around

Matching that up to your points, here's what that means:
1.  Web Mercator.  I have a Mercator implementation from another project I can 
pull in.  It looks like Web Mercator isn't all that complicated.  I can take 
this.
2.  Loading data based on height above ground.  This is actually the tricky 
part.  WhirlyGlobe data layers don't watch the globe view right now to see what 
it's up to.  I can do something simple for now, I think.
3.  OpenGL textures swapping.  This isn't as bad as you might think.  I have 
code to completely swap out the Texture Group in a spherical earth layer right 
now.  Similar concept.

That will do the trick for an MBTiles archive that covers the whole globe.  It 
won't address ones that cover just part of the globe.  It also won't address 
pulling data remotely.  If we can skip that for the first go, that would save 
some time.

What I'm signing up for is a brand new WhirlyGlobe data layer that reads from 
existing static MBTiles archives.  I'll probably just read them directly, 
rather than going through another API for now.

So the remaining questions are:
Will that do what you want?
When do you need it?

I have a bunch of comments about optimization as well, but let's wait until we 
actually have something working.

Original comment by s...@mousebirdconsulting.com on 24 Jan 2012 at 7:37

[GoogleCodeExporter]

Thanks for the followup. The new data layer bit makes sense, but the only thing 
I'm not clear on is the projection issue. The tiles in a given MBTiles are 
based on a Mercator projection image that's been cut up, and are why the above 
image is somewhat distorted. In order to reflect things properly, I think this 
would involved re-stitching the tiles, re-projecting (or rather, un-projecting) 
the large image, and re-tiling. I'm not sure how workable that is, but maybe 
I'm missing something about how are you are seeing this work. 

As for scope & timeframe, I think that yes, full-world local tilesets are a 
good first milestone. What's left would be a pretty good working library. We 
don't have any hard timeline on it. What kind of timeframe are you thinking? 

Original comment by jus...@developmentseed.org on 24 Jan 2012 at 9:41

[GoogleCodeExporter]

You're right about the reprojection, of course, but there's more than one way 
to resample image data.

The way the new layer will work is this.  It'll iterate through the tiles it 
needs to display in Web Mercator.  Tiles are constructed with an image and a 
set of triangles to draw.  Those are the triangles you might now see in the 
Spherical Earth layer if we were to turn on wireframe.  The vertices for those 
triangles have texture coordinates mapped from (0,0)->(1,1) across the image 
tile.  Easy enough so far.

Here's the reprojection part.  We transform the triangle vertices from Web 
Mercator through geodetic and then to our 3D display space.  That's how we do 
the resampling.  If we see distortion in the process, we can throw more 
triangles in based on the latitude.  Triangles are cheap.

Mathematically, this is basically a reprojection at key points and then affine 
transforms within each triangle.  Just, ya know, the graphics hardware is doing 
that last part.  If anyone cares, we could characterize the error 
mathematically and subdivide to a well defined tolerance.  That's way, way 
beyond what most users need, though.

As for timeline, I'm reasonably flexible at the moment.  I could commit to 
something a few weeks out.
Do you have a data set you'd like to use for testing?

Original comment by s...@mousebirdconsulting.com on 24 Jan 2012 at 9:56

[GoogleCodeExporter]

I think I follow the projection explanation, but let me rephrase to make sure. 
Basically the triangle vertices are mapping to geo points, and then those are 
transformed, essentially distorting the tiles, but that matters less as long as 
we have enough vertices to get things reasonably into place. That about right? 

Personally I'd like to see Geography Class[1] since it's a cool-looking set, 
but the folds might end up looking weird. World Bright[2] or World Light[3] are 
a little less flashy, but provide more levels of detail, so they might also be 
a good option. 

[1] http://tiles.mapbox.com/mapbox/map/geography-class
[2] http://tiles.mapbox.com/mapbox/map/world-bright
[3] http://tiles.mapbox.com/mapbox/map/world-light

Original comment by codesorc...@gmail.com on 24 Jan 2012 at 10:01

[GoogleCodeExporter]

Gah, above comment by me -- stupid multiple Google accounts. 

Original comment by jus...@developmentseed.org on 24 Jan 2012 at 10:05

[GoogleCodeExporter]

That's right.  We're just sampling with the vertices, but in a different way.

I like geography class.  I'll use that for testing.

Original comment by s...@mousebirdconsulting.com on 24 Jan 2012 at 10:10

[GoogleCodeExporter]

Also, regarding actually pulling the tiles, FMDB[1] is what we use in Route-Me 
right now. For single-thread reads, it works out well. I've started playing 
around with a fork[2] of Route-Me that is based on CATiledLayer and uses 
multiple threads to read in tiles. I've had good luck with the threadtests 
fork[3] of FMDB in this regard, which introduces queue-based reading and it 
performs quite well[4]. Let me know if you'd like any code help or info here. 

[1] https://github.com/ccgus/fmdb
[2] https://github.com/Alpstein/route-me
[3] https://github.com/ccgus/fmdb/tree/threadtests
[4] http://dl.dropbox.com/u/575564/fastass.mov

Original comment by jus...@developmentseed.org on 24 Jan 2012 at 10:15

[GoogleCodeExporter]

Thanks, I'll look into that after the first pass.

WhirlyGlobe uses two threads, the main one for rendering and a second one for 
data interaction and loading.  For the first pass I'll just make careful use of 
selectors and direct reads in the interaction/load thread.

When we start pulling data over the network, that'll introduce longer latencies 
and I'll have to actually give it some thought.  That's for a later version, 
though.

Original comment by s...@mousebirdconsulting.com on 24 Jan 2012 at 10:21

[GoogleCodeExporter]

Is there any way to tell what the image size is for a given tile other than 
reading it?
If not, are all the tiles always the same size in a MBTiles archive?

Original comment by s...@mousebirdconsulting.com on 24 Jan 2012 at 10:54

[GoogleCodeExporter]

Yeah, it's not technically in the spec[1] but in practice we always use 
256x256px. This might change in retina land but for now that's been the case. I 
think that's a safe assumption along with the above ones we've made (full 
world, non-network, etc.)

[1] https://github.com/mapbox/mbtiles-spec

Original comment by jus...@developmentseed.org on 24 Jan 2012 at 10:59

[GoogleCodeExporter]

Is "Web Mercator" the same thing as "Spherical Mercator"?
http://wiki.openstreetmap.org/wiki/Mercator#Spherical_Mercator

Original comment by s...@mousebirdconsulting.com on 25 Jan 2012 at 12:42

[GoogleCodeExporter]

Yes, they're the same thing. Sometimes also called Google Mercator. 

Original comment by jus...@developmentseed.org on 25 Jan 2012 at 12:51

[GoogleCodeExporter]

Okay, I've put together a first cut.

The WhirlyGlobeMBTileLayer layer takes a WhirlyGlobeMBTiles object and a level. 
 It'll pull the tiles out for that given level at startup.

It's taking the spherical mercator projection into account and tiling based on 
your scheme.  It's not dynamically loading, though.  That will be considerably 
harder.

The images attached are as follows:
Whole Earth - Zoomed out, showing the whole at the top, since the data doesn't 
cover the whole earth.
Algeria - A close up of the level 4 tiles.
Vectors Overlaid - How does vector data match up?  Perfectly.
Lofted Polys - An example of the lofted polygon layer working over the top.

The new layer is checked in.  You can see code for activating it in 
WhirlyGlobeTester in the trunk.  Just turn on UseMBTiles in the header for 
GlobeViewController.

This should be enough for you to get started.  I can fill in dynamic paging 
when you're farther along.

Original comment by s...@mousebirdconsulting.com on 30 Jan 2012 at 10:54

Attachments:

• [Whole Earth.png](https://storage.googleapis.com/google-code-attachments/whirlyglobe/issue-1/comment-15/Whole Earth.png)
• Algeria.png
• [Vectors Orverlaid.png](https://storage.googleapis.com/google-code-attachments/whirlyglobe/issue-1/comment-15/Vectors Orverlaid.png)
• [Lofted Polys.png](https://storage.googleapis.com/google-code-attachments/whirlyglobe/issue-1/comment-15/Lofted Polys.png)

[GoogleCodeExporter]

Awesome, checking this out now. 

Original comment by jus...@developmentseed.org on 1 Feb 2012 at 6:03

[GoogleCodeExporter]

I haven't built from trunk yet. I was able to get most of required libs via 
your starter pack (then adjust the header search paths), but you're also 
missing shapelib from trunk. 

Original comment by jus...@developmentseed.org on 1 Feb 2012 at 6:23

[GoogleCodeExporter]

I got this off the ground. Very nice. 

Original comment by jus...@developmentseed.org on 1 Feb 2012 at 6:56

[GoogleCodeExporter]

Trunk's missing all the external libraries.  There's a readme in there telling 
you to go get them.  For iOS development everyone seems to want everything in 
one package, which kind of conflicts with open source habits.  I could put 
shapelib in the starter pack, though.  I'll make a note.

But let's just do this.  Here's the framework:
http://dl.dropbox.com/u/29069465/WhirlyGlobe.framework.2_1plusplus.zip

Original comment by s...@mousebirdconsulting.com on 1 Feb 2012 at 6:58

[GoogleCodeExporter]

Yeah, I worked it out. I don't see the readme in the WhirlyGlobeLib trunk, but 
I worked through the build errors to figure it out. 

Original comment by jus...@developmentseed.org on 1 Feb 2012 at 7:01

[GoogleCodeExporter]

My mistake, notes.  Doesn't matter, though.

Okay, great!  That should be enough to support your development.  The biggest 
remaining issue is paging tiles.  The longer I think about that, the simpler 
the algorithm will be.

There are some minor issues around tile edges.  I need to step through my 
texture mapping logic and make sure I'm clamping everything correctly.  Should 
be minor, though.

Let me know what your development schedule looks like and we'll plan 
accordingly.  Of course, you can always move over into the "client" queue with 
cash if urgency is an issue.  If not, we can do this more on an open source 
schedule.

Original comment by s...@mousebirdconsulting.com on 1 Feb 2012 at 7:06

[GoogleCodeExporter]

Got it. I will put this through its paces and see how it fits into our app. If 
you're interested in following that, it's over here: 
https://github.com/incanus/Mapresent

Original comment by jus...@developmentseed.org on 1 Feb 2012 at 7:08

[GoogleCodeExporter]

I'm considering finishing up the MBTiles support soon.  Did you guys have any 
feedback on the first version?

Original comment by s...@mousebirdconsulting.com on 19 Mar 2012 at 7:15

[GoogleCodeExporter]

Hey Steve. I haven't had as much chance to integrate it yet as I would have 
liked, as we've been busy in many fronts. From what I did see, it's solid -- 
easy to put in, and we know already about the limited zoom support, the missing 
poles, and the startup time. I need to do more with integrating it into our app 
and getting/putting settings for the view before I can report more. 

Original comment by jus...@mapbox.com on 19 Mar 2012 at 7:24

[GoogleCodeExporter]

I think I'm way ahead of you now:
http://mousebirdconsulting.blogspot.com/2012/04/mapbox-tiles-paging.html

Original comment by s...@mousebirdconsulting.com on 12 Apr 2012 at 3:58

[GoogleCodeExporter]

Indeed -- this is amazing. Trying to catch up :-)

Original comment by jus...@mapbox.com on 12 Apr 2012 at 5:26