Tile reload issue

[fotiDim]

I am using MaplyQuadeImageTilesLayer. I generate tiles dynamically on runtime. At some point I perform routing and my tiles engine returns updated tiles with directions drawn on top. At this point WhirlyGlobe does not automatically update the tiles so I call the reload function. However it seems to reload all tiles from the outer zoom till the current. It seems like an unnecessary process. I am attaching a video to demonstrate the issue. Password is "navideck"

http://vimeo.com/74924762

[mousebird]

We load the intermediate levels first so we can zoom out quickly. That is by design, but I realize it's an unpopular way of doing it.

I have a client who's paying for a rewrite of this part of things, so look for an updated version in a couple of weeks.

[mousebird]

Those features are begging to be vectors. But I understand if you've got a lot invested in the image based rendering.

It occurs to me there's another way to do this. Rather than have a single tile source where you're rendering your features on top of the map tiles, you could have two tile sources. One for the map tiles and one for an overlay. Let the map tile source work as normal and force a reload of the overlay tile source.

It's pretty easy to do overlays in WhirlyGlobe-Maply. Just create a second layer and give it a priority higher than the base layer.

[SudoPlz]

One possible solution to prevent the loading of irrelevant tiles would be to

1) Set the allowed minZoom and maxZoom, to your current zoom (thus the getTileBitmap function gets called ONLY for the current zoom lvl)

2) run Reload

3) Set the minZoom and maxZoom values back to what they used to be before the reload.

What stops me from using that solution is that the

 - (int)minZoom
 {
     return customMinZoom;
 }

and

 - (int)maxZoom
 {
     return customMaxZoom;
 }

functions (implemented in the CustomTileSource) only get called when the CustomTileSource gets initialized therefore the object responsible for calling the getTileImage function does not re-check the min/max values each time it asks for a tile..

_How could we get past this limit?_ I really need to prevent getTileImage to run on zoom lvls lower than the one I'm currently in (thus solving the reload problem).

p.s: @mousebird, quick question: Why do the minZoom and maxZoom have to be functions, why not properties (which require @synthesize) like on MapBox or RouteMe?

[mousebird]

The tile sources don't have that sort of control. We just use them querying data, they don't force the layer to do things. Once the min and max zoom levels are queried, they're baked into internal data structures that don't change.

Max zoom level might be conceptually changed by returning failure for those tiles, but you can never change min. In short, it doesn't work like this.

I made min and max zoom functions and not properties because it's a protocol. I don't require the tile sources to let you set them. If they're read only properties, then it's really just a syntax difference anyway.

You can't get around what you don't like about reload this way, I'm afraid. There's an upgrade to reload coming in a week or two. I suggest waiting for that.

[SudoPlz]

The tile sources don't have that sort of control. We just use them querying data, they don't force the layer to do >things. Once the min and max zoom levels are queried, they're baked into internal data structures that don't change.

I've noticed that indeed.

Max zoom level might be conceptually changed by returning failure for those tiles, but you can never change min. >In short, it doesn't work like this. I made min and max zoom functions and not properties because it's a protocol. I don't require the tile sources to let >you set them. If they're read only properties, then it's really just a syntax difference anyway.

True.

You can't get around what you don't like about reload this way, I'm afraid. There's an upgrade to reload coming in a >week or two. I suggest waiting for that.

Cool I'll be waiting for that upgrade..! :+1:

[fotiDim]

Assuming that we have the possibility to switch to vector format... Our tile rendering & routing engine currently uses a proprietary vector maps format and then uses it to render raster tiles. We have the following options:

• We could upgrade it to provide vector tiles and then feed them into WhirlyGlobe, if WhirlyGlobe supports it.
• We could change the map format to something else (SQLlite, geojson, shapefiles) and feed simoultaneously to our routing engine and WhirlyGlobe. I have seen your port here: http://mousebirdconsulting.blogspot.gr/2012/11/whirlyglobe-component-vectors.html. What kind of format do you suggest to put our hopes into? Shapefiles have the reputation of being slow but I don't know how do they compare against geojson. SQLite sounds like a good option if it possible for WhirlyGlobe to handle it.

[mousebird]

If you have individual tiles and a reader for this format, I'd suggest create a MaplyQuadPagingLayer and implementing the MaplyPagingDelegate protocol. The OSM example does just that: http://mousebirdconsulting.blogspot.com/2013/09/vectors-openstreetmap-and-maply.html

If you wanted to convert to another format, geojson would work. The OSM example is using GeoJSON so you could just lift it for your purposes. Shapefiles are actually fine. They're binary and aren't too slow, but you'll get a million little files if you're doing tiles.

I'd like to do SpatiaLine (spatial extensions to SQLite) personally, but I just haven't gotten around to it. If you're in a hurry, it wouldn't take a little time.

[fotiDim]

I tried loading .shp maps but with no luck. I haven't found the proper way to load them. Can you provide some instrunctions?

This is my filelist for a single country's map: buildings.cpg buildings.dbf buildings.prj buildings.shp buildings.shx landuse.cpg landuse.dbf landuse.prj landuse.shp landuse.shx natural.cpg natural.dbf natural.prj natural.shp natural.shx places.cpg places.dbf places.prj places.shp places.shx points.cpg points.dbf points.prj points.shp points.shx railways.cpg railways.dbf railways.prj railways.shp railways.shx README roads.cpg roads.dbf roads.prj roads.shp roads.shx waterways.cpg waterways.dbf waterways.prj waterways.shp waterways.shx

[mousebird]

MaplyVectorDatabase vectorDataBaseWithShape: MaplyVectorDatabase fetchAllVectors

[fotiDim]

Wouldn't I need to create a layer out of the MaplyVectorDatabase? This is the most confusing part. With the MaplyQuadImageTilesLayer we defined a tile source and then added this layer to the view. How can this be done with the MaplyVectorDatabase?

[mousebird]

Look for the addVectors: methods in the view controller. There's an example in the WhirlyGlobeComponentTester app.

For these sorts of vectors you'll want to add them all at once. If you need to do tile based paging, then you have to set up MaplyQuadPagingLayer as I mentioned above. However your data must be correctly structured for that and it doesn't look like it is from that listing.

There's no layer for vectors and other features. They're handled by Managers (which you don't see) that can be called from any thread.

[mousebird]

There's an update in the develop branch. That should respect reloads that only pull from local storage. If you go out to the network, all bets are off, and it'll act like it always did, albeit a bit faster.

[SudoPlz]

@mousebird

1) Problem still not resolved:

• I tried your solution and I must say, it works very nice on the simulator, BUT when running on an Ipad 4 I have the same issue as before the update..! Tiles are still loading from lvl 1 till the current level.

2) A few more cases in which the problem occurs:

The exact SAME problem as described on this issue (tiles loading from lvl 1 till the current level) can occur in each of the cases below:

• Initializing the globe with a height closer to the earths surface (lets say for example 0.00009) (loads all the tiles not just the current zoom lvl).
• Upon zooming in (pinch) (loads all the tiles not just the current zoom lvl).

[mousebird]

Is this with a remote tile source or a local tile source?

[SudoPlz]

local... to be specific MaplyTileSource.

[mousebird]

Is this a custom tile source and, if so, are you filling in the isTileLocal method? If it's not a custom tile source, is this MBTiles?

[SudoPlz]

it is a custom tile source, and i am using the isTileLocal method giving it a YES. As I said it does work just fine in the simulator, but not on the real device..

[mousebird]

That's very odd. Well, I'm integrating it into a client project in the next few days so that should shake out a few bugs.

[mousebird]

It occurs to me we may be talking about different things. This particular issue is just related to reload. If you're generating the tiles yourself and you need the layer to reload them, it was displaying them from the lowest level on down as it loaded. Users find this annoying because it appears to flash.

The new mode will hold all the local tiles until the reload is finished. This should accomplish a seamless reload.

However, that doesn't cover the non-reload cases:

• When the layer first starts up you'll see it loading from the lowest res on down. Otherwise it would just be blank.
• When you zoom in it loads the lower resolutions first, then the higher.

[SudoPlz]

About the:

1) Problem still not resolved:

I tried your solution and I must say, it works very nice on the simulator, BUT when running on an Ipad 4 I have the same issue as before the update..! Tiles are still loading from lvl 1 till the current level.

this bug is still there on a real device, allthough it is fixed when running on the simulator (after your fixed it that is)

About the:

2) A few more cases in which the problem occurs:

The exact SAME problem as described on this issue (tiles loading from lvl 1 till the current level) can occur in each of the cases below:

• Initializing the globe with a height closer to the earths surface (lets say for example 0.00009)
• Upon zooming in (pinch)

and

When the layer first starts up you'll see it loading from the lowest res on down. Otherwise it would just be blank. When you zoom in it loads the lower resolutions first, then the higher.

Correct... Is this nessecary? I mean, it is a bit slow on doing that, cant we just load the highest res tiles and get on with it?

[mousebird]

There are a couple of reasons I start at the lowest res:

• I don't know for sure that the image pyramid isn't sparse. Loading tiles in this order is how we feel out the boundaries of the pyramid.
• On the sphere, we actually need most of those lower resolution tiles. They're being used for areas where we don't load high res tiles and we use them quickly when we zoom out.

If I were to skip the low res tiles and load the high res first I could get stuck. The low res might not be there when I needed them, thus sticking us with high res tiles in a select area... which we're no longer looking at.

Or I could fill in fake tiles. If you zoom out really fast in Apple Maps you'll see them do just that. I think this looks terrible.

That said, I may end up doing this for the 2D map anyway since people seem to expect it. So if a client pays me to do that, you'll have the same functionality in the 3D. But it'll have the same drawbacks as Apple Maps.

[SudoPlz]

Steve, I'm considering my options here, would all of these problems be solved if we used vector tiles?

[mousebird]

Depends on the application. Wait, are you also with NaviDeck or are you doing some other app?

[SudoPlz]

Yes I'm also with NaviDeck. Perhaps you've spoken with fdim about our project..

[mousebird]

Ah, got it.

Okay, for what you guys are doing, I believe it's basically vector overlays that you've rendered into images. Yeah, the image layer isn't well designed for that. It would take a bit of work to make that really nice.

However, the vector overlays are designed just for this sort of thing.

This is complex enough that it would be better just to discuss it in real time. Hit me up at sjg@mousebirdconsulting.com and we could do a Skype call or something. I'm around this afternoon or all next week.

[SudoPlz]

Sure lets get in touch, I'll add you.

[mousebird]

I think the latest iteration might do it for you. Set the waitLoad and waitLoadTimeout properties on the MaplyQuadImageTilesLayer. Develop branch, let me know it works for you.