RasterSource/RasterLayer not caching

[k-adams-smt]

Hello, we are using a 3rd party provider for a mapping layer and adding it as a RasterSource and RasterLayer. There's no issue with the generation, display, or performance of the layer. However, the layer is not being cached locally once downloaded from the 3rd party API like it did with the previous version of Mapbox (mapbox-gl-native-ios) that we used. I've found no information in the documentation regarding caching of Raster layers nor have I found anyone else with the same issue. Here's the only relevant info from the logs:

[Mapbox] [Warning, maps-core]: {com.mapbox.common.TileStoreFile}[General]: Failed to get datasets from URL template https://api.gic.org/wmts/GetTile/bluesky-high/default/EPSG_3857/{z}/{y}/{x}?format=image/png&AuthToken=XXXX

[fcosentino-nav]

Same issue here

[ZiZasaurus]

@k-adams-smt thank you for reaching out. Are you caching these tiles for offline or something else?

[k-adams-smt]

Hi @ZiZasaurus, thanks for following up! We'd love to cache them for offline but at the very least we need to cache them locally in order to prevent repeat network calls being made to our tile provider as we pay for each tile we consume. The previous Mapbox versions seemed to handle both of these automatically and worked perfectly but that doesn't seem to be the case anymore with regards to the 2nd point at least.

It does seem to be caching the tiles somewhere though as I can go fully offline (no network, no wifi) and the tiles are still rendering and showing up wherever I've previously been on the map. However, it's making repeat network calls to load the same tiles that are already cached on the map, which I've verified by using the .onEvery .resourceRequest method and parsed the data. The eventData dictionary is still loading these tiles with the data-source="network" property when they should be loading from the cache (assuming they are actually being cached properly).

Please let me know if there's any more information that I can provide. Thank you!

[ZiZasaurus]

@k-adams-smt we don't cache raster layers. If you would like to display raster tiles in your offline map, they would need to be uploaded to Mapbox studio and embedded within a style. From there you would need to use the offline tilestore, to manage retrieving and organizing your tile packs.

[k-adams-smt]

@ZiZasaurus Didn't the previous version of Mapbox cache them though? Would it be possible to add that functionality back to this library? Is there any other way for us to cache them locally since I don't think we can add them to Mapbox studio as the tiles come from a 3rd party provider and we can only access them via API requests.

[ZiZasaurus]

After some additional investigation, it appears I misspoke. We do cache raster tiles in v10. From the information you provided above, it sounds like the cache headers are not set correctly, which means that the expiration time isn’t calculated correctly and we may be checking more often than expected / needed.

If you are able to provide an endpoint or sample tile URLs, I'd be happy to investigate further.

[k-adams-smt]

@ZiZasaurus thanks for looking into it some more! That makes sense then as to why I was still seeing them show up when entering our app with no internet connection. Here's a URL of a request that we make:

https://api.gic.org/wmts/GetTile/bluesky-high/default/EPSG_3857/12/1517/1074?format=image/png&AuthToken=XXX

And here's the event data that we capture using the mapboxMap.onEvery(.resourceRequest) method:

▿ 6 elements ▿ 0 : 2 elements

• key : begin
• value : 1551593500495 ▿ 1 : 2 elements
• key : data-source
• value : network ▿ 2 : 2 elements
• key : response ▿ value : 5 elements ▿ 0 : 2 elements
  • key : size
  • value : 153796 ▿ 1 : 2 elements
  • key : no-content
  • value : 0 ▿ 2 : 2 elements
  • key : source
  • value : network ▿ 3 : 2 elements
  • key : must-revalidate
  • value : 0 ▿ 4 : 2 elements
  • key : not-modified
  • value : 0 ▿ 3 : 2 elements
• key : cancelled
• value : 0 ▿ 4 : 2 elements
• key : request ▿ value : 4 elements ▿ 0 : 2 elements
  • key : loading-method ▿ value : 1 element
    • 0 : network ▿ 1 : 2 elements
  • key : kind
  • value : tile ▿ 2 : 2 elements
  • key : url
  • value : https://api.gic.org/wmts/GetTile/bluesky-high/default/EPSG_3857/12/1517/1074?format=image/png&AuthToken=XXX ▿ 3 : 2 elements
  • key : priority
  • value : regular ▿ 5 : 2 elements
• key : end
• value : 1551594436260

Please let me know if you need anything else or if I can help in any way and thank you for all of your help!

[ZiZasaurus]

@k-adams-smt thank you for providing this! Are you able to share the auth token with me so we can test making the request?

[k-adams-smt]

@ZiZasaurus sure but I don't want to send it on here as public at least. Am I able to PM or email it to you please?

[ZiZasaurus]

Gotcha, okay so it looks like the server doesn’t send any cache headers whatsoever. Our servers send a cache header (Cache-Control: max-age=XXX) that tells our clients how long the resource is considered fresh. gic.org’s tile doesn’t have this header (or other similar headers), so we’re refreshing tiles more often because we assume they can’t be cached, or are expired. The server is usually considered the source of caching information.

You're able to see tiles in offline mode because we do cache them, they’re just not considered fresh anymore, so they must be revalidated. In a full offline scenario, we can’t do revalidation, so we’re making a decision to show the expired tile anyway.

[k-adams-smt]

@ZiZasaurus okay, thank you so much for the information! If gic.org was to add a caching header (or some other header to indicate expiration), would Mapbox be able to cache it and not make as many network requests to retrieve the tiles? Or is there any way we can mark tiles as valid/not-expired as a manual workaround to reduce the number of requests as well? We pay for each tile our users consume so we'd like to keep network requests at a minimum.

[ZiZasaurus]

@k-adams-smt, yes, having gic.org add a caching header would be the most efficient way to have Mapbox cache your raster tiles properly.

[egorf]

@ZiZasaurus

Hi. A question on this topic. Is there any documentation or insight on how caching works in the latest Mapbox versions? In our application, we allow customers to add their own raster layers, and it would sometimes be useful to save them for offline use.

Ideally, we want to be able to save tiles for offline use ourselves, with the help of the SDK. It seems like it's not possible at the moment, so it would be really nice to get an understanding of the internal caching rules. How long before Mapbox considers the tiles expired? Is there a storage limit? Are there any triggers that would clear the caches?

[VahanLab]

Hey @ZiZasaurus, I'd be really interested by your answer to @egorf question please! I'm building a hiking app, and to provide offline tiles access to a collection of national topographic raster layers is a mandatory feature. I can't create a custom mapbox style along with the tilestore API, because I don't own the data, and only have access to some server urls. Any help would be very much appreciated 🙂