Sprites for self hosted pmtiles

[daniel-j-h]

To get a map up and running we need

• a .pmtiles file for vector tiles
• a style file for how to style the vector tiles (this repo's styles)
• font glyphs in pbf format, tracked in https://github.com/protomaps/basemaps/issues/41
• symbols in sprite format (what this issue is about)

I wanted to open the discussion on sprites since that seems to be the last missing piece in the puzzle.

I'm thinking that the goal should be making it easy for folks to get started with to have a full map distribution up and running: have a default set of styles, fonts, sprite files they can put on a static host next to the .pmtiles and be done with it. At the moment we're focused on the .pmtiles file (and rightfully so) but then folks have to manually look into fonts and sprites and all these supporting parts.

Over the last days I checked out how to create custom sprites for maplibre and captured my learnings here.

Based on those learnings I'm thinking

1. Should we default to an icon set like the original Maki https://github.com/mapbox/maki (CC0 license)
2. Do we then provide two sprites (high-dpi and regular) in sdf format users can style however they want or do we provide two sprites per style? (think: sprites for the "light" style, sprites for the "dark" style, and so on)
3. Do we want the styles in this repo be free of any sprite references so it's up to users to set it all up? Or do we want to (optionally?) include references to the sprite symbols in the styles here already?
4. What's the best place for hosting these sprites so folks can simply grab them and static host them next to their .pmtiles file? I guess there's a similar question for fonts and to some degree maybe even the style. Could be off-topic for this ticket, tho.

Style Spec on Sprites

• https://maplibre.org/maplibre-style-spec/sprite/

[bdon]

We are finishing up the design based on the Nextzen icons https://www.nextzen.org

I expect to get this merged in the next 3 weeks as I'm currently at a conference.

There is separate sprite sheets for 1x and 2x DPIs, and light and dark themes.

It will be hosted the same place as fonts (possibly a separate assets repo)

[bdon]

https://github.com/protomaps/basemaps-assets/tree/main - proposed directory structure. We need to get this right because paths must be immutable for styles to reliably link to URLs

[spatialillusions]

To get a map up and running we need

a .pmtiles file for vector tiles a style file for how to style the vector tiles (this repo's styles) font glyphs in pbf format, tracked in https://github.com/protomaps/basemaps/issues/41 symbols in sprite format (what this issue is about)

Maybe a crazy idea, and I know my use of pmtiles in offline HTML "apps" are quite a unusual use case, but what about if style/font/symbols were built into the pmtiles file?

Then you could use range requests for all sorts of data that is needed, and you could still use other styles/fonts/symbols if you want to. This would only make the pmtile file a few Mb bigger, but you would always be sure to have everything you need, and your list above would simply be:

a .pmtiles file

[bdon]

@spatialillusions it's a nice idea, and theoretically you can stuff whatever you like into the "metadata" json section of pmtiles as base64 data.

• In practice, the relationship of styles/fonts/symbols are generally many:one - you might re-use a tileset among multiple applications.
• the fonts are the most painful part right now: 256 files are necessary for each font variant. One idea is to create a pmtiles-like format where the first 256*4 bytes are offsets Into a file - this would reduce 256 down to 1 for every font variant. But an even better solution is migrating maplibre away from pre-rendered font glyphs in favor of locally rendered glyphs (like it already does for CJK), with many other benefits like less network traffic, standard web fonts and improved sharpness. I would prefer everyone devotes efforts to making the latter work.

Maybe there's room for a higher-level "map bundle" spec? @wipfli has proposed this as well

[spatialillusions]

Including styles and other resources in the file would make it more like Esri's VTPK format. I really like the thought of being able to hand a single file to someone and they can read it and get a default visualization without needing any other resources.

Maybe v4 could mention something about storing all resources in the metadata, where a styles property could be an array of styles, and then a glyphs property with a glyphs object for easy lookup and similar for symbols.

For reference, if someone wants to do a similar thing as I do, this is what I do in my current code to this to be able to read glyphs bundled into the javascript:

// imports the font object I got, this could be read from metadata if we store glyphs there
import fonts from "./fonts.js";

function glyphs(params, callback) {
  let pattern = /glyphs:\/\/(.*)\/(.*)/i;
  let url = params.url.match(pattern);
  // Make sure we got the font in our font object
  if (fonts.hasOwnProperty(url[1]) && fonts[url[1]].hasOwnProperty(url[2])) {
    new Promise((resolve) => {
      resolve(
        Uint8Array.from(window.atob(fonts[url[1]][url[2]]), (c) =>
          c.charCodeAt(0)
        )
      );
    }).then((font) => {
      callback(null, font, null, null);
    });
  } else {
    // We don't have the font, so warn about what font we are missing
    console.warn(params);
    console.log("https://cdn.protomaps.com/fonts/pbf/" + params.url + ".pbf");
    callback(null, new Uint8Array(), null, null);
  }
}

Then when I set up the map:

maplibregl.addProtocol("glyphs", glyphs);

And in my style:

glyphs: "glyphs://{fontstack}/{range}",

[bdon]

For posterity's sake:

Should we default to an icon set like the original Maki https://github.com/mapbox/maki (CC0 license)

We are using https://github.com/tangrams/icons which is an MIT licensed repo and has copyright assigned to the Linux Foundation as part of the Mapzen project

Do we then provide two sprites (high-dpi and regular) in sdf format users can style however they want or do we provide two sprites per style? (think: sprites for the "light" style, sprites for the "dark" style, and so on)

in https://github.com/protomaps/basemaps/tree/main/sprites there is a Rust CLI program that uses the fabulous https://github.com/flother/spreet to generate 1x and 2x sprites for all 5 core themes.

Do we want the styles in this repo be free of any sprite references so it's up to users to set it all up? Or do we want to (optionally?) include references to the sprite symbols in the styles here already?

That was an original goal but it turns out sprites are necessary for basic use cases without POIs - for things that people perceive as "essential" to a basemap, like townspots that properly collide and highway shields, we need a sprite sheet.

What's the best place for hosting these sprites so folks can simply grab them and static host them next to their .pmtiles file? I guess there's a similar question for fonts and to some degree maybe even the style. Could be off-topic for this ticket, tho.

It's the same repo / GitHub pages deploy as fontstacks: https://github.com/protomaps/basemaps-assets

[bdon]

(Spritesheets added for all 5 core styles in styles v3.0.0)