Improve Open Graph previews

[lectrician1]

Users on the OSM World Discord Server frequently post openstreetmap.org links to various areas they're talking about, changesets, etc.

Discord has embeds that use the Open Graph protocol to display rich data.

Currently the only embed data that's displayed looks like this: A picture of the map for the linked area as <meta property="og:image"> included would be better.

or A picture of the map and maybe a bit of changeset data would be better.

Now that I think about it, the embedded image request is probably not feasible. It would require the server generating a image and its link for every view of the map that is requested. However, all of the OSM layers use raster tiles so maybe this could be possible?

At least a better embed description is possible. As to what should be in each description should probably be discussed below.

[tomhughes]

We added OG support years ago. We don't provide a map image because that requires a static map service and we only have a tiled map service.

[gravitystorm]

maybe a bit of changeset data would be better.

I agree - we can be a little more creative with the text that's shown for each page, rather than having the same "OpenStreetMap is a map of the world..." for every link.

As for maps - we could get creative. For example, we could include a tile that has the node on it, even if the node isn't centered. Or we can make our own internal static maps, by grabbing adjacent tiles and stitching them together and cropping the region of interest. That's how most scalable static map services work anyway.

[tomhughes]

Well yes we know we can do tile stitching. We also know that it has very difficult to resolve performance and scalability issues which is why we don't offer it.

Obviously we can offer better descriptions for specific pages but that really needs specific bugs saying "this type of page would be better described as X" rather than this general bug which has no concrete end point when we can declare it fixed.

The actual front page is hard because we don't know if the specific location where it was positioned is important - if the URL is meant to be for a "map of this place" or just as a general reference to OpenStreetMap given that copy and pasting a URL will almost always include a location even if the location wasn't important. It's also dangerously close the "end user mapping site" type experience that we explicitly consider out of scope.

[pnorman]

We'd pretty much have to do tile stitching because we need to support a number of different map layers, not all under OSMF control.

[gravitystorm]

we need to support a number of different map layers

I don't think we need to support multiple map layers for the OG previews - I would only expect to show the default map layer in these previews.

[gravitystorm]

We also know that it has very difficult to resolve performance and scalability issues which is why we don't offer it.

Can you describe more of the performance and scalability issues here? Elsewhere, I find the stitching-and-cropping-tiles approach works reasonably well.

For the OG preview images, I would guess that it's only a (comparatively) small amount of traffic. Each preview would involve ~4 tile requests, some CPU to stitch+crop, and the output image could be cached too.

[tomhughes]

So there's the first problem, that to make this work you don't just need to write a stitcher you need to provide a caching infrastructure because who knows how many thousands or millions of views something might get once it is shared on social media networks. So that immediately raises this from a programming challenge to a significant operational challenge.

Plus once you provide this it's not just going to be serving a few social media shares - we know that these services are very popular and will be used at large scale - people will use it to embed thousands of location maps on corporate web sites and the like.

The fundamental problem is that unlike tiles every single usage of this is unique, so any caching is limited to that specific use because even if a second person tries to share roughly the same map they're not going to choose exactly the same area.

It's not like we haven't seen what happens to these services - there are, or have been, various such things running on the dev server and even as non-heavily promoted dev things they become popular and problematic.

[gravitystorm]

we know that these services are very popular and will be used at large scale

I wasn't asking for a wide-open public static maps service. It could either be entirely internal to this codebase, or run separately and password protected so that only the production servers have access. The images produced would be fixed size thumbnails, and are unlikely to be in widespread demand.

So the scaling needs is only at a similar scale to what is already in place for serving the same number of requests for the logo. A few gigs of RAM will be enough to cache the most popular 99% of the generated images, and the billions of unique ones required for every other entity page can be generated on the fly for probably a fraction of the CPU involved in putting together the rest of the page.

[tomhughes]

Well the current "map export" endpoint was never intended as a general purpose static maps service either but it didn't stop people trying to use it as one - that's worse of course because each call is doing a custom render with mapnik.

But we can't really evaluate it until there is an actual concrete implementation proposal to assess.

[woodpeck]

https://github.com/geofabrik/GfStaticmap is a simple staticmap script with basic draw-on-map capabilities. It can do things like "make a X by Y pixel map and draw this polygon onto it and choose the zoom level such that the polygon is visible". It caches the stitched maps though I'm not sure if it caches them with stuff drawn on. It's Open Source and somewhat maintained. It is written in PHP though.

[bam80]

Could we design actionable shot-term plan to move forward?

[lectrician1]

@bam80 Developing a plan is not worthwhile unless we have someone who will commit to implementing it. Until someone from the community commits to developing the feature, this isn't going to move forward.

[bam80]

Plus once you provide this it's not just going to be serving a few social media shares - we know that these services are very popular and will be used at large scale - people will use it to embed thousands of location maps on corporate web sites and the like.

I think we could redirect the actual map image url to existing stale image if the amount of requests exceeds some threshold.

[bam80]

Until someone from the community commits to developing the feature, this isn't going to move forward.

Was it suggested for GSoC or something?

[lectrician1]

Was it suggested for GSoC or something?

@bam80 Maybe, but GSoC is over for this year and no commitment was made.

[tomhughes]

It's not really suitable - the actual code is trivial and could be written in a few hours.

The hard parts are all around how you actually run that as a production service - do we try and restrict it so it can only be used for this (but how - it's hard to see how to do that) or do we just accept that it will be more generally used and then what are the operational issues around scaling it and what hardware do we need to run it etc.

[1ec5]

As for maps - we could get creative. For example, we could include a tile that has the node on it, even if the node isn't centered. Or we can make our own internal static maps, by grabbing adjacent tiles and stitching them together and cropping the region of interest. That's how most scalable static map services work anyway.

The fundamental problem is that unlike tiles every single usage of this is unique, so any caching is limited to that specific use because even if a second person tries to share roughly the same map they're not going to choose exactly the same area.

One way to avoid stitching – and the need for generating a new image – is to only return one existing tile image. If the coordinate identified by the URL is close enough to the center of the tile, return that tile verbatim, not even centered or cropped. Otherwise, if the coordinate is too close to the tile boundary, return the containing tile one zoom level out. The downside is that different features are visible at a lower zoom level, but this is more about giving a rough idea of the page’s contents anyways.

This still leaves another technical hurdle that I haven’t seen mentioned here yet. Most people are going to share a URL that expresses the x/y/z coordinates in the URL fragment (hash), not a real URL query string. This hash is normally interpreted by the site’s JavaScript. Open Graph clients don’t execute JavaScript, so it’s up to the server to come up with an image URL, but the server will never see the URL fragment. There is an alternative syntax that predates the current website design, which uses the zoom, lat, and lon query parameters. However, nothing seems to use that syntax anymore, not even the share panel when you add a marker.

2007 seems more feasible to me, because a URL to an element or changeset will contain the ID as part of the URL’s path, which the server does have access to.

[bam80]

It's not really suitable - the actual code is trivial and could be written in a few hours.

So maybe we should start from writing that code first then?

[amandasaurus]

If you want to work on this, I have an initial improvement to OSM.org's OpenGraph tags here: https://github.com/openstreetmap/openstreetmap-website/pull/3942

[AntonKhorev]

I'm experimenting with generating images for osm elements.

Plus once you provide this it's not just going to be serving a few social media shares - we know that these services are very popular and will be used at large scale - people will use it to embed thousands of location maps on corporate web sites and the like.

If that turns out to be an issue, we can restrict images to osm forum and other places used mostly by osm contributors.