StrandedKitty / streets-gl

🗺 OpenStreetMap 3D renderer powered by WebGL2
http://streets.gl
MIT License
604 stars 45 forks source link

Requests to /api/0.6/relation/NNN/relations.json #53

Closed tomhughes closed 1 year ago

tomhughes commented 1 year ago

We are seeing large numbers (over 3 million so far today) of requests to /api/0.6/relation/NNN/relations.json from streets.gl users, which appear to be in violation of the usage policy which states that the API is provided in order to edit the map data and should not be used for read-only purposes or projects.

As far as I can tell the site only makes these requests when users turn off caching but it seems a substantial number of users must be doing that.

One thing I'm not clear on is how the cache in question (which I believe is the tiles.streets.gl request which redirect to a digital ocean endpoint) is generated - is that pre-populated from a planet dump? or does that fill on demand by making it's own API requests?

StrandedKitty commented 1 year ago

Hello, I'm sorry for misusing this API endpoint.

The reason why I've decided to use it in conjunction with Overpass API is because it greatly speeds up tile fetching process. In particular, /api/0.6/relation/ API endpoint allows to instantly get all the parent relations of a relation. In Overpass API the same can be done using rel(br), but unfortunately it adds several seconds to each request.

As far as I remember, fetching parent relations is required to corrently render Simple 3D Buildings which consist of multipolygons (e.g. outline and part members are multipolygons). I think this was my motivation behind implementing this.

I've disabled the functionality that sends requests to /api/0.6/relation and I will try figuring out how to optimize Overpass requests. As doing rel(br) is only required for building=* and building:part=* multipolygons it is probably possible to execute it only for these kinds of ways. (Edit: doing rel(br) even for a single relation is incredibly slow, it takes seconds. So probably this is not the way to go.)


Some information on how tile caching works in streets.gl:

Only some tiles are cached, around 40k of them. They were fetched by me at some point so that users have faster access to several popular areas/cities. I host them on Digital Ocean and clients fetch them if they are available for a particular area. In other case (if a tile at certain position is not available in this storage) the client fetches it using Overpass API.

In settings there's an option to stop using cached tiles. The client will always use Overpass in case this option is active.

westnordost commented 1 year ago

Multipolygon buildings are just one relation, not a relation that contains relations. Which building would consist of relations of relations?

StrandedKitty commented 1 year ago

@westnordost Usually it is, but it also must support relations. How would you map a part with hole in it without using a multipolygon? INB4, you can't split it in two, because what if you want a nice roof that uses straight skeleton (like hipped roof)? Or how would you mark an outline of a building with a hole?

The wiki page for building relation (https://wiki.openstreetmap.org/wiki/Relation:building) specifies that outline can by a relation. They don't specify it for parts though, which is strange and probably wasn't thought through. But it is just common sense to support parts that are multipolygons.

westnordost commented 1 year ago

Oof, I didn't know that existed.

It would be perfectly possible to map a building with (multipolygon) parts without that kind of relation - just check which building:part are in which building outline.

.... actually, knowing one has to do that anyway for 3d building without such relation - is such relation even necessary? As in, can you not just ignore these relations and achieve the same result?

StrandedKitty commented 1 year ago

.... actually, knowing one has to do that anyway for 3d building without such relation - is such relation even necessary? As in, can you not just ignore these relations and achieve the same result?

It was discussed in https://github.com/StrandedKitty/streets-gl/discussions/59 It's not necessary, relations are just less chaotic.