Loading time due to many small requests

[mmd-osm]

I noticed that ptmap issues quite a number of small requests and takes quite some time to fully load. I'm wondering a bit if you would be better off loading everything for your current bbox in one go. That's significantly faster and actually fast enough that I don't find much compelling reasons for client side caching (which is probably the reason for those many small requests).

[out:json][bbox:{{bbox}}];
(relation[type=route][route~"^(bus|trolleybus|tram|subway|ferry|train)$"]);
out meta 32 geom({{bbox}});

Example: http://overpass-turbo.eu/s/loY

For your current implementation, you might want to take a look at the following https://github.com/drolbr/Overpass-API/pull/358 - I believe this could be interesting for faster loading (check out the overpass turbo link in that pull request, and note the different server end point used there).

[plepe]

Oh, I didn't know that you can apply a bbox filter to the geom option of out. Interesting :-) I will research if this is useful. I might speed up loading. But it might take some time :-) Thanks for pointing that out!

I've done a lot of testing until I came to the current setup regarding queries. If I load everything for the current bbox in one go this would take several minutes (obviously this depends on the size and location of the bbox, let's say larger city area). That's why I load and render the data step by step. First I load the first bunch of relations. Then I prioritize the routes by their scale (bbox diagonal in pixel length divided by count of stops). Then I load the stops (inside the bbox) for the best routes. Then I load the ways (inside/intersecting the bbox) for the best routes. Then I load the next 32 relations, starting all over. The 32 relation limit might not be the best split factor though.

Btw, the server end point of the pull request is not available right now.

[mmd-osm]

That's a bit of a pity. We're replacing some faulty piece of hardware atm and hope to have everything up and running soon again.

Btw, the server end point of the pull request is not available right now

[plepe]

No problem, I will check back later :-)

What I forgot in describing the current loading algorithm: As I don't know the bbox'es of the stops and ways, I try to load all of them (by ID) and apply a bbox filter to select only the matching. For ways outside the bbox I request their bounds, so I don't need to retry. This for sure is one reason why the geom({{bbox}}) filter is way faster.

[plepe]

One more thing: When checking the output of http://overpass-turbo.eu/s/loY I noticed that there are some ways without a geometry object and some with a geometry object where all elements are null. I guess that the ways which have the bbox of the all-null-geometry-object intersect the requested bbox, but not the geometry itself?

By the way, getting the geometry of the ways is not enough data. I also need the node ids of the first and last node to calculate the way direction.

[mmd-osm]

I noticed that there are some ways without a geometry object and some with a geometry object where all elements are null

That's a really good question actually, one that needs to be answered by Roland. Without looking closer in the implementation, I tend to agree to your bbox assumption. I don't see much value in printing a way with all null values (except that you would get the number of nodes), so that may be something which could be addressed in an issue.

By the way, getting the geometry of the ways is not enough data. I also need the node ids of the first and last node to calculate the way direction.

I have to admit that I'm not really familiar with the details of your implementation and how you derive way direction from the first/last node. I wonder if you could just use the actual lat/lon values and forward/backward roles of the relation members to merge various parts of a relation.

[plepe]

In Public Transport scheme v2 the role of the ways of a route is empty -> no forward/backward to use. Also, the first/last node(s) of a way might be outside of the bbox and therefore null.

Even if the geometry of the first/last node of a way equals the first/last node of the previous/next way it might not be connected - it could be a different node anyway. It is not likely at all though.

[mmd-osm]

Stephan Bösch-Plepelits notifications@github.com schrieb am Di. 31. Jan. 2017 um 22:03:

No problem, I will check back later :-)

Fyi: the dev instance is up and running again.

Regarding the missing node information I probably need some more time to think about alternatives. Would additional node ids as part of the 'out geom' output for ways provide enough details for your use case? I agree that an analysis solely based on lat/lon values won't be able to tell if ways are in fact connected or not.