Bike lane layer drops in and out

[frankie2784]

Not sure why, but the bike lane layer can drop in and out randomly. Often this happens when driving on freeways. The phone's reception is fine, and it looks as though the data is still cached somewhere. But the existing data will still disappear for stretches of road before returning shortly afterwards (usually between 15-60 seconds). While the data is missing, tapping on the way no longer works. Video attached of this issue. Latest SC version on Pixel 6.

https://user-images.githubusercontent.com/19356194/233766509-174dc569-d17d-4432-a1e2-eea191517814.mp4

[Helium314]

Looks like it's not the layer (aka overlay), because you can still see the red lines on the right. Also notice that the change happens just before a download starts. Though I'm not sure this is connected.

Maybe the way being quite long has to do with this? https://www.openstreetmap.org/way/746812219

[westnordost]

I think this may have something to do with that there is no node on that stretch of way for a very long distance. If you would download that area and only that area in JOSM, that road would also not show up. The OSM API only returns all nodes in the given bounding box plus any way and any relation that uses these nodes. Thus, if no node of a way that goes through the bounding box is within the bounding box, the way is also not returned. I agree that SC behaves like the OSM API here (also when working with local data) is detrimental in these situations but overall this behavior was chosen I think for performance reasons and a more consistent behavior when either using the app offline or online.

I wonder if it would be worth the effort to be able to query at least locally for display in the UI (e.g. overlays, for UI interactions like tapping on a road in the address quest/overlay, for adding vertices) the OSM data by "real" bounding box. This would require the geometries table to have additional columns for the bbox and an index over that (again), plus the whole caching thing would need to be adapted to have the same in memory, too.... probably not worth it, considering that the in-memory cache would also have to get that functionality.

[frankie2784]

Strange that it seems to reappear before doing a second pull from the API

On Sat, 22 Apr 2023, 23:54 Tobias Zwick, @.***> wrote:

I think this may have something to do with that there is no node on that stretch of way for a very long distance. If you would download that area and only that area in JOSM, that road would also not show up. The OSM API only returns all nodes in the given bounding box plus any way and any relation that uses these nodes. Thus, if no node of a way that goes through the bounding box is within the bounding box, the way is also not returned. I agree that SC behaves like the OSM API here (also when working with local data) is detrimental in these situations.

I wonder if it would be worth the effort to be able to query at least locally for display in the UI (e.g. overlays, for UI interactions like tapping on a road in the address quest/overlay, for adding vertices) the OSM data by "real" bounding box. This would require the geometries table to have additional columns for the bbox and an index over that (again), plus the whole caching thing would need to be adapted to have the same in memory, too.... probably not worth it, considering that the in-memory cache would also have to get that functionality.

— Reply to this email directly, view it on GitHub https://github.com/streetcomplete/StreetComplete/issues/4964#issuecomment-1518665729, or unsubscribe https://github.com/notifications/unsubscribe-auth/AETVUIVR74PECTMIBJUTUT3XCPPKTANCNFSM6AAAAAAXHUH34I . You are receiving this because you authored the thread.Message ID: @.***>

[westnordost]

No, that's not strange, see

... SC behaves like the OSM API here ...

[frankie2784]

No, that's not strange, see

... SC behaves like the OSM API here ...

I'm confused. JOSM may not download a way if it doesn't have a node within the bounding box, but once it finds a node, it will download the entire way and that way and all its data remains available for the rest of the session. It doesn't drop in and out.

Similarly, the data for all previously downloaded ways in SC remain available in each session, even if they're not within my current bounding box - I can swipe back from my current location and find a way that I passed a hundred kilometres ago, and the data is still there. At some point, SC was able to ascertain that this particular way existed, so to me logically it should also remember it (even if there is temporarily no node within my current bounding box) until I restart the app/clear the cache?

[matkoniecz]

that is only guess why things may be going badly, this may happen as this part of data may be not present in data used for rendering

[westnordost]

Well, you don't need to know the details, just trust me that I know what I am talking about. The downloaded OSM data is persisted in a local database that mimics the behavior of the OSM API when queried. When you scroll the map, (not entirely correct but simplified:) only the portion that is visible on the screen plus a certain buffer is loaded from the local database into memory and displayed. (There is also an in-memory cache in-between the UI display and the local database but that cache also mimics the behavior of the database)

[frankie2784]

Ah I see, a fix would likely require something like querying on linestrings, instead of just points. That makes sense. I can think of some potential hacks, but they would increase the complexity of each redraw.

On Sun, 23 Apr 2023, 06:29 Tobias Zwick, @.***> wrote:

Well, you don't need to know the details, just trust me that I know what I am talking about. The downloaded OSM data is persisted in a local database that mimics the behavior of the OSM API when queried. When you scroll the map, (not entirely correct but simplified:) only the portion that is visible on the screen plus a certain buffer is loaded from the local database into memory and displayed.

— Reply to this email directly, view it on GitHub https://github.com/streetcomplete/StreetComplete/issues/4964#issuecomment-1518772718, or unsubscribe https://github.com/notifications/unsubscribe-auth/AETVUIRAZOMV5H2FZKQZVKTXCQ5SBANCNFSM6AAAAAAXHUH34I . You are receiving this because you authored the thread.Message ID: @.***>

[westnordost]

Yeah... so I will close this as will not fix

[frankie2784]

I'll do some more testing to see if the issue lines up with the bounding box hypothesis and report back, I have a suspicion it's not the whole story

[frankie2784]

Tested this out a little more today.

The bounding box issue, or something similar, definitely seems to be a thing. See, e.g.

https://user-images.githubusercontent.com/19356194/234122458-5970fc3e-02a2-4a8f-9302-a8192d88dbae.mp4

However there is something else weird going on with the data being a bit glitchy...

https://user-images.githubusercontent.com/19356194/233969335-6f30b9c3-7872-4226-983e-427008893ab3.mp4

[mnalis]

@frankie2784 The first example is as explained by westnordost, and second example is likely too - note that while data is being downloaded/processed (that circling animation around star in the top left corner), it will likely be inconsistent (i.e. "in some unknown state between previous and new one") until that circling animation stops.

And if you were to cut out parts of the video with "circle animation", on quick look it would seem that this second example would be the same as first example?

[frankie2784]

I think the difference between the first and second example is that the first consistently disappears at a specific part of the way (in the middle of a straight section with no nodes). In the second example, the data flickers even though nodes on the way are completely within view (on a bend and an intersection). So it seems to me there's something else going on that is distinct from the bounding box issue.

On the same stretch of straight road as the first video, you also see odd glitches as you scroll back and forth, which I don't think can be explained by the issue of nodes coming into and out of frame.

https://user-images.githubusercontent.com/19356194/234063881-f0dfefc0-5de6-4be7-9898-75af03f93261.mp4

It's not exactly a deal breaker in terms of UX, but just curious as to what could be causing this.

[Helium314]

The short flickering could be caused by tangram (the map library). Sometimes you can even see pins of already solved quests flickering like this.

[frankie2784]

However there is something else weird going on with the data being a bit glitchy...

Actually, this might be partly because I was moving at the time (you can see the blue location dot off screen at the top). SC would have refreshed the data for my location - not what was being displayed on the screen at the time - which might have temporarily confused the UI as to which data to display.

[westnordost]

Well, that is quite a "heavy" operation.

Imagine you would continuously download new bounding boxes in JOSM every 20 seconds or so. But on a smartphone.

For performance reasons, StreetComplete does not show all data it downloaded (i.e. is in local storage) at once, but only the data that is within view. This means that when scrolling, the app oftentimes needs to fetch the data from the new bounding box from the local database and then push that data for display to tangram-es. While tangram-es processes this data, there can be a short while in which no data is displayed. How long tangram-es needs to process the data for display depends on the size of the data.

The size of the data in the current implementation depends on how long you scrolled until you last solved a quest or enabled/disabled a quest (IIRC). Because when you scroll, the data in the newly visible area is added to the displayed data so that when you scroll back, as it happens frequently on a on-foot survey, the data doesn't need to be refreshed again. Now, I think there is no cap on it, so the data to display just adds up and up - as long as you do not make any changes (edit in overlay, solve quest, maybe even just switch overlays). I am not 100% sure if this still works this way, @Helium314 changed a lot (for the better) half a year ago in that code.

[Helium314]

I am not 100% sure if this still works this way

Not any more. The updating of so much data could sometimes take several seconds if the app had been used for a longer time. Now with the cache, fetching recently viewed data is quite fast (if it's still in the cache), so only the data in view is forwarded to tangram. There are more updates than previously (sometimes causing a little flickering), but those updates are much faster.