Explore migration from tangram to MapLibre

[smichel17]

Originally posted by @westnordost in https://github.com/streetcomplete/StreetComplete/issues/1892#issuecomment-889991925

MapLibre (ex Mapbox SDK) now supports Metal, so an iOS port would probably (need to) use MapLibre instead of tangram-es. This requres to

• create a mapstyle for mapbox / maplibre (for jawg.io map tiles) that matches the map style for tangram
• check it out, see if it has all features required by StreetComplete (probably yes)

As tangram is not really developed further, it might be a good choice to change to using the maplibre SDK for Android too. Or - bluesky - use/create a map view that directly renders the OSM data.

Additionally, there's issues like https://github.com/streetcomplete/StreetComplete/issues/3101 which are complicated to fix due to working around tangram limitations.

From the parts I am familiar with, I think StreetComplete already has a decent wrapper around Tangram (e.g. KtMapController), so in many places it would "just" be a matter of swapping out the implementation of those interfaces, and then plugging any leaks in the abstraction. There's also the scene files; I don't yet know how they map to MapLibre features. And any other tangram features we use that I'm not familiar with.

[westnordost]

• the tangram wrapper is very tangram specific. There is no sense in keeping if if moving to MapLibre
• scene files are incompatible. A custom style would have to be written anew.

[smichel17]

My first impression was that MapLibre isn't quite ready for use on Android— I wasn't able to find any docs besides this README, which still contains many links back to the mapbox docs. Based on the commit history it also seems like the Android version is not under particularly more active development than tangram (equivalent history). (That could be misleading, though, if most of the work is happening in code that's shared between platforms.)

[smichel17]

Reading the MapBox camera guide, it seems similar enough to tangram's api, and perhaps the center the camera within a map area feature could be used to fix #3101. So if these are still accurate, perhaps it will not be as much effort as I thought.

[westnordost]

I consider MapLibre / MapBox SDK somewhat more mature than tangram-es though. (Without having had a closer look though.): It is older, i.e. had some years headstart, and now for 3 years, tangram-es wasn't really much developed on.

[smichel17]

I'm reading through the MapBox docs now, and I agree it seems that way. Particularly interesting to me: it looks like dynamic display of quests may be possible without re-rendering the entire map. The main problem is that I'm not sure what features have changed since the MapLibre split; I didn't find any MapLibre documentation, and I don't see any way to reference the MapBox guides at the version before the split (the javadocs are versioned, so that's fine).

edit: Here's a good overview: https://docs.mapbox.com/help/getting-started/add-markers/. Unfortunately,

• Mapbox Maps SDKs v10 includes wrappers supporting this approach.
• Mapbox Maps SDKs less than v10 offer plugins supporting a limited version of this approach.

The MapLibre fork happened just before version 9.3, so we might be limited to the latter.

Even still, combined with data driven styling, I think we could easily enough do things like make the quest dots the same color as the background of the pin that will appear when they're expanded.

[smichel17]

They have a few different render & camera modes, which I think could replace our custom "compass mode" https://docs.mapbox.com/android/maps/guides/location-component/#rendermode (although with this approach, we give up some control / ability to tweak later)

[westnordost]

So the following steps need to be taken here:

Phase 1

1. create a requirements catalogue, i.e. what features must the library provide, what would be good but is not required, what is extra
2. create a minimal Android app and implement/demo all the features mentioned above
3. finally, evaluate. Are all necessary features covered? Estimate how long it would take to migrate and evaluate what we gain from that. (Do we need less code because there are less workarounds? Do we need to add additional workarounds? etc.)

After the evaluation, if we decide it is worth it, there may be a phase 2 - to actually do the migration. I can help with the requirements catalogue, i.e. when it is almost complete, I'll review it and also add anything you may not have thought of (because features that require this don't exist yet).

[matkoniecz]

create a requirements catalogue, i.e. what features must the library provide, what would be good but is not required, what is extra

generic mandatory features:

• decent performance (hundreds of SVG icons should not harm performance)
• decent battery performance (at least not even worse than Tangram)
• actively maintained
• usable as far as documentation and bugginess goes
• on a compatible license

specific mandatory features:

• show map from a vector source
  • compatible with data schema used by Jawg
  • compatible with data schemas used by alternative providers (see branches of map style)
• custom styling of a vector map
• display of interface above map (zoom/locate/undo/hamburger icons, star count)
• show many SVG based icons at specified positions (quest icons)
  • raise interface on clicking them
• display of a custom node geometry (quest object, hidden dots, current location)
• display of a custom way geometry (quest object, GPS tracks)
• display of a custom area geometry (quest object)
• display of a custom labels and icons (display of nearby objects for some quests)
• display of rotated image at specific point (to display direction of view, pointer to the current location when offscreen)
• fetching map data into cache
• smooth zooming (on quest selection, map panning, following GPS location)
• 3D view (is it a critical feature or nice to have?)

nice to have:

• preferably not controlled by one company that may drop support at any time
• smarter/faster update of displayed quests ( https://github.com/streetcomplete/StreetComplete/issues/3269 )
• easy disabling/enabling of map rotation ( https://github.com/streetcomplete/StreetComplete/issues/3022 )
• smarter handling of rotating map while note is being placed ( https://github.com/streetcomplete/StreetComplete/issues/3101 )
• supporting Android version range includes all versions supported by SC (I guess that dropping support for some rare versions would be survivable but avoiding this is highly preferable)

What is missing in this listing?

pinging @westnordost as it is relatively important

pinging @smichel17 as help with finding missed issues is likely here

[westnordost]

Hmm, the following more, otherwise I think it is complete

further mandatory:

• possibility to display a line left and a line right of the "real" line, each with a different color. (For sidewalk/cycleway layer). Very likely possible, see e.g. zlant parking lanes viewer
• fetching map data into cache: This means, it should be possible to fill the cache for MapLibre using a process that has nothing to do with MapLibre (or MapLibre has specifically the functionality to pre-download an area). That's what we do currently - download an area into the cache that tangram uses.
• zooming into and/or calculating which camera position is necessary to fully display a given geometry (with offsets, because the bottom sheet obscures a part of the map)
• regarding quest pins: 1. it should be performant to display thousands of them simultaneously. 2. it should be performant to add/remove single quest pins.
• iOS support. But that is a given.

nice to have:

• simultaneous camera animations. E.g. the view is animating the rotation with the movement direction but then also an animation for zoom-in (after user presses button) starts

[westnordost]

@matkoniecz did not start working on it yet, so for the time being, this is free for the taking / can be done by other people interested in this. If you do, I can assign you to "reserve" this.

[adrianclay]

I've made a start exploring this.

I've hacked together a version of SC with a side-by-side of Tangram / MapLibre, as I believe it will be (for me) an easier way of demoing desired requirements.


For now I'll be focusing on exploring the list of requirements above.

[westnordost]

Whoa, cool!

Note that for some time, I have been playing with the thought whether it would not be less effort / more flexible / faster to skip vector map libraries and directly render the OSM data (etc.) onto the map like Vespucci or iD does. I.e. facilitate some (hardware accelerated) canvas-drawing of sorts.

Advantages:

• less download necessary. No map cache necessary. I.e. no vector map service provider needs to be used at all or at least only for low zoom levels
• absolute flexibility
• no need to bend the code to the API of the map rendering library (which has been a strain sometimes for tangram-es)

Not sure:

• Effort. Drawing some lines will be easy, and all the additional stuff the app is drawing and generally doing with the map has been a strain to implement with tangram-es. It could turn out to be equally as difficult (or not possible) with MapLibre. On the other hand, for doing the drawing all by oneself, the devil is probably in the details. E.g. rendering road names on such lines would probably be something that falls onto ones feet.

Disadvantages:

• Coastlines will be a problem. Maybe to circumvent this, always show the satellite view and ditch the light/dark mode
• tilting the map and possibly rotating the map will probably be much harder to implement (when thinking about simply drawing all the stuff on a simple 2D canvas)

---

Anyway, these were my thoughts so far for the unicorny outlook of maybe rendering the map by oneself, using the Canvas or somesuch. I felt I needed to record this here before looking into MapLibre starts in earnest, just to mention another alternative.

That considered, I am of course very happy that someone looks into this, as tangram-es issues unfortunately have been piling up and piling up, to the point where I do not even bother to open bug reports in their repo anymore as it looks like it has finally been abandoned. Moving to the widely used MapLibre looks very much like the step in the right direction here.

[smichel17]

tilting the map and possibly rotating the map will probably be much harder to implement (when thinking about simply drawing all the stuff on a simple 2D canvas)

I wonder if we could borrow significant amounts of code from MapLibre/Tangram for this.

[westnordost]

If you have any updates or news, I'd be glad to hear about it! This is one remaining abandonware thorn in the sides of StreetComplete it would be very nice to get rid of (or explore how it can be done at least). If you have the code somewhere public, it can also help for future attempts even if you do not continue to look into it.

[adrianclay]

Hey - I'm afraid to say I've been unable to prioritise this issue and give it the effort it needs (which is a lot).

I've pushed up the changes I'd made here to 8f2d59678f5e14409284f00c3783a36239de53e4

I've documented some of my thinking within the commit message. If anybody is interested in taking this over, I'd be happy to spend some time talking through the code and answering questions over a video call.

My super summarised view is that I can't see any major reason not to migrate, other than the effort required in doing so 😄. Any time spent on this is time not spent on other changes, conversely the sooner this migration happens the less painful it will be.

Bonne chance.

[Helium314]

I played a bit with the mapLibre work of @adrianclay, and added some more stuff: https://github.com/Helium314/SCEE/tree/maplibre After updating mapLibre to 10.0.2 it was necessary to update OkHttp to 4.10, leading to some more small changes.

As for the feature list

generic mandatory features:

• [x] decent performance (hundreds of SVG icons should not harm performance)
  • small but noticeable preformance impact, icons need to be converted to bitmap before passing to MapLibre, so actually not SVG icons
• [ ] decent battery performance (at least not even worse than Tangram)
  • Actually there is definitely higher CPU usage when scrolling and zooming, even without any added icons, so effect on battery is quite likely. Though there is a chance this is caused by the jawg default style used for testing.
• [x] actively maintained
• [x] usable as far as documentation and bugginess goes
• [x] on a compatible license
• [x] iOS support. But that is a given.

specific mandatory features:

• [x] show map from a vector source
  • [x] compatible with data schema used by Jawg
  • [ ] compatible with data schemas used by alternative providers (see branches of map style)
  • raster tiles not tested, but should be possible
• [x] custom styling of a vector map
  • current style is basically what is provided by jawg, just switched from streets v2 + hillshade v1 tiles to pure streets v2 (this is what SC uses now): https://github.com/streetcomplete/StreetComplete/blob/031b98460fce046cf679cba7ca9a1dfb677e2a1c/app/src/main/assets/map_theme/jawg-streets.json
• [x] display of interface above map (zoom/locate/undo/hamburger icons, star count)
• [x] show many SVG based icons at specified positions (quest icons)
  • not actually SVG based, as all drawables are converted to bitmaps by MapLibre
  • [x] raise interface on clicking them
• [x] display of a custom node geometry (quest object, hidden dots, current location)
• [x] display of a custom way geometry (quest object, GPS tracks)
  • tracks not tested, but way geometry works
• [x] display of a custom area geometry (quest object)
• [x] display of a custom labels and icons (display of nearby objects for some quests)
• [ ] display of rotated image at specific point (to display direction of view, pointer to the current location when offscreen)
  • not tested, but rotated direction of view is displayed
• [x] fetching map data into cache
• [ ] smooth zooming (on quest selection, map panning, following GPS location)
  • manual zooming is smooth, but zoom animations on quest selection are choppy
• [x] 3D view (is it a critical feature or nice to have?)
• [x] possibility to display a line left and a line right of the "real" line, each with a different color. (For sidewalk/cycleway layer). Very likely possible, see e.g. zlant parking lanes viewer
• [x] fetching map data into cache: This means, it should be possible to fill the cache for MapLibre using a process that has nothing to do with MapLibre (or MapLibre has specifically the functionality to pre-download an area). That's what we do currently - download an area into the cache that tangram uses.
• [x] zooming into and/or calculating which camera position is necessary to fully display a given geometry (with offsets, because the bottom sheet obscures a part of the map)
• [ ] regarding quest pins: 1. it should be performant to display thousands of them simultaneously. 2. it should be performant to add/remove single quest pins.
  • 1. there is a small, but noticeble slowdown compared to no pins
  • 2. performance comparison is difficult but changing single pins require re-setting all data for the source. Rhough Removing (or rather hiding) might be possible using filters.

nice to have:

• [x] preferably not controlled by one company that may drop support at any time
• [ ] smarter/faster update of displayed quests ( https://github.com/streetcomplete/StreetComplete/issues/3269 )
  • pins are not gone as in the linked issue, but is it really faster or smarter?
• [x] easy disabling/enabling of map rotation ( https://github.com/streetcomplete/StreetComplete/issues/3022 )
• [x] smarter handling of rotating map while note is being placed ( https://github.com/streetcomplete/StreetComplete/issues/3101 )
• [x] supporting Android version range includes all versions supported by SC (I guess that dropping support for some rare versions would be survivable but avoiding this is highly preferable)
• [ ] simultaneous camera animations. E.g. the view is animating the rotation with the movement direction but then also an animation for zoom-in (after user presses button) starts
  • not tested

not in the list, but relevant and not tested:

• anything related to changing the map style

I think the next steps should be:

• try the untested mandatory features (especially offline tile management, including a defined tile cache size and raster tiles for satellite view)
• actually make things show the same as tangram (overlays, quest dots, nearby element highlighting, hiding pins when highlighting a quest)
• remove tangram and test performance (to make sure side-by-side view doesn't do bad things here)
• cosmetic stuff (map style, actually showing pins instead of just the round icons,...)

[westnordost]

display of rotated image at specific point (to display direction of view, pointer to the current location when offscreen)

This would not really be necessary, as the location and view direction could also be shown as normal views on top of the map, just like that arrow that points to your location if it is out of the screen. It may even be favourable, as AFAIK the fact that these things are currently drawn by tangram-es trigger a re-draw basically every frame

But since camera movements with animation are supported, I guess smooth zooming is possible.

I mean, they are not really supported by tangram-es (not multiple properties at the same time) which is why I built this wrapper around it. There is at least one bug associated with this which is why it would be better if maplibre supported this out of the box.

[Helium314]

Current possible issues:

• 3D works only in MapLibre JS and not on Android, so likely no 3D buildings, https://github.com/maplibre/maplibre-gl-native/issues/777
• I didn't (yet) find a way of downloading map tiles for offline use, unless the map style is downloaded together with the map tiles, https://github.com/maplibre/maplibre-gl-native/issues/609

[westnordost]

I didn't (yet) find a way of downloading map tiles for offline use, unless the map style is downloaded together with the map tiles, https://github.com/maplibre/maplibre-gl-native/issues/609

Oh, that'd be bad. There are two possibilities to do this:

• via some kind of cache control: I.e. tell maplibre where to put the cached tiles (or know where they are) and force it to have a lifetime of X. The offline tiles downloader does currently do exactly that - download the tiles into the same directory as tangram-es would for caching and tell tangram-es not to delete it if it is newer than X
• via some kind of offline datasource. I think this is possible in tangram-es too, but it is more fiddly because one would have to switch between online/offline source somehow

[Helium314]

via some kind of cache control

This will probably not work. Tiles are in a SQLite database, and if we add tiles to the database, we would somehow need to make sure the database is not opened / accessed by MapLibre while the tiles are persisted.

via some kind of offline datasource

So far I haven't found anything useful. There is a CustomGeometrySource, but this seems to expect Features and not tiles.

[mnalis]

This will probably not work. Tiles are in a SQLite database, and if we add tiles to the database, we would somehow need to make sure the database is not opened / accessed by MapLibre while the tiles are persisted.

Don't know about specific MapLibre case, but AFAICT SQLite should generally support multiple clients reading/writing to the database at the same time. e.g. https://stackoverflow.com/questions/34409275/concurrency-in-sqlite-database

[westnordost]

Tiles are in a SQLite database, and if we add tiles to the database, we would somehow need to make sure the database is not opened / accessed by MapLibre while the tiles are persisted.

You mean other than tangram-es, maplibre does not use a http cache at all? Are you sure? This sounds a bit over-engineered, especially if some kind of offline-feature is not supported at all.

[Helium314]

Don't know about specific MapLibre case, but AFAICT SQLite should generally support multiple clients reading/writing to the database at the same time. e.g. https://stackoverflow.com/questions/34409275/concurrency-in-sqlite-database

Thanks! I remember I had read some warning to not do this, but looks like it's worth a try.

This sounds a bit over-engineered, especially if some kind of offline-feature is not supported at all.

Well, there is real offline functionality (though this is definielty a bit over-engineered for what we need). But as far as I understand, the entire OfflineRegion that is downloaded is connected to a specific map style, and this style can only be dowloaded (https://github.com/maplibre/maplibre-gl-native/issues/609). I haven't tried downloading and then accessing the map tiles with default style... let's see if I can put a minimal style JSON with the correct tile URL somewhere online (not exposing the API key complicates this a bit).

Actually for some part I like having a database for the cached tiles, because this eliminates the potentially huge file system overhead, see #3417. Though it can't be simply removed by Android when running out of disk space...

[Helium314]

I haven't tried downloading and then accessing the map tiles with default style... let's see if I can put a minimal style JSON with the correct tile URL somewhere online (not exposing the API key complicates this a bit).

Ok, this seems to work. Tiles downloaded for a style that shows nothing are visible on a map that's using default style. see https://github.com/streetcomplete/StreetComplete/commit/93ab626a0abc0160759871cb26054ec58decef04 So I see this as working and won't do more on the download now.

[HolgerJeromin]

I have no insight, but could the offline-manager be used for pushing the already downloaded osm data into mapLibre?

[Helium314]

Many things already work now, with some issues:

• Overlays and geometry highlighting is more or less working now, but there are no icons for highlighted geometry, and no dashed lines
• ~Clearing highlighting also removes overlay geometry (this is only becuase of the lazy testing implementation)~
• There is no "annotation" (icons & geometry) order, so quest pins happily change order, and geometry is usually displayed above quest icons
• Anything regarding quest dots, showing icons dependent on zoom, and collision behavior is missing
  • Edit: now quest icons are switched to dots on zoom 14

I have no insight, but could the offline-manager be used for pushing the already downloaded osm data into mapLibre?

I don't think so, you can't define an OfflineRegion without an URL (well you can, but then it's useless). Though maybe if SC would setup a local http server from which MapLibre could download? Most likely not worth the work.

[Helium314]

If anyone wants to test and find issues, a working version (with different package name) is here

Already known issues:

• location circle missing
• location direction only updating when position changes
• no dots on high zoom
• dots overlap on low zoom
• no order/priority for anything, which quest icon is shown is not defined, and geometry might be displayed above quest icons
• icons for nearby geometry not displayed (e.g. telephones, bicycle parking,...)
• icons for shop overlay are white
• no 3D buildings
• quest icons for other quests remain when clicking a quest
• mapLibre displays another compass
• mapLibre icons smaller than tangram (may depend on device)
• width and offset values in annotations don't change with zoom, which is bad for overlays because highway width and position of sidewalks is wrong (except on a specific zoom level)

Semi issues (fix known, but too much work for now):

• tile download not happening when downloading an area (needs style json with correct api key online)
• weird camera behavior: mostly tangram camera is used, and mapLibre set to similar but somewhat wrong values when camera position is updated. Except when clicking on an element / quest that needs to be focused, then mapLibre is focusing nicely while tangram does nothing.
• quest icons shown at center, without pin
• different map style

If anyone wants to help: the most important thing to be solved is how to best (or at all) add things to the map (geometry, quests, overlays). Best means it should

• be fast enough
• have a way of defining order / priority
• allow disabling an entire type (e.g. hiding quests while an element is focused)
• make quest dots work well enough

Currently annotations are used, but there are other ways that may or may not work for SC. What's probably worth investigating are layers, sources and filters (they are all connected, but I don't see through this yet).

Edit: Having many annotations clearly has a bad effect on performance. When there are many quests, zooming is noticeably stuttering (on my old device), while it's fine when all quests are disabled. This is not an issue in tangram. Even without quests tangram feels a bit smoother than MapLibre, though this might be caused by the more detailed map style used for the MapLibre map. Also MapLibre pan/zoom behavior doesn't feel right. I have the impression there is some delay before move/zoom is happening and it doesn't seem to be a performance issue.

So my impression is that we shouldn't use annotations, because of performance and because of the overlay issues,

[Helium314]

I had a closer look at how the annotations work. Each AnnotationManager adds a GeoJsonSource and a layer (symbol, circle, fill or line). Adding / updating source data could be done in a more efficient way, but so far I didn't even test whether that is actually necessary. But after filling the data, I think there is no / little difference to other ways I investigated, so I see no way of improving the general map performance over the current state. Which is really not great if many quests are in view, especially if you have a smoothly working tangram map next to it...

[westnordost]

On slack us, there is a channel called #maplibre on which you may ask maplibre related questions, fyi.

[Helium314]

Looks like it's more or less as fast as it can get now. The main performance issue when scrolling/zooming appears to be sorting, which is noticeably faster (though still not great) when using one unique imporance value per QuestType, instead of one per Quest. And even faster, to a point where it's equal to unordered, is when we set the z-Order to the order of the source. Since for every change in displayed quest pins we need to provide all features to the GeoJsonSource, this is feasible (and sorting pin list by importance is fast).

Note: Single pins can be added/removed using AnnotationManagers, but internally they set the full list of Annotations in a GeoJsonSource, so essentially it's the same.

As for performance of updating a lot of pins, I'd need a way of testing. The update is done in background by MapLibre, so there is no simple way of measuring time. Appears to be fast enough though.

[Helium314]

Checklist updated, remaining things:

• decent battery performance (at least not even worse than Tangram)
  • The comment implies that Tangram has bad battery performance, which in my opinion is not true. But anyway, MapLibre definitely generates higher CPU load than Tangram during normal operation, even without any pins being displayed. Depending on settings, pins may make things considerably worse, but currently have little impact. With Tangram, zooming and panning is smooth even with high CPU load. In MapLibre things are quite choppy in the same situation. In general MapLibre is fast enough, but it expect it to eat more battery than Tangram.
• compatible with data schemas used by alternative providers (see branches of map style)
  • compatible with jawg, but raster tiles are not tested (I very much expect it to work though)
• display of rotated image at specific point (to display direction of view, pointer to the current location when offscreen)
  • I did not test the pointer, but the direction of view is displayed (currently only updated when position is updated though)
• smooth zooming (on quest selection, map panning, following GPS location)
  • Manual zooming is smooth once the pins are loaded. The zoom animations on quest selection are quite choppy, unless there are very few quests in view (but this is on S4 mini). Looks better in Tangram.
• regarding quest pins: 1. it should be performant to display thousands of them simultaneously. 2. it should be performant to add/remove single quest pins.
  1. It's fast enough after some optimizations. Issues were: Sort order with many distinct values (one value per quest is really bad), fade-out of quest pins (simple (dis)appear like in Tangram is better), large quest icons (now scaled to 0.35, which results in same size as Tangram)
  2. Like in Tangram, single pins can't be added / removed. Actual performance comparion is not easy, as MapLibre does the updates in background. My guess from visual comparison: there is not much differnce.
• smarter/faster update of displayed quests ( https://github.com/streetcomplete/StreetComplete/issues/3269 )
  • Pins never disappear as in the linked issue, not sure whether this is really faster or smarter tough.
• simultaneous camera animations. E.g. the view is animating the rotation with the movement direction but then also an animation for zoom-in (after user presses button) starts
  • not tested

What I don't like about MapLibre:

• performance when panning / zooming the map is worse than Tangram
• panning / zooming behavior / feel is not as good as in Tangram
  • especially the fling gesture animation is really weird, accelerates when detected (to speed faster than I moved the map), and then stops rather suddenly
• most info you find is for JS, not android native, which is quite annoying (though most of the relevant info is now already in the comments)
• documentation often pretty scarce

So I think I'm more or less done with exploring, and will leave another apk for everyone who's interested. The actual code is incomplete in some places (camera, offline maps, actual pins instead of round icons, overlays,...). Here it should be clear what needs to be done from the existing code and comments. I think it's a reasonable base if someone wants to work on it (I will not, at least for quite a while).

[matkoniecz]

The comment implies that Tangram has bad battery performance, which in my opinion is not true. But anyway, MapLibre definitely generates higher CPU load than Tangram during normal operation, even without any pins being displayed

SC seems to be eating battery faster than other apps which require continuous screen display and some people commented on how fast battery disappears when using it

[Helium314]

For me, the biggest battery eaters were auto-upload and the theme (on OLED display). When using auto-upload, auto-downlad (can switch it off in SCEE) and light theme, my phone used to get very warm. Now with dark theme and auto-up/download off this doesn't happen any more. My guess is that actually auto-download might be the main reason for high battery usage (network traffic, and high CPU load when persisting data and creating quests).

Btw the performance difference between Tangram and MapLibre seems to be smaller in non-debug version, but still MapLibre isn't as smooth as Tangram.

[mnalis]

My guess is that actually auto-download might be the main reason for high battery usage (network traffic, and high CPU load when persisting data and creating quests).

Data traffic is huge battery eater, definitely. For example, using offline OsmAnd with online showing of notes (which downloads them as-you-go, similar to SC), I can roughly double battery time by disabling showing of notes / data traffic.

Even in SC-only situation, I can greatly extend battery life when I have data disabled, and then stop every few kilometers to enable data for a minute or two, load new areas, and then disable data again for next hour of walking; instead of having data enabled all the time and let SC autodownload stuff every dozen seconds.

SC seems to be eating battery faster than other apps which require continuous screen display and some people commented on how fast battery disappears when using it

Another big power user (as opposed to say notepad app with screen on) is GPS (and bluetooth/wifi if used for positioning) and CPU, as noted.

[Helium314]

I lack the motivation of continuing here, but I updated the comments a little and merged the latest SC changes into https://github.com/Helium314/SCEE/tree/maplibre in case someone is interested on working on this.

The comments contain some information on how things work and what is still to be done. Some short and incomplete summary:

• Currently the map is accessed from everywhere using a companion object in MainMapView. This is super simple for testing, but should be changed to be more compatible with current SC practices.
• I didn't get the MapLibre LocationComponent to work properly, maybe is should be replaced by the old approach using a SymbolManager (still in the file, just not used)
• Quest icons lack the pin, though that should be a rather simple change
• Loading the icons at start seems a bit slow, maybe some caching like it's done for the tangram sprite sheet would help
• Dashes don't work properly, they change size when the (integer) zoom level is changed
• Map style is still some default style from jawg. Much too detailed for SC and quite different looks.
  • The style is in a json file, and seems much less flexible than the tangram yaml files. But the json string can be ajusted when loading the style, e.g. some colors (or color placeholders) could be replaced by dark style versions
  • All of the map style things related to quests / overlays / highlighted elements are not part of the json style. This is not necessary, but I found it easier to work with (json doesn't allow comments...)
• Downloading map tiles together with map data requires a map style on a server, because MapLibre only accepts urls pointing to style json, and not locally available styles. So the offline stuff is working, but currently commented (you can find a working dummy json there)
• The fling animation is truly horrible, but with the next update a better one should come

[1ec5]

Note: Single pins can be added/removed using AnnotationManagers, but internally they set the full list of Annotations in a GeoJsonSource, so essentially it's the same.

If you’re working with a lot of annotations (like several dozen or more) and rendering performance matters to you, you should avoid AnnotationManager in favor of setting up a lower-level symbol layer.

AnnotationManager is optimized for the use case of a single marker on screen, or several markers that share nothing in common and pop in and out at different times. Each instance of this class manages a separate GeoJSON source and style layer, so you could increase the style’s complexity very rapidly by adding many annotations to the map.[^ios]

StreetComplete’s use case calls for a single GeoJSON source and a single symbol layer, using the data-driven styling technique to vary each symbol according to each underlying feature’s properties. Just as with the icons and labels in the basemap, the performance impact of this approach would be negligible unless you animate the camera continuously on a steep tilt (common in navigation applications but uncommon in editors like StreetComplete).

[^ios]: MGLAnnotation on iOS doesn’t work like this. It’s much more performant because it relies on a shared, spatially indexed source type that isn’t exposed publicly. The Android annotations plugin doesn’t have access to this source type.

[Helium314]

If you’re working with a lot of annotations (like several dozen or more) and rendering performance matters to you, you should avoid AnnotationManager in favor of setting up a lower-level symbol layer.

That's the current state of the test implementation, with exception for the layer showing the current user position. But it looks like currently the whole annotation / symbol thing is being worked on. It might be good to wait until MapLibre guys are done with this...

What could be done now and pretty much independent from other progress is creating a StreetCompete map style json. I really didn't want to work on this because being unable to simply comment some parts of the file is absolutely horrible.

[westnordost]

It would make sense if we know that sooner or later we will switch to Maplibre. Is that certain? Your comments so far seemed to be somewhat critical of this.

[Helium314]

There are some checkmarks missing in https://github.com/streetcomplete/StreetComplete/issues/3123#issuecomment-1454681659, but I'm not sure whether this is really the latest state (didn't re-read other comments).

I'm not sure about animations, here we'll have to wait for the next release which should contain some improvements (at very least for the fling animation, maybe more). And my impression is that performance will not be better than Tangram, at best it's the same.

[westnordost]

I recreated the StreetComplete map style for Tangram-ES in MapLibre. Not everything is possible with MapLibre that was possible in Tangram, but since I spent so much time on this, I thought I could also make the stylesheet a bit prettier, too:

• ❌ disabled 3D buildings. The outline cannot be extruded and without it, they don't look so good, IMO
• ❌ no shore wave animation (MapLibre has no shader support), instead a simple white outline that fades in at zoom 19 or so
• ✨ tunnels now have a dashed outline and private ways have some dashed grey lines on top of them - most maps do it that way, I think it is prettier, too. (This would be possible in Tangram, too, of course)
• ✨ path color is now completely brown, to better distinguish from roads. (This would be possible in Tangram, too, of course)
• ✨ bridges are now above non-bridges which are above tunnels. I remember there were some issues in the Tangram style (would probably have been possible to fix them with effort in the Tangram-style, too, though)
• ✨ smooth fading in and out of texts, minor roads, buildings, landuse etc. (where possible)
• ✨ improved railway rendering a bit: when you zoom in enough, you can actually see both tracks
• ✨ hill shading
• ✨ pitches have a slim outline now, so they are distinguishable from other "green" areas

One thing that doesn't render yet is the oneway-arrow. Don't know what's wrong, will have a look at it some other time.

In general I ended up with a little less thick lines at the same zoom level, and also in general features tend to be visible about one zoom level later. But for both, I think it is a good thing.

Test it here: https://www.westnordost.de/misc/map.html

Repo is here: https://github.com/streetcomplete/maplibre-streetcomplete-style Writing styles for MapLibre in JSON is actually so cumbersome and involves a lot of copy&paste that I rather specified the style in a simple DSL I whipped up in Kotlin. So that Kotlin app outputs the style (normal and night-style).

Maybe we need to end up completely defining this style in code later on to be able to dynamically generate it at runtime, because I didn't find a way to inject into the currently loaded map style such things as which language for places to prefer, font scaling etc. and we certainly want to avoid having a streetcomplete.json for every preferred language. We'll see, for now I think it is fine.

---

This is just the styling for the Jawg background map. The style for whatever StreetComplete renders additionally can hopefully be defined in code and doesn't need to be "baked" into the style json like it was necessary for Tangram.

[1ec5]

❌ no shore wave animation (MapLibre has no shader support), instead a simple white outline that fades in at zoom 19 or so

You can bring in a custom shader using a custom style layer.

Writing styles for MapLibre in JSON is actually so cumbersome and involves a lot of copy&paste that I rather specified the style in a simple DSL I whipped up in Kotlin. So that Kotlin app outputs the style (normal and night-style).

Yes, if you aren’t using a visual design studio like Fresco, then code generation is a sound approach.

Maybe we need to end up completely defining this style in code later on to be able to dynamically generate it at runtime, because I didn't find a way to inject into the currently loaded map style such things as which language for places to prefer, font scaling etc. and we certainly want to avoid having a streetcomplete.json for every preferred language. We'll see, for now I think it is fine.

Yes, this is why OSM Americana builds the style at runtime. It also outputs a prebuilt stylesheet, but that stylesheet can’t adapt to the user’s language and other environment settings.

[westnordost]

Fresco... it looks really good. Too bad, it wasn't (really) linked from the OSM wiki on the topic of MapBox styles, so I used Maputnik but on first glance I like Fresco more, because it is closer to the code. (And anyway, I was using Maputnik basically only to view the style I have been editing in code.)

Maybe we need to end up completely defining this style in code later on to be able to dynamically generate it at runtime, because I didn't find a way to inject into the currently loaded map style

Oh, actually, someone in #maplibre-android told me that one can access and edit properties of layers from the style:

maplibreMap.style?.getLayerAs<FillLayer>("someLayer")?.setProperties(...)

[1ec5]

Good point, added it to https://wiki.openstreetmap.org/wiki/Mapbox_styles#Editors

[westnordost]

Regarding the status of the (exploration of the) migration to MapLibre:

I asked in the chat recently about the next version of MapLibre, and it seems to mainly just contain some renamings of classes (replace MapBox with MapLibre) and some work on Metal (for iOS). In any case, nothing big. (I didn't look at the unmerged PRs, though)

@Helium314, would you consider the work you did to advance this in the maplibre branch on SCEE to be a good base to finish the implementation, or would you say that it was rather a prototype to have a stab at how (and if) things could be done with MapLibre? Regarding your post from April, have you lost motivation to work on it for good, or is this something you may carry on with later? (Maybe spurred by the creation of that MapLibre style? 😛)

---

By the way, I am amazed how much more smoother and cleaner the map looks with maplibre, especially in direct comparison between the style for Tangram and MapLibre. Yes, I've made some style changes, but look at for example the font rendering.

[Helium314]

@Helium314, would you consider the work you did to advance this in the maplibre branch on SCEE to be a good base to finish the implementation, or would you say that it was rather a prototype to have a stab at how (and if) things could be done with MapLibre? Regarding your post from April, have you lost motivation to work on it for good, or is this something you may carry on with later? (Maybe spurred by the creation of that MapLibre style? 😛)

I'd definitely call it a prototype. Some parts are probably fine, while others are more experimental. E.g. all the stuff in MainMapFragment is mostly for testing how to do things, contains a lot of comment on what works well and what doesn't, contains overlay styles that should better be in some style json, contains stuff that should not be in there,...

[Helium314]

I think I would work on it again, but can't really say whether I'll have the necessary time

[westnordost]

Cool! It would also be the most efficient way to go about this, because you already familiarized yourself somewhat with the framework. I'll consider this ticket reserved for now (there is enough other things that potential contributors could do).

[Helium314]

I started checking the current state again and worked on some stuff that was still missing.

Anyway, I remembered the main issue with the current state: interaction with the map (sources and annotation managers) must be done on UI thread, and for testing I simply exposed the MainActivity in its companion object for runOnUiThread. This works, but is not the way to go. How should it be handled properly?

[westnordost]

Hm well, any interaction with the UI has to be done on the UI thread in Android. So I guess, the answer is, to

• (optional:) annotate every method that controls the Maplibre map with @UIThread to make the IDE understand and highlight that it is an error to not call it from a UI thread
• wrap any data updates the same way as it is done for UI updates (viewLifecycleScope.launch { ... })

[westnordost]

FYI work in progress here: https://github.com/Helium314/SCEE/pull/516