Performance issues

[giaur500]

I noticed performance when zooming/moving is actually poor in compare to native solutions like Google Maps or Mapbox official api. On low end devices map is almost unusable.

From other side, raster tiles from flutter map plugin are still very fast. Are you aware performance issues and is there anything possible to make it working faster?

[FaFre]

If you are interested, there are some past conversations in the closed issues where you can have a read and might add some ideas. The tile rendering (drawing on canvas) is currently only possible in the flutter main isolate (https://github.com/flutter/flutter/issues/75755), when I'm not mistaken this is still the bottleneck.

You can improve performance by tuning your style. Feel free to do your own experiments and share it with us :)

[greensopinion]

We've done a lot of work to address performance issues, but it's still not as good as raster or native maps. Poor performance is most noticeable with tiles having a lot of data (e.g. in LA or London). You can find previous discussions by searching for other issues

The theme can make a big difference as noted by @FaFre (thanks!) but you may find that raster tiles are needed to get the performance that you want, depending on your use-case.

I've added some examples that you may find helpful here: https://github.com/greensopinion/flutter-vector-map-tiles-examples There are two custom themes in there that may help with performance.

[giaur500]

As far as I understand, currently is not possible to optimize it becuse of Flutter limitations? Ok, I will try themes, Indeed, a lot of data causes poor performance.

[FaFre]

@greensopinion I pushed the flutter issue and got a response that flutter_isolate plugin might improve performance and gain access to dart ui: https://github.com/flutter/flutter/issues/75755#issuecomment-1434935593

However, this plugin is only Android/iOS compatible, do you see any chance this would improve the current performance?

[greensopinion]

Yes, there's a good chance that it could be used to offload some CPU-intensive tasks off the UI thread. We are already offloading some work to isolates, but there are many things that can't be offloaded due to use of dart:ui.

I imagine that the easiest and most beneficial thing to do would be to render a raster image on an isolate. This could either be integrated directly with the existing flutter_map raster tile capabilities, or perhaps in vector_map_tiles. The idea would be to render vector tiles as images and supply those to flutter_map. The benefit over using raster tiles directly would be choice of themes client-side, and rendering tiles at higher zoom levels that aren't normally available. At 2x (retina) size raster tiles look pretty sharp, but we could even render tiles at incremental zoom levels (e.g. a tile at 14.25 zoom instead of 14 then scaled to 14.25)

I'm a little reluctant to invest in functionality that is using officially unsupported flutter/dart capabilities, such as dart:ui on non-UI isolates. As noted on flutter/issues/75755 things may already break with Impeller.

If we consider moving forward with this idea, we should first evaluate how Impeller affects performance. For example, a comparison running the example library in the profiler with Impeller and without would be really helpful.

[giaur500]

@greensopinion do you think it's possible to disable Impeller and use isolates on Android only?

[greensopinion]

At least for now it's possible - but that will likely change once Flutter fully adopts Impeller since they won't want to maintain two separate rendering frameworks for Android indefinitely.

[giaur500]

I don't know how much work is it, but I think better to use isolates on Android, even as temporary solution. Still more than nothing.

[greensopinion]

However, this plugin is only Android/iOS compatible, do you see any chance this would improve the current performance?

I've pushed an experiment to https://github.com/greensopinion/flutter-vector-map-tiles-raster-experiment Feel free to try it out!

[giaur500]

It works pretty fast, almost the same performance as native Google Maps. Very good result in my opinion. Not tried on iOS though

[aamirki]

Hey I just tried out that experiment on iOS and the performance is really good!

[giaur500]

After more testing. It seems to be better, but still not as good as rester map or native maps like Google Maps. I will test without any style applied to see how does it work, possible only style applied causes better performance

[greensopinion]

I've created a branch in this repository for experimentation: flutter-vector-map-tiles/tree/raster-experiment

There are a few things needed to take it further:

1. there may be a memory leak
2. the API is unnatural and needs revisiting
3. there is no way to plug in rendering in a separate isolate
4. tuning

[Pepslee]

However, this plugin is only Android/iOS compatible, do you see any chance this would improve the current performance?

I've pushed an experiment to https://github.com/greensopinion/flutter-vector-map-tiles-raster-experiment Feel free to try it out!

It works really cool. But there is some problem in my case. When I'm panning for a long distance it seems that a huge number of isolates creates for all panned tiles. Seems like tile provider doesn't drop deprecated isolates for tiles which was rapidly panned. For example, I have 100 tiles on the map and I`m trying to pan from 1 tile to 99 tile by one swipe. Tile provider starts to create 99 isolates and tries to process them all but not just skips unnecessary tiles and process only several last tiles.

[greensopinion]

The experiment is only intended as a proof-of-concept. There is a lot of work needed to make it ready for real usage.

I've been working on #129 to introduce raster images as an alternative to vectors. Raster images have the advantage of improved frame rate (less jank) and they only need to be rendered the first time since they can be stored in the cache for future use. The result is quite good. Of course, raster images aren't as sharp at fractional zoom levels. #129 is now available as version 3.3.0 of this library.

[Pepslee]

The experiment is only intended as a proof-of-concept. There is a lot of work needed to make it ready for real usage.

I've been working on #129 to introduce raster images as an alternative to vectors. Raster images have the advantage of improved frame rate (less jank) and they only need to be rendered the first time since they can be stored in the cache for future use. The result is quite good. Of course, raster images aren't as sharp at fractional zoom levels. #129 is now available as version 3.3.0 of this library.

Thank you, I have already tried this update, it work really faster, but I still have freezes ((. In any case any calculation in main thread entail freezes of the UI. Even if they won't be visible for the human eye, but they will still affect any timer or animation in app. I hope this problem will be solved with impeller, and there will be a mechanism for rendering in non main thread.

[JaffaKetchup]

@greensopinion Please note https://pub.dev/packages/flutter_map/versions/4.0.0-dev.1/changelog

[greensopinion]

Thanks Luke. Is the update to let me know about API changes or does it specifically relate to performance?

On Tue, Apr 4, 2023 at 5:15 AM Luka S @.***> wrote:

@greensopinion https://github.com/greensopinion Please note https://pub.dev/packages/flutter_map/versions/4.0.0-dev.1/changelog

— Reply to this email directly, view it on GitHub https://github.com/greensopinion/flutter-vector-map-tiles/issues/120#issuecomment-1495869846, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEKDQ7WZYRQ3S72XYZHLMNDW7QGGPANCNFSM6AAAAAAU4Q2O2E . You are receiving this because you were mentioned.Message ID: @.***>

[JaffaKetchup]

@greensopinion Both. Probably not a massive performance improvement, but there is a chance it will help. There are major API changes - but I'm not sure if any of them will affect you. Probably the most major for plugin maintainers is the renaming of Coords to TileCoordinates.

[greensopinion]

Thanks @JaffaKetchup - created #136 in preparation for flutter_map major version change.

[mdmm13]

Hey @greensopinion, was trying to follow up on your discussion here about using flutter_isolate to offload processing yet still able to call dart.ui functions outside the main isolate. It's not the full solution I understand from your comment here, but part of it. The other part being the complexity of the theme, which you're trying to solve by selectively delaying the rendering if I understood correctly.

I'm not sure I got it right, but per nmfisher's (flutter_isolate maintainer) last comment here, the status on this PR being merged into dart's main branch and the comment here, it appears that the functionality that Impeller previously blocked on iOS should now be back again with Dart 3?

Does this mean, flutter_isolate + selective delays COULD solve many of our performance issues? What am I missing?

[DavidDohmen]

Hey, @greensopinion - have you considered drawVerticesRaw for rendering? It seems to be orders of magnitude faster than using widgets or canvas. This video explains it very thoroughly and the bookmarked section shows a comparison between the different methods; https://youtu.be/pD38Yyz7N2E?t=742

[mdmm13]

Hey, @greensopinion - have you considered drawVerticesRaw for rendering? It seems to be orders of magnitude faster than using widgets or canvas. This video explains it very thoroughly and the bookmarked section shows a comparison between the different methods; https://youtu.be/pD38Yyz7N2E?t=742

Would probably solve all problems with vector maps without going GL?

[JaffaKetchup]

Just for context, we're looking into potentially using drawVertices to draw polygons in FM, but that's still in mostly private discussion. You may also be interested in dart_earcut (self-promo :D), which will perform the triangulation needed for drawVertices extremely quickly (it doesn't work well on self-intersecting polys, they'll need to be simplified into multiple simple polys).

[greensopinion]

Looks promising! Feel free to prototype something.

[DavidDohmen]

Unfortunately I have neither the time nor skillset to do so. We would however contribute financially with a bounty, if a certain speed level is reached, because then flutter_map would become feasible as an alternative to mapbox for us. So if anyone with the right skillset wants to pick this up, let's talk!

[j-bbr]

I might be able to spend some time on this in the near to medium term. So I was wondering, first of all, whether somebody is already working on this or on similar things with drawVertices in FM as @JaffaKetchup mentioned?

[JaffaKetchup]

@j-bbr Right now we are not implementing drawVertices, and if we do, it'll be for Polygons only: we have no plans to make a vector layer for now.

[greensopinion]

Go for it!

On Tue, Dec 12, 2023 at 5:56 AM j-bbr @.***> wrote:

I might be able to spend some time on this in the near to medium term. So I was wondering, first of all, whether somebody is already working on this or on similar things with drawVertices in FM as @JaffaKetchup https://github.com/JaffaKetchup mentioned?

— Reply to this email directly, view it on GitHub https://github.com/greensopinion/flutter-vector-map-tiles/issues/120#issuecomment-1852086285, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEKDQ7X4E6PH7CN5XGCBW7LYJBPARAVCNFSM6AAAAAAU4Q2O2GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJSGA4DMMRYGU . You are receiving this because you were mentioned.Message ID: @.***>

[j-bbr]

@greensopinion Just to keep an open mind for which improvements to go for I was wondering whether I could get your opinion on the following so I don't go down a path that has already turned out to be a dead-end for you: Besides drawing faster, potentially with drawVertices there is of course also the option to draw slower :) meaning in this case to spread drawing out over multiple frames. You're of course already doing that with Text but maybe this could be extended to partial drawing of layers (at the moment all layers except symbols are drawn in one frame, please correct if I am wrong here)?

Rough Todo-List: keeping track of partially drawn tiles partially drawing layers (fixed number initially, maybe within a time budget later) retaining the partially drawn state efficiently for continuation in the next frame (not possible afaik with CustomPainter, maybe the VectorTileLayerMode.raster code can be a start for this, or triangle lists for drawVertices)

Any thoughts, pitfalls you can already see here?

[greensopinion]

I have a tough time imagining a good user experience if maps have features appearing/disappearing while zooming or panning. I'd focus on two approaches:

1) make what we do today faster, and

2) experiment with raster base layer tiles with vector detail

Raster tiles could be rendered using vector data on an isolate (eg with flutter_isolate) without text or other symbols. This would offload that work from the UI thread.

Before deciding on either it's probably worthwhile looking at the profiler to see what's expensive. For example, if polygons aren't contributing much to jank then moving those off to a raster base layer might not improve anything. maybe you'll have new/better ideas after checking the profiler since right now I'm just guessing.

Have fun!

On Thu, Dec 14, 2023 at 1:31 AM j-bbr @.***> wrote:

@greensopinion https://github.com/greensopinion Just to keep an open mind for which improvements to go for I was wondering whether I could get your opinion on the following so I don't go down a path that has already turned out to be a dead-end for you: Besides drawing faster, potentially with drawVertices there is of course also the option to draw slower :) meaning in this case to spread drawing out over multiple frames. You're of course already doing that with Text but maybe this could be extended to partial drawing of layers (at the moment all layers except symbols are drawn in one frame, please correct if I am wrong here)?

Rough Todo-List: keeping track of partially drawn tiles partially drawing layers (fixed number initially, maybe within a time budget later) retaining the partially drawn state efficiently for continuation in the next frame (not possible afaik with CustomPainter, maybe the VectorTileLayerMode.raster code can be a start for this, or triangle lists for drawVertices)

Any thoughts, pitfalls you can already see here?

— Reply to this email directly, view it on GitHub https://github.com/greensopinion/flutter-vector-map-tiles/issues/120#issuecomment-1855490233, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEKDQ7WKNVRFM5IQMWX6YEDYJLBP3AVCNFSM6AAAAAAU4Q2O2GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJVGQ4TAMRTGM . You are receiving this because you were mentioned.Message ID: @.***>

[greensopinion]

I've pushed a prototype rendering raster tiles from vector tiles. To use it, you can check out the following branches:

• https://github.com/greensopinion/flutter-vector-map-tiles/tree/raster-on-isolate
• https://github.com/greensopinion/flutter-vector-map-tiles-examples/tree/raster-on-isolate

This prototype is pretty rough, with notable deficiencies:

• isolates are leaked whenever the tile provider is recreated
• data transfer between isolates causes each tile to incur about 2s of latency, which in turn causes queueing
• queueing causes additive latency
• conversion of raster images to png occurs twice

Switching between raster/vector and panning provides a good contrast in jank and tile latency.

[Soap-141]

I don't know if it would be useful for you guys but I tried Here SDK for Flutter and the map using vectors doesn't have any performance issue even in 3D. It's insane and I wonder how they are doing it.

[JaffaKetchup]

@Soap-141 They're probably using a platform view ~which is cheating :D~. But on a serious note, FM is 100% Dart, which means maximum flexibility and stability, but potentially sometimes at the cost of performance.

[mdmm13]

I don't know if it would be useful for you guys but I tried Here SDK for Flutter and the map using vectors doesn't have any performance issue even in 3D. It's insane and I wonder how they are doing it.

They're using native SDKs with a Flutter layer on top AFAIK (https://www.here.com/docs/bundle/sdk-for-flutter-explore-api-reference/page/core.engine/SDKNativeEngine-class.html). You'll find plenty of references to JNI etc. in their changelogs.

[JaffaKetchup]

Just an update, we are now working on drawVertices for FM Polygons, and there's a working implementation for non self-intersecting polys (holes are supported). It's drastically improved raster times, and uses dart_earcut for triangulation. Maybe it could be helpful for those wanting to see how to do it here. There's an open PR by me in FM.

[greensopinion]

That's cool Luka. Let us know how it goes.

On Wed, Jan 31, 2024 at 6:32 AM Luka S @.***> wrote:

Just an update, we are now working on drawVertices for FM Polygons, and there's a working implementation for non self-intersecting polys (holes are supported). It's drastically improved raster times, and uses dart_earcut for triangulation. Maybe it could be helpful for those wanting to see how to do it here. There's an open PR by me in FM.

— Reply to this email directly, view it on GitHub https://github.com/greensopinion/flutter-vector-map-tiles/issues/120#issuecomment-1919222292, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEKDQ7QAQQAQHZZHQHZWYRTYRJIWDAVCNFSM6AAAAAAU4Q2O2GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJZGIZDEMRZGI . You are receiving this because you were mentioned.Message ID: @.***>

[ignatz]

Hey folks, just found this thread. Inspired by https://github.com/fleaflet/flutter_map/pull/1800, I took a quick stab at performance improvements:

1. I did a bunch of micro-optimizations. Most are probably not even worth.
2. Batched the canvas draws
3. (I also made web rendering work)

As a result (mostly of (2)) my raster times more than halved, at least for simple pans (my setup: linux notebook and compiled in profile mode and concurrency set to 0 to make this as straightforward as possible):

vanilla: https://github.com/greensopinion/flutter-vector-map-tiles/assets/111986/92d08665-3d15-4104-ba37-2a4f5dc0886d

optimized: https://github.com/greensopinion/flutter-vector-map-tiles/assets/111986/bfded73d-bea0-4111-9470-c778e539cb12

Depending how I'm moving the map I'm still seeing quite a bit of (presumably) shader jank. Hopefully impeller will help with that.

I'm also hopeful that using drawVertices would improve raster performance even further.

[ignatz]

I also quickly switched to drawVertices, it does help (avg raster times drop from 3.5ms to 3ms) but most of the raster time is spent on lines:

https://github.com/greensopinion/flutter-vector-map-tiles/assets/111986/ed16be9d-f2cd-41c3-87b3-2b2c32faab16

EDIT: After selectively disabling many of the "features", I wouldn't necessarily say that lines are the most expensive. Even when just rendering the fills, I'm averaging 2ms for the same scene.

I would hope that zoom-level dependent geometry simplifications and impeller are obvious remaining low hanging fruit.

[josxha]

As far as I've seen the line renderer currently uses drawPath(). Do you think drawPoints() or event drawRawPoints with PointMode.lines could help here?

[ignatz]

As far as I've seen the line renderer currently uses drawPath(). Do you think drawPoints() or event drawRawPoints with PointMode.lines could help here?

I've experimented with both for the polygon layer, so I'd naively say no. Then again, based on my late edit, I'm not even sure that lines are that dominant. I'd expect geometry simplifications for both lines and files to probably help more.

[ignatz]

I quickly tried Impeller on Android. The result was shockingly bad. I hope this is to blame on the alpha status. Maybe someone with an iOS device can try?

[JaffaKetchup]

That's interesting @ignatz. I haven't run it on this plugin yet, but I can try soon hopefully (would be grateful if you could post a link to the implementation you're running above).

I would've expected Impeller to make a big difference, especially to lines. It certainly does for the polygon layer on my Android device - can you test that on your device as well? Does your device have Vulkan support?

[mdmm13]

Possibly related, but the Flutter issues are full of performance problems as of 3.16 (e.g. here, here), all related to vector graphics. They concluded that it's caused by tessellation.

What does your CPU profile look like? Same cause?

One interesting aspect is that there appears to be a limit on the number of canvas.drawX operations of ~100 per frame. Anything above that and it degrades massively. Sounds like our issue?

If that's true, then our only angle is to spread out the drawing of different layers across multiple frames in an order of importance? Assume most vector maps do something like this whereby you first see the background and a few frames later the roads and labels appear.

[mdmm13]

And here you've got that drawVertices took a hit with Impeller. The popular YouTube video we used to come up with the drawVertices idea is from Jan 2023. It was not till May 2023 that Flutter made Impeller the default in iOS.

Thoughts?

[ignatz]

That's interesting @ignatz. I haven't run it on this plugin yet, but I can try soon hopefully (would be grateful if you could post a link to the implementation you're running above).

Happy to share my code but it's a messy hodgepodge with inter-project dependencies. Meanwhile, I'd naively expect the effect to be the same independent of drawVertices and batching.

I would've expected Impeller to make a big difference, especially to lines. It certainly does for the polygon layer on my Android device - can you test that on your device as well? Does your device have Vulkan support?

My device is pretty ancient but should support vulkan. I can try the polygon layer in due time as well but so far I wasn't too interested since I don't feel to keen chase down buggy alpha-quality software.

What does your CPU profile look like? Same cause?

I haven't profiled the raster thread yet. I only gave it a quick spin after the raving reviews by Jaffa and feeling a bit too cocky after lowering the raster times by 70% already. As I said, I'm not particularly keen on chasing down bugs in alpha quality software. For the time being, I feel my efforts would be spent better somewhere else, e.g. geometry simplifications.

Tessellation or not, the issue you linked sounds very relevant and plausible: https://github.com/flutter/flutter/issues/131345 . If that was the case, we're not talking bugs but rather crappy implementation. According to the issue, a solution is on halt for "higher priority" work. (tangential rant: gotta love flutter government, hope they'll eventually hand it over to a foundation). Meanwhile, playing with the blending might provide some insights :woman_shrugging:

[JaffaKetchup]

See https://github.com/flutter/flutter/issues/131345#issuecomment-1925338105 for some performance comparisons.

[ignatz]

I spent a bit more time doing some more draw batching. For the same basic panning example my raster times are now down to 2.4ms on average compared to 9.4ms originally. That's roughly a 75% reduction, without any geometry simplifications.

https://github.com/greensopinion/flutter-vector-map-tiles/assets/111986/c7536de1-1907-44b2-90df-cf9938b18d7c

Web is still a bit janky for the same scene at <20FPS but since I'm getting almost 60 FPS for scenes with less geometry in them, I'm fairly optimistic that simplifications might even make web viable :woman_shrugging:

I did clean up my changes a bit, in case folks would like to join: https://github.com/ignatz/dart-vector-tile-renderer/commits/draw_less_smile_more/

[mdmm13]

@JaffaKetchup's discussion here with the Flutter team confirms your hypothesis that batching and cutting down the number of individual draws is where it's at. Very interesting read!

[greensopinion]

Hey all, great to see the collaboration here. I haven't had a chance to catch up on all of the comments but it looks like some promising progress.

If anyone's interested in helping to move things along, #196 is in need of a review.