stamen / chartographer

https://stamen.github.io/chartographer/
MIT License
19 stars 4 forks source link

Move SVG data into store and render in chunks #46

Closed aparlato closed 1 year ago

aparlato commented 1 year ago

Closes https://github.com/stamen/chartographer/issues/41

@ebrelsford this moves the svg data into a store so that we don't need to recalculate the svg data from the expanded layers. It improves performance, but not so much that this feels like the final solution here: After layer expansion, on the giant openstreetmap-americana style, the fills page (on second load after we fill the store so just render time) takes about ~500-900 ms to mount vs the previous ~2000 ms. This is still pretty long for a user.

In #41, I thought we might be able to handle this lag time by running a loading screen, but because this time is the time the browser is taking to actually try and render such a huge SVG, it's blocking and we can't render a load screen in the interim.

Potential path forward on performance

I timeboxed a little experimenting in experiment-perf using @svelte/svelte-virtual-list to only render the SVGs we are actively showing on the screen at a time. This brings the mount/render time mentioned above down to ~3 ms.

NOTE: If you check out that branch to take a look, know that I've only worked on the fill chart so far and expect it to be fairly janky because it hasn't been finished. I only went far enough to see proof of concept for myself.

The side effect of the approach there is that we need to rethink our SVG export because:

Question

Obviously, styles of this size (openstreetmap-americana) are an edge case, but I wonder if we should rethink how to export our SVG images to allow the greater good of much faster performance/rendering when just using the web app.

If we decide to rethink this, do we know how often cartographers are currently using the export capability and whether it's currently achieving its goals?

@ebrelsford curious what you think about all of this and whether we should merge this as a minor improvement, close as overwrought for the benefit it provides, take the perf improvement path mentioned above and rethink exports in this PR, or as handle that perf improvement as followup?

aparlato commented 1 year ago

@ebrelsford after some more experimenting, I was able to find another way to handle rendering here. I wasn't having any luck with a variety of setups I expected to "just work". Most of the asynchronous setups I found were based in fetching data where the fetch request was blocking, whereas we just want to render a ton of stuff we already have without totally blocking the main thread.

So the solution in place here is to break the svg data we want to render into chunks (of 100) which render fast, then use a setTimeout as a throttle to add more svg data to the array of data we're displaying whenever the last item in a chunk renders. We know when items render because we dispatch an event onMount from SlotWrapper which just returns its own children (parts of the SVG), but gives us insight into when that child has rendered. The setTimeout/throttle keeps it from being blocking and keying each item in the each block prevents rerenders when we extend the list so it only renders new stuff.

This also preserves the SVG download we already have in place.

I've also added a mini loader in the right corner while it's still loading more layers to display.

Also, as followup, as the fill and line charts increase in complexity I'm inclined to think https://github.com/stamen/chartographer/issues/17 is becoming higher priority to prevent this sort of duplicate work.