Closed Rich-Harris closed 7 months ago
This might upset svelte-native unless it is only run on html namespace (and not svg or TNS), otherwise I guess I could add a runtime html parser
This could possibly be further optimized using a tagged template.
I experimented with benchmarking a simple example.
Feel free to point out anywhere I got this wrong, or if there's a better JS/DOM benchmarking site you prefer.
My desktop running Chrome shows that the current imperative code may be faster?
domc is one of the fastest libraries in the js-frameworks-benchmark (the benchmark is mentioned in https://github.com/sveltejs/svelte/issues/1260). It clones elements rather than using createElement
and says it's much faster. There's a full explanation on their readme of what they do.
stage0, by the same author, is also much faster than Svelte. It looks to primarily use template
with innerHTML
. Though the author suggests you may be able to just use a string with innerHTML
for IE11 support.
Also interesting is that they said they got some ideas from morphdom. On the morphdom readme it says that they render to HTML string when doing SSR and perform 10x faster than React as a result, so that might be another idea worth pursuing.
@tivac thanks for running this benchmark. It was a really helpful starting point. I've been playing around with it quite a bit and think I've discovered a number of issues:
I got a really significant speedup on js-framework-benchmark using both the approach suggested here as well as using stage0 to implement a similar approach https://github.com/sveltejs/svelte/issues/1260#issuecomment-640220248. It was about 15% faster on my machine in both cases. I tend to trust js-framework-benchmark
more than jsben.ch because I know some people that are highly experienced with JS benchmarking are looking at it, it tests script load and parse times as well, and it returns more consistent results for me.
The solution from the original post here creates extra bind
attributes. Because Svelte is a compiler, I feel like that's probably not necessary. E.g. consider what domc does.
Both the benchmark included above as well as js-framework-benchmark
create the same fragment numerous times, which seems to be where template + cloneNode
approaches like the one Rich posted above and the stage0 solution (https://github.com/sveltejs/svelte/issues/1260#issuecomment-640220248) would shine. It'd be interesting to test on a real-world example which creates a whole HTML page with a mix of fragments being created just once, fragments being created in an each
, etc. but I'm not sure anyone has the time to create such tooling. @krausest, @leeoniya, and @ryansolid do you guys have any thoughts from your experience on whether template + cloneNode
might still perform better than naive js dom creation if you're creating a fragment only once or in an overall real world example?
I've spent a decent amount of time here.
The problem is you need to get the nodes to clone in the first place. This means generally innerHTML it at some point. Usually on file load. I haven't found that too problematic but it's something. Once you have the nodes cloning is generally more performant once you are creating more than a couple at once. I use the same cloning technique in Solid, as does Mikado, Sinuous. I converted the Vanilla implementations to do it too. Basically the whole top tier of JS Frameworks benchmark use this cloning technique as it improves creation time and actually lowers memory usage. That being said Surplus does not, nor does Fidan and they are still up there. I like using these as a baseline since they very similar internals o Solid. We are talking about 5ms on 1000 rows and maybe like 50ms on 10k.
But some important details. We can't discount the cost of traversing the nodes to attach bindings or the method of doing so. I don't know what that querySelector is doing in the jsbench example but depending a simple directed walk can be more performant. Also Document Fragments are intrinsically more expensive to attach to the DOM than single elements. I don't know exactly how under the hood, but it's like there is an internal traversal. With a single child fragment probably unnoticeable anyway. Again small gains but I generally clone the element rather than the template.content
if possible.
All that being said. Realworld I doubt it makes a big difference. But I like the hydration story. What you find is now the creation code resembles the hydration code. In one case the server-rendered the template and the other you cloned it. Your work then is more or less the same.
Thanks for taking the time to provide such a detailed response Ryan!
@ryansolid:
The problem is you need to get the nodes to clone in the first place. This means generally innerHTML it at some point
The goal is to move the html creation out of js land and into the browser load. No innerHTML, no JS to parse and evaluate. Just let them browser load html in template tags. I think this is the part of the approach people are missing here. No JavaScript will always be faster than some JavaScript.
We can't discount the cost of traversing the nodes to attach bindings or the method of doing so. I don't know what that querySelector is doing in the jsbench example but depending a simple directed walk can be more performant
Turns out querySelector is 3-5 times faster. I did my benchmarks a few years ago, but I doubt that has changed. I tried using childNode indexes, firstChild, etc., but no contest. I suspect the browser does some indexing on creation to support css. It may even make the process linear, but I haven't tested that. And this doesn't even take into account that there is no js to parse and evaluate
Also Document Fragments are intrinsically more expensive to attach to the DOM than single elements
If you know your template fragment only has one child (which you would in a pre-processed framework) you can always optimize by acting on the firstChild in lieu of the fragment. I'd like to see some more benchmarks on this though.
For the HTML creation I was thinking about that with SSR techniques. I imagine for pure clientside Svelte that still isn't an option unless we are talking HTML imports. That would be interesting.
You've convinced me to revisit QuerySelector. I use QuerySelector to grab templates out for hydration since it is much faster than TreeWalker etc, but I am interested to test whether adding an extra attribute on each element and query selecting versus having a pre-calculated traversal path from compilation using firstChild/nextSibling has differences. That would be interesting because all those libraries use the firstChild approach and I'm always looking for an edge.
As for fragments. I was testing an idea of batch creation,. https://jsperf.com/scaling-clonenode A little bit different but the idea was if you knew you were going generally make 100 rows per page couldn't we just do all 100 in one clone operations. In this test I try creating rows 1, 10, 100, and 1000 at a time. In the process I noticed some interesting results. Compare biggestDiv with biggestBlock, this is 1000 rows in a fragment vs 1000 in a div. Second compare oneBlock and oneTr which comes a single row cloning the fragment and a single row cloning the Tr. I use oneTr effectively in Solid but I was looking at different ways to optimize.
Finally I should mention, which I missed in my first response. We can't forget with cloning in general there is the overhead of markers for text that wraps dynamic expressions. They will be joined as a single text node which will screw up insertion points. I use comment blocks in Solid that I don't remove. I just insert before them. In this example they are "sveltetext" text being replaced at creation time. I imagine this might even save a little initial creation time. But it's worth pointing out that there are gotcha's like this more prevalent in the hydration story when you go this way versus per element creation.
At least in the Svelte js-framework-benchmark, I didn't see a big difference between query selector vs referencing the node via children
. The really big difference was in adopting cloneNode
to avoid rebuilding the rows from scratch
children
is super expensive as is childNodes
they basically convert something that is a link list in the underlying implementation into an array. The second you access it you take a hit. If you want to compare use firstChild
or nextSibling
You guys had me testing my sanity but turns out the methods I've been using are still the fastest. Here's the improved benchmark:
Template should be much faster now.
EDIT: And here is the version that leaves the QuerySelector which is slightly slower: https://jsben.ch/8oVCY
As you can see in this case walking the tree manually is faster.
I see template only being 65-80% as fast as imperative in that benchmark, depending on whether I use Firefox (65.34%) or Chrome (82.56%). Multiple test runs flopped results around, but imperative was always faster?
I ran it multiple times. I see the opposite in Chrome 83. Imperative being around 50% of Template. But Chrome 85 does flip it again although it's much closer. Interesting. I wonder how much this varies version to version.
Firefox 78
I would love this as a renderer toggle, otherwise poor old svelte-native is going to need a heck of a lot of work to adapt
I don't know how much I trust this test runner, the more I run it the more all over the place the results seem. I tried in Firefox 78.0b5 and every other run seems to switch.
To be fair JS Frameworks Benchmark is only ran in Chrome so that could explain it favoring certain methods. Templates were definitely faster at the time I changed the vanilla implementation of the JS Framework Benchmark.
I tried porting the test to other test runners which confirm imperative still seems to be faster for this test although it's closer. Not sure if I'm still just missing something, but this deserves closer examination.
EDIT: Compared both methods with VanillaJS version of JS Framework Benchmark and template is still faster. There might be something around this example/implementation that leads to different results. I guess the next thing to try is different variations of handling textnodes.
I do not trust jsben.ch at all. See my comment above: https://github.com/sveltejs/svelte/issues/3898#issuecomment-640372279
@halfnelson I'm not that familiar with svelte-native. How would it be impacted?
I realized that the SSR code already uses template strings. It might be interesting to test and see if that's faster on the client-side. It could potentially reduce the amount of code to maintain if that were the only code that was necessary and we no longer needed a lot of the DOM code
I mean the templates still become DOM nodes that get traversed and bindings added. You do get to save a number of create/appends calls, but the server/client code ends up being still very different. However where the real advantage comes in is unifying hydration. It isn't hard to write the node traversal code to be identical for the render and hydration case since they walk over the same nodes either created from cloneNode or rendered from the server. This means that Svelte which I believe currently ships 2 separate paths could ship a single path potentially reducing code size considerably in this case, even more than the reductions from the pure render case as you'd have those reductions as well.
The best way to handle this though is the way Solid currently does I think which is still generate separate code for the hydratable case so that the client only render case can be the most optimal still. Hydration still requires walking over dynamic parts to gather nodes which can be completely avoided in the optimal client render approach.
As to transfering the Templates in the SSR rendered code.. it depends. On first render I think this has a lot of merit but I wonder if the cacheability of the JavaScript might trump having to send the templates each time. Assuming dynamic content the HTML is not cached but the JS is. But to be fair there are a lot of things that could affect this once you start comparing cache behavior. If our goal is just to prove the smallest JS bundles this is definitely the way to go without much contest.
I was interested in this issue, so I created an experimental implementation using <template>
and cloneNode
.
https://github.com/tomoam/svelte/tree/efficient-fragment-creation
The implementation is only makeshift, but passes the tests. The tests have not been changed except for the test cases in the test/js
directory and a very few test cases.
The implementation of hydration
and namespace
is quite messy.
Here is the keyed result of running krausest/js-framework-benchmark with this implementation. svelte-experimental
is the benchmark score using this implementation.
Note: The scores for
partial update
,select row
,swap rows
, andclear rows
are not helpful. This is because in my environment (M1 MacBook Air, 16GB Memory), the scores for these four test cases are random and vary widely each time these are run, no matter which framework.
It is worth noting that the scores for create rows
, replace all rows
, create many rows
, and append rows to large table
are much improved.
If we are going to improve the performance of Svelte in the future, the use of <template>
and cloneNode
is worth considering.
@tomoam this is impressive! Thanks! Svelte is getting better, day after day! 😄
@tomoam if you remove cpu throttle from the benchmark(there is a line setting it you can comment out) you can get more stable results. M1 has some issue with cpu throttling. Those results are all over due to that.
But yeah as I said a year ago this is definitely good for about 7% improvement on the official setup. If Svelte implements this it won't be in Solid range but it should be in Inferno range in the benchmark. But that isn't even the biggest benefit. Id expect considerable code reduction in the hydration case.
@ryansolid Thanks for the advice. I ran the benchmark in a different environment (Macbook Pro 2016, 32GB Memory) and got the following results. As you said, the score is in Inferno
range.
For Startup Metrics
, the score of this implementation remained as good as the current Svelte.
For Memory allocation
, there was a slight improvement.
after the previous experiment(https://github.com/sveltejs/svelte/issues/3898#issuecomment-912924098, https://github.com/sveltejs/svelte/issues/3898#issuecomment-915751932), I came up with an idea to reduce the bundle size by extending this implementation.
The implementation is here.
I tried to build site
to test it, and the bundle size was reduced.
I compared the total size of the files in the site/build/assets/_app
directory, excluding the css
files and codemirror
.
minified(KiB) | gzip(KiB) |
---|---|
303.64 | 104.05 |
minified(KiB) | gzip(KiB) |
---|---|
276.71 | 96.89 |
minified size is 8.87% smaller, and gzip compressed size is 6.88% smaller.
comparing only the files in the site/build/assets/_app/pages
directory, minified size is 13.67% smaller and gzip compressed size is 8.13% smaller.
my implementation code is a messy, so I don't think I should submit a PR right now.
if the core team is interested, I will try to write a detailed description (maybe, to svelte/rfcs?).
@tomoam thank you so much for this work! The team talked about your efforts today and we're definitely interested and see the value in this. That being said, it looks like a rather large change with a bit more chance of introducing issues than a usual change. We'll at some point in the (possibly distant) future be doing a compiler overhaul and the feeling was that it'd be best to try to make this change then. So let's definitely keep this code around and we welcome experiments to find these performance improvements and edge cases, but maybe wait until Svelte 4 to PR it.
@benmccann Thank you for replying and for bringing up this topic at the maintainer's meeting.
That being said, it looks like a rather large change with a bit more chance of introducing issues than a usual change. We'll at some point in the (possibly distant) future be doing a compiler overhaul and the feeling was that it'd be best to try to make this change then.
I agree with you. The code generated by this experimental compiler is quite different from that of the current Svelte, and has some problems (for example, it may work in production
mode, but not in development
mode. Also, the code is not maintainable ). If the results of this experiment could be of any help in making Svelte better, I would be happy.
I listened to the JS Party Podcast (to be exact, I translated and read the transcript), where Rich mentioned error boundaries, along with a solution to bundle size. That's not included in my experimental implementation, so perhaps Rich has some other great idea. I'm looking forward to seeing it come to fruition.
I hope I can help out here. I've been dissecting Svelte's generated JS output for a while, and I have been thinking if the functionality of create_fragment
could be moved to helpers (e.g. text()
, element()
, etc.), the compiler could be simplified at the cost of bundling the helpers. I have a miniature version of that concept as a private and experimental library; however, the current attribute and event listener diffing is not as efficient as in Svelte or Solid, and my library does not even have conditional or list rendering yet. I can make my repository public if anyone wants to dissect my library or take it for a test run. :)
You guys had me testing my sanity but turns out the methods I've been using are still the fastest. Here's the improved benchmark:
Template should be much faster now.
EDIT: And here is the version that leaves the QuerySelector which is slightly slower: https://jsben.ch/8oVCY
As you can see in this case walking the tree manually is faster.
I am sorry to reply here, and this is probably kinda offtopic: but i clicked one of the jsbench links and it forwarded me to a mac notification center scam website. is this "jsben.ch" website legit in any way?
This will be addressed as part of Svelte 5: https://twitter.com/Rich_Harris/status/1688581184018583558
Will Svelte 5 include the build option to bundle templates into the .html page?
It's not exactly clear what you're referring to, but check out SvelteKit's prerendering, which sounds like what you're looking for
I'm going to close this issue as completed
No, I was talking about general Svelte 5 (which is looking awsome btw). You got all the way to:
var frag = $.template(`<div>Hello World <button> </button></div>`);
Which uses js to create a dom template to create the fragment, but...
...shame to stop there when you could do:
<!-- index.html -->
<template id="abc123"><div>Hello World <button> </button></div></template>
var frag = $.templateById('abc123');
And offload that to the browser (really good with html) and get all of those kb of html out of your javascript at load time.
🤔
At present, fragments are built up programmatically one at a time. For a template like this...
...we get this:
One way we could potentially improve this is by using
<template>
andcloneNode
to create the entire fragment in one go. This suggestion came from @martypdx, who co-authored Ractive (Svelte's predecessor): https://gist.github.com/martypdx/96b0a0900d2769a982ba36a066c1e38a. Riffing on the ideas in that gist, imagine we had a helper like this......we could then generate this code instead:
This is less code (notwithstanding the cost of the helper), and also very possibly less work (though I haven't benchmarked anything yet).
There's also potential for further benefits in future — by putting markup in
<template>
elements that are on the page (generated during SSR), or in HTML modules if we ever get those. This would mean less JavaScript, and we can take advantage of the browser's parser (which is fast!).A few things to consider:
bind
attributes, used for selecting nodes we need references to, would stick around in the DOM. Not the worst, but not ideal. Also, maybe we need to pick a name that's less likely to conflict with something that gets added to the DOM in future (maybe justsvelte
)