Open onegreyonewhite opened 7 months ago
ms178
commented
7 months ago Could you tell us more details on the used system (e.g. CPU and which Thorium build has been used, the AVX or AVX2 build)?! Benching different versions against each other could be misleading as there are indeed newer features/optimizations in newer builds that might have a performance impact.
Needless to say that also having the same modifications/options enabled in both builds could make a difference.
onegreyonewhite
commented
7 months ago Could you tell us more details on the used system (e.g. CPU and which Thorium build has been used, the AVX or AVX2 build)?! Benching different versions against each other could be misleading as there are indeed newer features/optimizations in newer builds that might have a performance impact.
Needless to say that also having the same modifications/options enabled in both builds could make a difference.
As I write on top of the issue, I used deb packages from the official repo. No manual builds.
Processor Intel i5-10500H. GPU is Nvidia 3060M (laptop). Enabled additionally only GPU's based rendering. Without it, the result is almost the same.
ms178
commented
7 months ago As I write on top of the issue, I used deb packages from the official repo. No manual builds.
As there are several Thorium deb packages from the official repo under https://github.com/Alex313031/thorium/releases/tag/M122.0.6261.132 please specify which one you used. Your CPU should be compatible with the AVX2 build that is available under the given link and should provide the best performance for your system, please test that one if you haven't already.
To make a fair comparison, I'd suggest to benchmark Thorium AVX2 and Google Chrome on the same revision and with the same settings/config options.
onegreyonewhite
commented
7 months ago As I write on top of the issue, I used deb packages from the official repo. No manual builds.
As there are several Thorium deb packages from the official repo under https://github.com/Alex313031/thorium/releases/tag/M122.0.6261.132 please specify which one you used. Your CPU should be compatible with the AVX2 build that is available under the given link and should provide the best performance for your system, please test that one if you haven't already.
Official debian/ubuntu repo.
To make a fair comparison, I'd suggest to benchmark Thorium AVX2 and Google Chrome on the same revision and with the same settings/config options.
What's profit of that? Official debian repos provide different versions. Users don't hold a version of Chrome. Currently provided version is good point to comparison.
ms178
commented
7 months ago Official debian/ubuntu repo.
Then please test the AVX2 version that is provided under the release link I gave you as I cannot say which compiler flags Ubuntu uses to compile their provided Thorium version and if they honor the aggressive flags used by Thorium by default.
What's profit of that? Official debian repos provide different versions. Users don't hold a version of Chrome. Currently provided version is good point to comparison.
As you haven't noticed yourself yet, you are comparing an "unofficial" Ubuntu-provided build of Thorium which is most likely not using all the capabilities of your CPU properly against a newer revision of Chrome. That's not a fair comparison for these following two most important reasons:
1) The official Thorium build uses optimized compiler flags and offers many builds on a per-capability basis to squeeze out more performance of newer CPUs, chances are that Ubuntu uses less optimized compiler flags in their repo by default as they don't ship x86-64-v3 repos yet (which would enable AVX2), This means that you end up with a less performant version using Thorium from the Ubuntu repo. So please just use the official AVX2 Thorium build that is provided from this repo on Github and not the Ubuntu-provided build to show the best case scenario for Thorium on your CPU. I hope you realize that making generalized claims about the lack of Thorium's performance while not using the best official build for your CPU is just dishonest to the Thorium project.
2) Fair benchmarking is an art. As already stated, there might be performance work on the source code level in newer revisions, you'd want to rule out that variable in a fair comparison to assess the qualities of Thorium vs. vanilla Chrome. What you do is an apples to oranges comparison at the moment that has no meaningful value, I am afraid.
onegreyonewhite
commented
7 months ago Official debian/ubuntu repo.
Then please test the AVX2 version that is provided under the release link I gave you as I cannot say which compiler flags Ubuntu uses to compile their provided Thorium version and if they honor the aggressive flags used by Thorium by default.
What's profit of that? Official debian repos provide different versions. Users don't hold a version of Chrome. Currently provided version is good point to comparison.
As you haven't noticed yourself yet, you are comparing an "unofficial" Ubuntu-provided build of Thorium, which is most likely not using all the capabilities of your CPU properly against a newof Chrome.
OFFICIAL Thorium repository. I wrote "official". And official Google repository. That means we have two providers who give almost the same product. And I am trying to compare them.
sudo rm -fv /etc/apt/sources.list.d/thorium.list && \
sudo wget --no-hsts -P /etc/apt/sources.list.d/ \
http://dl.thorium.rocks/debian/dists/stable/thorium.list && \
sudo apt updateAsk @Alex313031, which flags he used in his repo.
onegreyonewhite
commented
7 months ago I test latest thorium release (123) with AVX2 support (got from latest gh release for the integrity of the experiment) and latest chrome stable release (124).
Each browser was run with clear user profile dir (run with --user-data-dir=/tmp/[browser]). That means disabled all flags and used native capabilities.
| Test | Thorium (ms) | Chrome (ms) | Ratio (Thorium/Chrome) |
|---|---|---|---|
| Geomean | 85.49 ± 2.30 (2.7%) | 77.66 ± 1.66 (2.1%) | 1.10 |
| TodoMVC-JavaScript-ES5 | 105.10 ± 12.33 (11.7%) | 87.57 ± 6.14 (7.0%) | 1.20 |
| TodoMVC-JavaScript-ES6-Webpack-Complex-DOM | 144.20 ± 9.15 (6.3%) | 119.06 ± 7.00 (5.9%) | 1.21 |
| TodoMVC-WebComponents | 36.71 ± 1.91 (5.2%) | 35.82 ± 1.49 (4.1%) | 1.02 |
| TodoMVC-React-Complex-DOM | 93.32 ± 12.65 (13.6%) | 80.75 ± 5.63 (7.0%) | 1.16 |
| TodoMVC-React-Redux | 85.80 ± 3.27 (3.8%) | 84.62 ± 3.72 (4.4%) | 1.01 |
| TodoMVC-Backbone | 67.25 ± 4.81 (7.2%) | 58.74 ± 2.79 (4.8%) | 1.14 |
| TodoMVC-Angular-Complex-DOM | 79.02 ± 8.90 (11.3%) | 74.42 ± 5.90 (7.9%) | 1.06 |
| TodoMVC-Vue | 60.82 ± 4.50 (7.4%) | 55.58 ± 5.89 (10.6%) | 1.09 |
| TodoMVC-jQuery | 305.23 ± 14.59 (4.8%) | 216.14 ± 21.59 (10.0%) | 1.41 |
| TodoMVC-Preact-Complex-DOM | 39.19 ± 6.00 (15.3%) | 32.94 ± 5.96 (18.1%) | 1.19 |
| TodoMVC-Svelte-Complex-DOM | 33.75 ± 3.18 (9.4%) | 28.23 ± 2.56 (9.1%) | 1.20 |
| TodoMVC-Lit-Complex-DOM | 43.44 ± 6.77 (15.6%) | 43.31 ± 6.79 (15.7%) | 1.00 |
| NewsSite-Next | 186.57 ± 5.86 (3.1%) | 170.46 ± 7.15 (4.2%) | 1.09 |
| NewsSite-Nuxt | 159.65 ± 4.81 (3.0%) | 145.22 ± 8.34 (5.7%) | 1.10 |
| Editor-CodeMirror | 43.44 ± 5.90 (13.6%) | 45.12 ± 9.46 (21.0%) | 0.96 |
| Editor-TipTap | 147.55 ± 9.30 (6.3%) | 125.66 ± 6.11 (4.9%) | 1.17 |
| Charts-observable-plot | 97.28 ± 7.44 (7.7%) | 90.22 ± 6.78 (7.5%) | 1.08 |
| Charts-chartjs | 103.88 ± 12.47 (12.0%) | 111.20 ± 14.39 (12.9%) | 0.93 |
| React-Stockcharts-SVG | 143.95 ± 3.87 (2.7%) | 135.70 ± 5.45 (4.0%) | 1.06 |
| Perf-Dashboard | 70.06 ± 2.93 (4.2%) | 72.39 ± 12.18 (16.8%) | 0.97 |
Alex313031
commented
7 months ago @onegreyonewhite You should be comparing identical versions. Using M124 for Chrome is unfair.
For example, M123 got a nice speed boost from upstream over M122. So comparing the two would be unfair to the M122 build.
onegreyonewhite
commented
7 months ago @onegreyonewhite You should be comparing identical versions. Using M124 for Chrome is unfair.
For example, M123 got a nice speed boost from upstream over M122. So comparing the two would be unfair to the M122 build.
I don't agree with you, because users uses the latest available version. So now you're behind one version. Users should know current difference in performance.
Anyway, now I hold current chrome version and I'm going to wait 124 version of Thorium for the fairest comparing.
@Alex313031 you should know that I like thorium. I think you make really useful job. I highly respect you for your work. This is an impartial comparison.
onegreyonewhite
commented
6 months ago Hi, everyone!
As you can see from the results below, the official Debian packages for Google Chrome often show much better performance. In situations where Google extensions and services are running, you can occasionally see better results with Thorium for older technologies like ES5. However, anything built with more current JavaScript standards shows significant or slight performance degradation.
I would look into which rendering acceleration flags can be used. For example, enabling GPU rendering and zero-copy significantly speeds up Thorium compared to Chrome (the tests already include these flags, so the difference isn’t as dramatic without "powered anon"). It might be worth running individual tests to pinpoint where the slowdown occurs and see if it's possible to address it.
Additionally, it would be beneficial to focus on more current code porting from Chromium. Yes, this is the most challenging part, but it actually provides the greatest speed boost.
Thorium - 124.0.6367.218 Chrome - 124.0.6367.207 (older but same)
| Test | Thorium (ms) | Chrome (ms) | Ratio (Thorium/Chrome) |
|---|---|---|---|
| Geomean | 90.43 ± 3.53 (3.9%) | 84.55 ± 2.53 (3.0%) | 1.07 |
| TodoMVC-JavaScript-ES5 | 107.47 ± 5.54 (5.2%) | 118.81 ± 13.46 (11.3%) | 0.90 |
| TodoMVC-JavaScript-ES6-Webpack-Complex-DOM | 149.62 ± 4.87 (3.3%) | 143.59 ± 9.05 (6.3%) | 1.04 |
| TodoMVC-WebComponents | 37.60 ± 2.99 (8.0%) | 35.29 ± 5.59 (15.8%) | 1.07 |
| TodoMVC-React-Complex-DOM | 98.33 ± 20.18 (20.5%) | 86.43 ± 6.81 (7.9%) | 1.14 |
| TodoMVC-React-Redux | 88.27 ± 6.68 (7.6%) | 95.53 ± 17.08 (17.9%) | 0.92 |
| TodoMVC-Backbone | 59.78 ± 2.30 (3.9%) | 64.86 ± 6.14 (9.5%) | 0.92 |
| TodoMVC-Angular-Complex-DOM | 94.68 ± 16.17 (17.1%) | 75.27 ± 5.11 (6.8%) | 1.26 |
| TodoMVC-Vue | 58.96 ± 6.95 (11.8%) | 51.35 ± 5.86 (11.4%) | 1.15 |
| TodoMVC-jQuery | 362.98 ± 23.36 (6.4%) | 329.75 ± 21.95 (6.7%) | 1.10 |
| TodoMVC-Preact-Complex-DOM | 37.39 ± 5.27 (14.1%) | 30.38 ± 2.77 (9.1%) | 1.23 |
| TodoMVC-Svelte-Complex-DOM | 37.91 ± 3.65 (9.6%) | 33.23 ± 4.10 (12.3%) | 1.14 |
| TodoMVC-Lit-Complex-DOM | 42.44 ± 2.82 (6.6%) | 39.37 ± 1.23 (3.1%) | 1.08 |
| NewsSite-Next | 214.35 ± 10.65 (5.0%) | 195.58 ± 7.58 (3.9%) | 1.10 |
| NewsSite-Nuxt | 174.61 ± 8.29 (4.7%) | 158.13 ± 5.30 (3.3%) | 1.10 |
| Editor-CodeMirror | 44.34 ± 2.54 (5.7%) | 43.72 ± 7.86 (18.0%) | 1.01 |
| Editor-TipTap | 143.13 ± 15.03 (10.5%) | 137.13 ± 9.98 (7.3%) | 1.04 |
| Charts-observable-plot | 96.70 ± 7.93 (8.2%) | 101.59 ± 7.42 (7.3%) | 0.95 |
| Charts-chartjs | 119.93 ± 13.88 (11.6%) | 109.07 ± 6.42 (5.9%) | 1.10 |
| React-Stockcharts-SVG | 177.51 ± 13.75 (7.7%) | 155.21 ± 11.39 (7.3%) | 1.14 |
| Perf-Dashboard | 80.86 ± 3.46 (4.3%) | 77.05 ± 4.97 (6.4%) | 1.05 |
Actual versions without access to google account and without any extensions (I call it "powered anon") and same versions:
Thorium - 124.0.6367.218 Chrome - 124.0.6367.207 (older but same)
| Test | Thorium (ms) | Chrome (ms) | Ratio (Thorium/Chrome) |
|---|---|---|---|
| Geomean | 76.24 ± 3.20 (4.2%) | 66.73 ± 1.84 (2.8%) | 1.14 |
| TodoMVC-JavaScript-ES5 | 89.20 ± 8.31 (9.3%) | 80.54 ± 7.20 (8.9%) | 1.11 |
| TodoMVC-JavaScript-ES6-Webpack-Complex-DOM | 114.04 ± 7.98 (7.0%) | 102.40 ± 9.87 (9.6%) | 1.11 |
| TodoMVC-WebComponents | 30.80 ± 1.42 (4.6%) | 27.99 ± 0.75 (2.7%) | 1.10 |
| TodoMVC-React-Complex-DOM | 79.85 ± 6.04 (7.6%) | 69.09 ± 4.35 (6.3%) | 1.16 |
| TodoMVC-React-Redux | 82.57 ± 6.25 (7.6%) | 69.95 ± 1.97 (2.8%) | 1.18 |
| TodoMVC-Backbone | 52.89 ± 2.05 (3.9%) | 46.43 ± 1.58 (3.4%) | 1.14 |
| TodoMVC-Angular-Complex-DOM | 74.23 ± 4.19 (5.6%) | 65.90 ± 5.45 (8.3%) | 1.13 |
| TodoMVC-Vue | 48.43 ± 4.02 (8.3%) | 41.94 ± 2.90 (6.9%) | 1.15 |
| TodoMVC-jQuery | 280.94 ± 41.42 (14.7%) | 188.39 ± 11.79 (6.3%) | 1.49 |
| TodoMVC-Preact-Complex-DOM | 30.64 ± 5.29 (17.3%) | 24.66 ± 1.22 (5.0%) | 1.24 |
| TodoMVC-Svelte-Complex-DOM | 28.88 ± 4.90 (17.0%) | 24.38 ± 4.56 (18.7%) | 1.18 |
| TodoMVC-Lit-Complex-DOM | 38.43 ± 2.89 (7.5%) | 36.33 ± 5.60 (15.4%) | 1.06 |
| NewsSite-Next | 189.43 ± 11.96 (6.3%) | 158.49 ± 4.55 (2.9%) | 1.20 |
| NewsSite-Nuxt | 146.84 ± 3.99 (2.7%) | 141.83 ± 7.40 (5.2%) | 1.04 |
| Editor-CodeMirror | 42.74 ± 6.45 (15.1%) | 35.92 ± 5.95 (16.6%) | 1.19 |
| Editor-TipTap | 149.36 ± 47.73 (32.0%) | 115.46 ± 5.59 (4.8%) | 1.29 |
| Charts-observable-plot | 79.13 ± 6.22 (7.9%) | 73.37 ± 5.41 (7.4%) | 1.08 |
| Charts-chartjs | 99.85 ± 10.65 (10.7%) | 98.75 ± 8.19 (8.3%) | 1.01 |
| React-Stockcharts-SVG | 148.92 ± 8.98 (6.0%) | 133.58 ± 7.49 (5.6%) | 1.11 |
| Perf-Dashboard | 65.68 ± 1.83 (2.8%) | 64.20 ± 4.30 (6.7%) | 1.02 |
Current available stable versions on ubuntu with google services and enabled extensions:
Thorium - 124.0.6367.218 Chrome - 125.0.6422.60 (newer)
| Test | Thorium (ms) | Chrome (ms) | Ratio (Thorium/Chrome) |
|---|---|---|---|
| Geomean | 90.43 ± 3.53 (3.9%) | 80.76 ± 1.91 (2.4%) | 1.12 |
| TodoMVC-JavaScript-ES5 | 107.47 ± 5.54 (5.2%) | 113.55 ± 21.31 (18.8%) | 0.95 |
| TodoMVC-JavaScript-ES6-Webpack-Complex-DOM | 149.62 ± 4.87 (3.3%) | 141.97 ± 6.70 (4.7%) | 1.05 |
| TodoMVC-WebComponents | 37.60 ± 2.99 (8.0%) | 32.39 ± 3.82 (11.8%) | 1.16 |
| TodoMVC-React-Complex-DOM | 98.33 ± 20.18 (20.5%) | 79.90 ± 5.95 (7.5%) | 1.23 |
| TodoMVC-React-Redux | 88.27 ± 6.68 (7.6%) | 88.11 ± 9.18 (10.4%) | 1.00 |
| TodoMVC-Backbone | 59.78 ± 2.30 (3.9%) | 59.95 ± 4.62 (7.7%) | 1.00 |
| TodoMVC-Angular-Complex-DOM | 94.68 ± 16.17 (17.1%) | 75.22 ± 5.24 (7.0%) | 1.26 |
| TodoMVC-Vue | 58.96 ± 6.95 (11.8%) | 55.28 ± 7.18 (13.0%) | 1.07 |
| TodoMVC-jQuery | 362.98 ± 23.36 (6.4%) | 332.88 ± 16.17 (4.9%) | 1.09 |
| TodoMVC-Preact-Complex-DOM | 37.39 ± 5.27 (14.1%) | 27.93 ± 4.02 (14.4%) | 1.34 |
| TodoMVC-Svelte-Complex-DOM | 37.91 ± 3.65 (9.6%) | 27.76 ± 1.54 (5.6%) | 1.37 |
| TodoMVC-Lit-Complex-DOM | 42.44 ± 2.82 (6.6%) | 37.73 ± 1.99 (5.3%) | 1.13 |
| NewsSite-Next | 214.35 ± 10.65 (5.0%) | 186.28 ± 8.21 (4.4%) | 1.15 |
| NewsSite-Nuxt | 174.61 ± 8.29 (4.7%) | 155.83 ± 5.08 (3.3%) | 1.12 |
| Editor-CodeMirror | 44.34 ± 2.54 (5.7%) | 51.95 ± 9.18 (17.7%) | 0.85 |
| Editor-TipTap | 143.13 ± 15.03 (10.5%) | 127.89 ± 4.43 (3.5%) | 1.12 |
| Charts-observable-plot | 96.70 ± 7.93 (8.2%) | 91.96 ± 6.53 (7.1%) | 1.05 |
| Charts-chartjs | 119.93 ± 13.88 (11.6%) | 95.35 ± 4.89 (5.1%) | 1.26 |
| React-Stockcharts-SVG | 177.51 ± 13.75 (7.7%) | 152.92 ± 5.74 (3.8%) | 1.16 |
| Perf-Dashboard | 80.86 ± 3.46 (4.3%) | 68.80 ± 2.10 (3.1%) | 1.18 |
Actual versions without access to google account and without any extensions (I call it "powered anon"):
Thorium - 124.0.6367.218 Chrome - 125.0.6422.60 (newer)
| Test | Thorium (ms) | Chrome (ms) | Ratio (Thorium/Chrome) |
|---|---|---|---|
| Geomean | 76.24 ± 3.20 (4.2%) | 62.67 ± 2.41 (3.9%) | 1.22 |
| TodoMVC-JavaScript-ES5 | 89.20 ± 8.31 (9.3%) | 69.97 ± 8.14 (11.6%) | 1.28 |
| TodoMVC-JavaScript-ES6-Webpack-Complex-DOM | 114.04 ± 7.98 (7.0%) | 93.34 ± 5.82 (6.2%) | 1.22 |
| TodoMVC-WebComponents | 30.80 ± 1.42 (4.6%) | 27.25 ± 0.91 (3.3%) | 1.13 |
| TodoMVC-React-Complex-DOM | 79.85 ± 6.04 (7.6%) | 67.29 ± 6.00 (8.9%) | 1.19 |
| TodoMVC-React-Redux | 82.57 ± 6.25 (7.6%) | 67.64 ± 4.38 (6.5%) | 1.22 |
| TodoMVC-Backbone | 52.89 ± 2.05 (3.9%) | 45.09 ± 1.80 (4.0%) | 1.17 |
| TodoMVC-Angular-Complex-DOM | 74.23 ± 4.19 (5.6%) | 63.20 ± 6.25 (9.9%) | 1.17 |
| TodoMVC-Vue | 48.43 ± 4.02 (8.3%) | 38.25 ± 1.56 (4.1%) | 1.27 |
| TodoMVC-jQuery | 280.94 ± 41.42 (14.7%) | 193.06 ± 9.12 (4.7%) | 1.45 |
| TodoMVC-Preact-Complex-DOM | 30.64 ± 5.29 (17.3%) | 28.54 ± 5.61 (19.7%) | 1.07 |
| TodoMVC-Svelte-Complex-DOM | 28.88 ± 4.90 (17.0%) | 20.18 ± 1.97 (9.7%) | 1.43 |
| TodoMVC-Lit-Complex-DOM | 38.43 ± 2.89 (7.5%) | 32.15 ± 0.99 (3.1%) | 1.20 |
| NewsSite-Next | 189.43 ± 11.96 (6.3%) | 158.27 ± 5.49 (3.5%) | 1.20 |
| NewsSite-Nuxt | 146.84 ± 3.99 (2.7%) | 139.56 ± 5.78 (4.1%) | 1.05 |
| Editor-CodeMirror | 42.74 ± 6.45 (15.1%) | 33.51 ± 6.70 (20.0%) | 1.28 |
| Editor-TipTap | 149.36 ± 47.73 (32.0%) | 108.66 ± 7.41 (6.8%) | 1.37 |
| Charts-observable-plot | 79.13 ± 6.22 (7.9%) | 68.72 ± 5.19 (7.6%) | 1.15 |
| Charts-chartjs | 99.85 ± 10.65 (10.7%) | 79.14 ± 12.34 (15.6%) | 1.26 |
| React-Stockcharts-SVG | 148.92 ± 8.98 (6.0%) | 126.21 ± 5.16 (4.1%) | 1.18 |
| Perf-Dashboard | 65.68 ± 1.83 (2.8%) | 56.08 ± 2.01 (3.6%) | 1.17 |
onegreyonewhite
commented
5 months ago Actual packages comparing:
Thorium - 124.0.6367.218 Chrome - 126.0.6478.55 (2 major newer)
| Test | Thorium (ms) | Chrome (ms) | Ratio (Thorium/Chrome) |
|---|---|---|---|
| Geomean | 76.24 ± 3.20 (4.2%) | 61.80 ± 2.35 (3.8%) | 1.23 |
| TodoMVC-JavaScript-ES5 | 89.20 ± 8.31 (9.3%) | 59.88 ± 4.51 (7.5%) | 1.49 |
| TodoMVC-JavaScript-ES6-Webpack-Complex-DOM | 114.04 ± 7.98 (7.0%) | 77.11 ± 3.55 (4.6%) | 1.48 |
| TodoMVC-WebComponents | 30.80 ± 1.42 (4.6%) | 27.56 ± 2.66 (9.6%) | 1.12 |
| TodoMVC-React-Complex-DOM | 79.85 ± 6.04 (7.6%) | 66.94 ± 5.51 (8.2%) | 1.19 |
| TodoMVC-React-Redux | 82.57 ± 6.25 (7.6%) | 77.63 ± 11.00 (14.2%) | 1.06 |
| TodoMVC-Backbone | 52.89 ± 2.05 (3.9%) | 45.70 ± 3.11 (6.8%) | 1.16 |
| TodoMVC-Angular-Complex-DOM | 74.23 ± 4.19 (5.6%) | 63.22 ± 6.09 (9.6%) | 1.17 |
| TodoMVC-Vue | 48.43 ± 4.02 (8.3%) | 38.62 ± 2.40 (6.2%) | 1.25 |
| TodoMVC-jQuery | 280.94 ± 41.42 (14.7%) | 185.15 ± 10.53 (5.7%) | 1.52 |
| TodoMVC-Preact-Complex-DOM | 30.64 ± 5.29 (17.3%) | 24.05 ± 5.10 (21.2%) | 1.27 |
| TodoMVC-Svelte-Complex-DOM | 28.88 ± 4.90 (17.0%) | 19.70 ± 1.43 (7.3%) | 1.47 |
| TodoMVC-Lit-Complex-DOM | 38.43 ± 2.89 (7.5%) | 35.09 ± 4.77 (13.6%) | 1.10 |
| NewsSite-Next | 189.43 ± 11.96 (6.3%) | 163.44 ± 12.51 (7.7%) | 1.16 |
| NewsSite-Nuxt | 146.84 ± 3.99 (2.7%) | 134.99 ± 6.78 (5.0%) | 1.09 |
| Editor-CodeMirror | 42.74 ± 6.45 (15.1%) | 33.97 ± 5.57 (16.4%) | 1.26 |
| Editor-TipTap | 149.36 ± 47.73 (32.0%) | 102.32 ± 6.44 (6.3%) | 1.46 |
| Charts-observable-plot | 79.13 ± 6.22 (7.9%) | 71.35 ± 5.72 (8.0%) | 1.11 |
| Charts-chartjs | 99.85 ± 10.65 (10.7%) | 79.00 ± 10.88 (13.8%) | 1.26 |
| React-Stockcharts-SVG | 148.92 ± 8.98 (6.0%) | 128.97 ± 9.30 (7.2%) | 1.15 |
| Perf-Dashboard | 65.68 ± 1.83 (2.8%) | 59.64 ± 4.40 (7.4%) | 1.10 |
I'm excited to test with new AVX512 profile and compare with similar versions (Thorium 126 vs Chrome 126). I believe it should be interesting. I hope we see it soon 🙏🏼
mindplay-dk
commented
4 months ago My benchmarks on browserbench.org today:
Thorium Chrome Firefox
Speedometer 3.0 23.3 27.0 17.8
MotionMark 1.3 458.89 564.84 464.65
JetStream 2.2 303.063 344.589 222.316Thorium Version 124.0.6367.218 (Official Build) (64-bit) (link)
Chrome Version 126.0.6478.127 (Official Build) (64-bit)
Some system specs:
Edition Windows 11 Pro
Version 23H2
OS build 22631.3880
Processor AMD Ryzen 9 7900 12-Core Processor, 3701 Mhz, 12 Core(s), 24 Logical Processor(s)
Installed Physical Memory (RAM) 32,0 GB
BaseBoard Product MAG B650M MORTAR WIFI (MS-7D76)
Adapter Type Intel(R) Arc(TM) A770 Graphics Family, Intel Corporation compatible
Adapter RAM 2,00 GB (2.147.479.552 bytes)
Driver Version 31.0.101.5592I wonder if my Intel GPU is throwing off the results or something - driver performance with the Intel GPU drivers (in games) often have surprising performance differences vs other more mainstream GPUs.
onegreyonewhite
commented
3 months ago Ubuntu 22.04
Thorium - 126.0.6478.231 Chrome - 126.0.6478.55 (almost same version)
| Test | Thorium (ms) | Chrome (ms) | Ratio (Thorium/Chrome) |
|---|---|---|---|
| Geomean | 68.73 ± 2.12 (3.1%) | 61.80 ± 2.35 (3.8%) | 1.11 |
| TodoMVC-JavaScript-ES5 | 71.75 ± 10.43 (14.5%) | 59.88 ± 4.51 (7.5%) | 1.20 |
| TodoMVC-JavaScript-ES6-Webpack-Complex-DOM | 99.32 ± 9.20 (9.3%) | 77.11 ± 3.55 (4.6%) | 1.29 |
| TodoMVC-WebComponents | 32.95 ± 3.70 (11.2%) | 27.56 ± 2.66 (9.6%) | 1.20 |
| TodoMVC-React-Complex-DOM | 75.92 ± 6.92 (9.1%) | 66.94 ± 5.51 (8.2%) | 1.13 |
| TodoMVC-React-Redux | 74.60 ± 4.01 (5.4%) | 77.63 ± 11.00 (14.2%) | 0.96 |
| TodoMVC-Backbone | 52.16 ± 3.45 (6.6%) | 45.70 ± 3.11 (6.8%) | 1.14 |
| TodoMVC-Angular-Complex-DOM | 66.44 ± 4.09 (6.2%) | 63.22 ± 6.09 (9.6%) | 1.05 |
| TodoMVC-Vue | 43.48 ± 1.74 (4.0%) | 38.62 ± 2.40 (6.2%) | 1.13 |
| TodoMVC-jQuery | 236.90 ± 21.77 (9.2%) | 185.15 ± 10.53 (5.7%) | 1.28 |
| TodoMVC-Preact-Complex-DOM | 28.44 ± 5.39 (19.0%) | 24.05 ± 5.10 (21.2%) | 1.18 |
| TodoMVC-Svelte-Complex-DOM | 25.06 ± 2.53 (10.1%) | 19.70 ± 1.43 (7.3%) | 1.27 |
| TodoMVC-Lit-Complex-DOM | 34.75 ± 3.53 (10.2%) | 35.09 ± 4.77 (13.6%) | 0.99 |
| NewsSite-Next | 167.76 ± 4.28 (2.5%) | 163.44 ± 12.51 (7.7%) | 1.03 |
| NewsSite-Nuxt | 139.91 ± 3.86 (2.8%) | 134.99 ± 6.78 (5.0%) | 1.04 |
| Editor-CodeMirror | 35.31 ± 5.09 (14.4%) | 33.97 ± 5.57 (16.4%) | 1.04 |
| Editor-TipTap | 108.62 ± 8.23 (7.6%) | 102.32 ± 6.44 (6.3%) | 1.06 |
| Charts-observable-plot | 83.42 ± 8.05 (9.6%) | 71.35 ± 5.72 (8.0%) | 1.17 |
| Charts-chartjs | 85.49 ± 12.44 (14.6%) | 79.00 ± 10.88 (13.8%) | 1.08 |
| React-Stockcharts-SVG | 135.46 ± 7.56 (5.6%) | 128.97 ± 9.30 (7.2%) | 1.05 |
| Perf-Dashboard | 61.78 ± 2.65 (4.3%) | 59.64 ± 4.40 (7.4%) | 1.04 |
With latest Chrome:
Thorium - 126.0.6478.231 Chrome - 127.0.6533.99 (one point in major version gap)
| Test | Thorium (ms) | Chrome (ms) | Ratio (Thorium/Chrome) |
|---|---|---|---|
| Geomean | 68.73 ± 2.12 (3.1%) | 63.53 ± 2.45 (3.9%) | 1.08 |
| TodoMVC-JavaScript-ES5 | 71.75 ± 10.43 (14.5%) | 63.89 ± 6.03 (9.4%) | 1.12 |
| TodoMVC-JavaScript-ES6-Webpack-Complex-DOM | 99.32 ± 9.20 (9.3%) | 77.85 ± 3.79 (4.9%) | 1.28 |
| TodoMVC-WebComponents | 32.95 ± 3.70 (11.2%) | 29.95 ± 4.16 (13.9%) | 1.10 |
| TodoMVC-React-Complex-DOM | 75.92 ± 6.92 (9.1%) | 70.82 ± 9.28 (13.1%) | 1.07 |
| TodoMVC-React-Redux | 74.60 ± 4.01 (5.4%) | 63.76 ± 2.43 (3.8%) | 1.17 |
| TodoMVC-Backbone | 52.16 ± 3.45 (6.6%) | 46.38 ± 2.15 (4.6%) | 1.12 |
| TodoMVC-Angular-Complex-DOM | 66.44 ± 4.09 (6.2%) | 61.37 ± 7.34 (12.0%) | 1.08 |
| TodoMVC-Vue | 43.48 ± 1.74 (4.0%) | 40.72 ± 3.00 (7.4%) | 1.07 |
| TodoMVC-jQuery | 236.90 ± 21.77 (9.2%) | 181.64 ± 8.99 (5.0%) | 1.30 |
| TodoMVC-Preact-Complex-DOM | 28.44 ± 5.39 (19.0%) | 26.69 ± 4.64 (17.4%) | 1.07 |
| TodoMVC-Svelte-Complex-DOM | 25.06 ± 2.53 (10.1%) | 23.18 ± 3.92 (16.9%) | 1.08 |
| TodoMVC-Lit-Complex-DOM | 34.75 ± 3.53 (10.2%) | 34.48 ± 2.14 (6.2%) | 1.01 |
| NewsSite-Next | 167.76 ± 4.28 (2.5%) | 168.15 ± 16.63 (9.9%) | 1.00 |
| NewsSite-Nuxt | 139.91 ± 3.86 (2.8%) | 144.36 ± 10.83 (7.5%) | 0.97 |
| Editor-CodeMirror | 35.31 ± 5.09 (14.4%) | 35.75 ± 5.45 (15.2%) | 0.99 |
| Editor-TipTap | 108.62 ± 8.23 (7.6%) | 101.71 ± 5.64 (5.5%) | 1.07 |
| Charts-observable-plot | 83.42 ± 8.05 (9.6%) | 73.81 ± 5.82 (7.9%) | 1.13 |
| Charts-chartjs | 85.49 ± 12.44 (14.6%) | 84.46 ± 11.11 (13.2%) | 1.01 |
| React-Stockcharts-SVG | 135.46 ± 7.56 (5.6%) | 132.16 ± 6.24 (4.7%) | 1.02 |
| Perf-Dashboard | 61.78 ± 2.65 (4.3%) | 59.86 ± 3.93 (6.6%) | 1.03 |
The results are starting to improve. I've noticed a trend that the TodoMVC-React-Redux test has quite a large variance in results over multiple runs. This test might be mentally excluded because it seems highly dependent on external factors. The other results are generally more stable.
Therefore, I'm observing that, overall, for Ubuntu 22.04, there isn't a significant difference between browsers. Yes, Google Chrome often shows better results, but they aren't significantly impactful for most users.
I believe the project's advantages don't lie in speed. Instead, they are in supporting different hardware, older operating systems, useful technologies, and disabling some unnecessary tracking features. Some time ago, there was a performance advantage (about a year ago), but over time, Google has caught up with all the necessary patches.
Overall, it's an impressive result for a solo development effort!
mindplay-dk
commented
3 months ago Sadly, I'm not sure any of it matters anymore.
All of the Chromium based browsers will lose Manifest v2 support, which is most browser extensions, and almost everything related to privacy and ad blocking. Google just killed off JPEG XL for no reason.
The way things are going, the only viable browsers will be Firefox or Firefox-based browsers, fast or not. ☹️
onegreyonewhite
commented
3 months ago That's not entirely accurate. The browser that will win is the one with the most community support. The more people who have a particular browser on their devices, the more developers will be interested in supporting the project.
However, to attract users to a specific browser, a killer feature is needed. This will help draw attention from the public and the developer community.
So far, the killer feature is the support for MV2 and a certain set of patches. I had hoped performance would be the breakthrough, but as it turns out, it's within the margin of error.
I'm sure there's still room to optimize by using compilation flags to make more efficient use of memory.
On my part, I plan to try publishing information about the releases on interested platforms to draw some attention to it.
It would be great if someone joined the development efforts to help refine the browser and speed up updates to newer releases (currently, there's a stable lag of one major release).
mindplay-dk
commented
3 months ago That's not entirely accurate.
I know, I'm just feeling very negative - I had just weeks earlier gotten the notification that more than half of my Chrome extensions will be banned and removed from the store. That was the final drop for me, so I spent a week testing every Chromium based browser available, and finally decided on Vivaldi. Then I discovered a post on the Vivaldi forum where they confirmed they won't be able to keep Manifest v2 support either.
There is a huge ecosystem of useful extensions, most of which are still Manifest V2 - either because the developer doesn't want to spend a lot of time finding new ways to make their extension work with V3, or because they can't. Developers have railed on V3 and Google's response has been very minor symbolic concessions, mostly to appease. Only a few projects, such as UBlock, have had the persistence to even attempt to create something that works with V3, and with severe trade offs.
Sorry, maybe this is not the place to rail about this. 😅
I just wish there was a reasonably fast browser I could switch to - but it looks like, basically anything Firefox-based is inherently slow. Probably not slow enough that you would notice a difference for daily browsing, but in larger apps like Figma or Photopea you definitely notice the difference.
So far, the killer feature is the support for MV2 and a certain set of patches.
I think we could probably draw attention to the fact that Google is about to cut the entire ecosystem of extensions in half?
Either way, I will probably end up running two browsers - one Chromium based for larger apps, and one Firefox based for everything else. 😕
I had hoped performance would be the breakthrough, but as it turns out, it's within the margin of error.
What do you mean?
The performance difference between Chromium and Firefox based browsers is pretty substantial, isn't it?
onegreyonewhite
commented
3 months ago @mindplay-dk
What do you mean?
The performance difference between Chromium and Firefox based browsers is pretty substantial, isn't it?
I mean that Chrome and Chromium have the same performance as Thorium.
About extensions. It would be great if Thorium create it's own marketplace with MV2 support. But I suppose that isn't so easy and requires some finances.
mindplay-dk
commented
3 months ago About extensions. It would be great if Thorium create it's own marketplace with MV2 support.
wait, Thorium is going to keep MV2 support?
I thought all Chromium based browsers were going to lose this?
onegreyonewhite
commented
3 months ago About extensions. It would be great if Thorium create it's own marketplace with MV2 support.
wait, Thorium is going to keep MV2 support?
I thought all Chromium based browsers were going to lose this?
Yep, long how it possible.
Here text from 60th release:
Manifest V2 support force enabled (Starting in M128 they are experimenting with disabling MV2). It will be completely removed in M136 (10 months from now), and when they finally do remove the actual code for loading MV2 extensions, it will be restored, because F**k Google! Even if it takes a crapload of work, I am determined to restore it, because without UBlock Origin working properly in Thorium, I wouldn't even want to use my own browser! If you want to use other Chromium based browsers, you will eventually be out of luck, and will either need to use Firefox, or find another Chromium fork that has MV2 support when the time comes.
mindplay-dk
commented
2 months ago I wish the Chromium-based browsers would band together and maintain a Chromium-fork. They could preserve MV2 and potentially add or reject other features (e.g. advertising and tracking) in the interest of the public good.
It would make sense, wouldn't it?
I'm pretty sure the only party actually interested in removing MV2 are Google. 🤔
mindplay-dk
commented
2 months ago (oh, and they could bring back JPEG-XL while they're at it!)
Alex313031
commented
2 months ago @mindplay-dk Thorium has JPEG-XL if you didn't know. I restored it after they removed it. Similarly, I will be restoring MV2 when the time comes, but users will have to manually install them, because I don't have the know-how, nor the server infrastructure to maintain a separate "Store" for MV2 extensions.
And yeah @onegreyonewhite Those are disappointing to me. I will test non-optimized Thorium, versus optimized Thorium, versus vanilla Chromium, versus vanilla Google Chrome, all at the same revision, to get a better idea of the overall picture.
One thing I suspect is that I have seen code references to AFDO (Automatic Feedback Directed Optimization), which seems to only be available internally, so Chrome might have a proprietary "trick up its sleeve" to make it faster than any other Chromium based browser. However, pending the above testing, I will see what flags I can tweak and if @robrich999 (the compiler flag master lolol), has anything to say about it. Because yeah, one of the major selling points is to maximize performance over other Chromium based browsers, so if it isn't living up to that, then there is a problem.
mindplay-dk
commented
2 months ago @mindplay-dk Thorium has JPEG-XL if you didn't know. I restored it after they removed it. Similarly, I will be restoring MV2 when the time comes
😀👍
but users will have to manually install them, because I don't have the know-how, nor the server infrastructure to maintain a separate "Store" for MV2 extensions.
could we make them installable from addons.mozilla.org?
Firefox do not have plans to remove MV2 extensions, so this will be the largest remaining catalog of MV2 extensions.
gz83
commented
2 months ago Thorium will still try to keep MV2 support, probably until the third-party community runs out of ways to keep MV2 support.
@mindplay-dk
Alex313031
commented
2 months ago @mindplay-dk @gz83 And beyond, by keeping a download directory on the website of popular MV2 extensions, and re-adding MV2 code when they finally remove it from Chromium.
Recently, I noticed that official Chrome seemed to render pages a little faster. I decided to take some measurements.
Thorium 122.0.6261.132 (from official repo) Google Chrome 124.0.6367.60 (from official repo)
Ubuntu 22.04 LTS with all updates.
https://browserbench.org/Speedometer3.0/
Somehow it doesn't look very good. Are these two major versions really packed with so many optimizations?