Performance Benchmarks for WordPress 6.7

mukeshpanchal27 commented 1 month ago

As we enter the beta phase for WordPress 6.7, I am opening this issue to gather and track all performance benchmarks. This will help us collaborate more effectively, identify potential regressions, and ensure all performance-related data is centralized in one place.

Feel free to contribute with your benchmarks, results, or any findings related to performance in WordPress 6.7.

mukeshpanchal27 commented 1 month ago

WP 6.7 Beta 1 Performance Benchmark Report:

I ran benchmarks comparing WordPress 6.7 Beta 1 and the latest released 6.6 branch (6.6.2). I took measurements with the twentytwentyone (TT1) theme using en_US locale, both with object caching disabled and enabled, then again with twentytwentyfour (TT4), This version of WordPress shows a regression in server rendering time from the previous release for TT4.

With object caching disabled, Classic themes (TT1) show a ~3% regression (0.35ms slower) in wp-before-template measurements; LCP remains relatively stable. Block themes (TT4) show a ~6% regression in TTFB (6ms slower) and a ~8% regression (8 ms slower) for wp-total measurements. LCP shows a ~4.5% regression (7ms). Running the tests with object caching enabled shows similar regressions.

Here are the results of the test runs:

Caching Disabled: Classic theme - https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11122906286 Block theme - https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11123131063

Caching Enabled: Classic theme - https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11123295178 Block theme - https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11123530293

cc. @felixarntz @joemcgill @adamsilverstein @westonruter @swissspidy

joemcgill commented 1 month ago

Thanks @mukeshpanchal27. Repeating what I said in our weekly performance team chat earlier today:

the performance metrics for classic themes looks pretty steady, which is great! For TT4, it seems like we need to investigate the cause of the additional performance regression during template rendering, because adding 10% is not great.

I started trying to investigate the likely cause by looking at the WordPress Code Vitals dashboard and realized that the last commit recorded to that dashboard seems to be this one from five months ago 😱. It's likely that the following commit, r58165, likely broke these.

Fortunately, the Gutenberg tests are still being recorded, and it looks like this commit is worth investigating further. It was synced to WordPress in r58936. Here's the performance run from that commit in WP-dev.

joemcgill commented 1 month ago

I've opened https://core.trac.wordpress.org/ticket/62153 to fix the dashboard issue.

mukeshpanchal27 commented 1 month ago

I did the quick compare through xhprof. Here is some finding.

Use of strpos is significantly more then the previous version.

The compression between 6.6.2 and 6.7 beta1

6.7 beta1 `strpos` function details:

6.6.2 `strpos` function details:

mukeshpanchal27 commented 1 month ago

Fortunately, the Gutenberg tests are still being recorded, and it looks like this commit is worth investigating further. It was synced to WordPress in r58936. Here's the performance run from that commit in WP-dev.

@joemcgill The code https://github.com/WordPress/wordpress-develop/blob/trunk/src/wp-includes/class-wp-theme-json.php#L3239-L3251 is too expensive see the profiling result. I removed the code and did the comparison.

Server-timing

Mertic	Beta 1	Beta 1 with remove code	Diff %	Diff abs.
Response Time	252.84	247.12	-2.26%	-5.72
wp-before-template	76.26	76.28	0.03%	0.02
wp-before-template-db-queries	3.3	3.35	1.52%	0.05
wp-template	173.42	167.66	-3.32%	-5.76
wp-total	249.87	244.23	-2.26%	-5.64
wp-template-db-queries	2.6	2.59	-0.38%	-0.01

🔴 It clear shows that the code introduce ~3% regression

mukeshpanchal27 commented 1 month ago

JFYI, I re-open https://core.trac.wordpress.org/ticket/61858 so editor team also can check it.

mukeshpanchal27 commented 1 month ago

Opened PR https://github.com/WordPress/wordpress-develop/pull/7486 that solve the regression. Waiting for the feedback.

felixarntz commented 1 month ago

Since we now reinstated the Code Vitals dashboard, I just took a look at it and noticed https://www.codevitals.run/project/wordpress/home-block-theme-lcpMinusTtfb shows a notable regression that must have happened at some point while it was not working.

It's interesting that our Web Vitals benchmarks for "LCP - TTFB" didn't show a similar regression. Since the Code Vitals dashboard is still using the TT3 theme (which we should probably update at some point), it would be helpful to run another Web Vitals benchmark comparison using TT3, to check whether we see the regression there too - in which case the problem might be something that mostly affects TT3 for some reason.

mukeshpanchal27 commented 1 month ago

I ran performance benchmarks for Beta 1 with TT3 theme: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11228728635

Web Vitals benchmarks

Mertic	WP 6.6.2	Beta 1	Diff %	Diff abs.
FCP	99.7	99.15	-0.55%	-0.55
LCP	99.7	99.15	-0.55%	-0.55
TTFB	54.8	56	2.19%	1.20
LCP-TTFB	44.6	42.8	-4.04%	-1.80

The benchmarks didn't show the regression that we see in dashboard.

mukeshpanchal27 commented 1 month ago

WP 6.7 Beta 2 Performance Benchmark Report:

With object caching disabled,

Classic themes (TT1) web vitals and server-timing remains relatively stable.
Block themes (TT3) show slight regression but remains relatively stable.
Block themes (TT4) show a ~6% regression in LCP shows a ~4.5% regression (7ms), TTFB shows a ~8% regression (6ms), wp-template shows a ~10% regression (6ms) and wp-total shows a ~10% regression (6ms)

Here are the results of the test runs:

TT1: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11228940062 TT3: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11229064497 TT4: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11229259874

TT1 (twentytwentyone) benchmarks

Web Vitals

Metric	WP 6.6.2	WP 6.7 Beta 2	Diff %	Diff abs.
FCP	102	102.65	0.64%	0.65
LCP	102	102.65	0.64%	0.65
TTFB	47.4	47.25	-0.32%	-0.15
LCP-TTFB	54.8	55.2	0.73%	0.40

Server-Timing

Metric	WP 6.6.2	WP 6.7 Beta 2	Diff %	Diff abs.
Response Time	39.8	39.71	-0.23%	-0.09
wp-before-template	11.47	11.61	1.22%	0.14
wp-template	27.33	27.21	-0.44%	-0.12
wp-total	38.93	38.86	-0.18%	-0.07

TT3 (twentytwentythree) benchmarks

Web Vitals

Metric	WP 6.6.2	WP 6.7 Beta 2	Diff %	Diff abs.
FCP	99.35	99.3	-0.05%	-0.05
LCP	99.35	99.3	-0.05%	-0.05
TTFB	55	55.95	1.73%	0.95
LCP-TTFB	43.35	42.55	-1.85%	-0.80

Server-Timing

Metric	WP 6.6.2	WP 6.7 Beta 2	Diff %	Diff abs.
Response Time	47.64	47.9	0.55%	0.26
wp-before-template	15.89	15.77	-0.76%	-0.12
wp-template	30.76	31.15	1.27%	0.39
wp-total	46.63	46.89	0.56%	0.26

TT4 (twentytwentyfour) benchmarks

Web Vitals

Metric	WP 6.6.2	WP 6.7 Beta 2	Diff %	Diff abs.
FCP	155.55	162.75	4.63%	7.20
LCP	155.55	162.75	4.63%	7.20
TTFB	81.55	88.35	8.34%	6.80
LCP-TTFB	73.8	73.7	-0.14%	-0.10

Server-Timing

Metric	WP 6.6.2	WP 6.7 Beta 2	Diff %	Diff abs.
Response Time	74.11	80.16	8.16%	6.05
wp-before-template	16.31	16.38	0.43%	0.07
wp-template	56.7	62.62	10.44%	5.92
wp-total	72.95	79.01	8.31%	6.06

🔴 alert for TT4, In https://github.com/WordPress/wordpress-develop/pull/7486 we are checking one approach to solve the background image regression that account for 3-4% regression in block theme.

joemcgill commented 1 month ago

Following up on https://github.com/WordPress/performance/issues/1572#issuecomment-2397340481, I noticed while testing a fix to the Performance Workflow that the LCP-TTFB metric has returned to a more "normal" range but doesn't seem related to any specific code change, so I'm thinking there was a temporary problem with the workflow that has cleared itself up.

mukeshpanchal27 commented 1 month ago

It means we should debug as the difference it ~6ms for normal code commit.

mukeshpanchal27 commented 1 month ago

WP 6.7 Beta 3 Performance Benchmark Report:

With object caching disabled,

Classic themes (TT1) web vitals and server-timing remains relatively stable.
Block themes (TT3) show slight regression but remains relatively stable.
Block themes (TT4)
- LCP shows a ~3% regression (~5ms)
- TTFB shows a ~4% regression (~4ms)
- wp-template shows a ~9% regression (5ms)
- wp-total shows a ~7% regression (5ms)

Here are the results of the test runs:

TT1: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11339147727 TT3: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11339238140 TT4: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11339330952

TT1 (twentytwentyone) benchmarks

Web Vitals

Metric	WP 6.6.2	WP 6.7 Beta 3	Diff %	Diff abs.
FCP	100.5	99.7	-0.80%	-0.80
LCP	100.5	99.7	-0.80%	-0.80
TTFB	45.7	45.7	0.00%	0.00
LCP-TTFB	54.4	53.65	-1.38%	-0.75

Server-Timing

Metric	WP 6.6.2	WP 6.7 Beta 3	Diff %	Diff abs.
Response Time	37.87	38.07	0.53%	0.20
wp-before-template	10.43	10.74	2.97%	0.31
wp-template	26.66	26.52	-0.53%	-0.14
wp-total	37.14	37.31	0.46%	0.17

TT3 (twentytwentythree) benchmarks

Web Vitals

Metric	WP 6.6.2	WP 6.7 Beta 3	Diff %	Diff abs.
FCP	97.4	97.45	0.05%	0.05
LCP	97.4	97.45	0.05%	0.05
TTFB	53.95	54.5	1.02%	0.55
LCP-TTFB	42.9	42.75	-0.35%	-0.15

Server-Timing

Metric	WP 6.6.2	WP 6.7 Beta 3	Diff %	Diff abs.
Response Time	47.29	47.74	0.95%	0.45
wp-before-template	15.88	15.95	0.44%	0.07
wp-template	30.3	30.56	0.86%	0.26
wp-total	46.19	46.59	0.87%	0.40

TT4 (twentytwentyfour) benchmarks

Web Vitals

Metric	WP 6.6.2	WP 6.7 Beta 3	Diff %	Diff abs.
FCP	157.2	162.05	3.09%	4.85
LCP	157.2	162.05	3.09%	4.85
TTFB	81.3	85.6	5.29%	4.30
LCP-TTFB	75.45	75.3	-0.20%	-0.15

Server-Timing

Metric	WP 6.6.2	WP 6.7 Beta 3	Diff %	Diff abs.
Response Time	71.41	76.4	6.99%	4.99
wp-before-template	14.99	15.02	0.20%	0.03
wp-template	55.5	60.38	8.79%	4.88
wp-total	70.5	75.44	7.01%	4.94

Beta 3 shows improvement compared to Beta 2, but there is still a regression in Web Vitals and Server-Timing.

mukeshpanchal27 commented 1 month ago

Just for visibility, I found another background image related PR https://github.com/WordPress/wordpress-develop/pull/6836#issuecomment-2412994228 that is responsible for regression.

ramonjd commented 1 month ago

Just for visibility, I found another background image related PR https://github.com/WordPress/wordpress-develop/pull/6836#issuecomment-2412994228 that is responsible for regression.

Following the set up in the docs, I'm having trouble replicating the figures you quoted here locally.

Comparing with and then without those changes, my benchmark stats are unmoved or are neglible in TT5, TT4 and TT3.

I know I'm probably doing something wrong though.

E.g., TT3,

Metric	Percentile	WP trunk	WP trunk without the changes in #6836	Diff %
wp-before-template	p10	16.58	17.21	3.80%
wp-before-template	p25	16.78	17.38	3.58%
wp-before-template	p50 (median)	17.28	17.79	2.95%
wp-before-template	p75	18.04	18.21	0.94%
wp-before-template	p90	18.47	18.78	1.68%
wp-template	p10	12.71	12.79	0.63%
wp-template	p25	12.8	12.84	0.31%
wp-template	p50 (median)	12.88	12.95	0.54%
wp-template	p75	13.01	13.14	1.00%
wp-template	p90	13.41	13.7	2.16%
wp-total	p10	29.32	30.03	2.42%
wp-total	p25	29.66	30.32	2.23%
wp-total	p50 (median)	30.2	30.87	2.22%
wp-total	p75	31	31.22	0.71%
wp-total	p90	31.81	32.6	2.48%

That the "without change" measurement is higher for me I cannot explain, but I ran it multiple times and it swings +- by 0.5 on each side.

mukeshpanchal27 commented 1 month ago

Again ran the benchmark for TT4

Server-Timing benchmarks

Mertic	Beta 3	WP trunk without the changes in #6836	Diff %	Diff abs.
Response Time (p10)	65.43	66.71	1.96%	1.28
Response Time (p25)	65.72	67.1	2.10%	1.38
Response Time (p50)	66.6	68.14	2.31%	1.54
Response Time (p75)	67.65	69.45	2.66%	1.80
Response Time (p90)	69.65	72.08	3.49%	2.43
wp-before-template (p10)	29.91	30.51	2.01%	0.60
wp-before-template (p25)	30.19	30.77	1.92%	0.58
wp-before-template (p50)	30.48	31.18	2.30%	0.70
wp-before-template (p75)	30.82	31.89	3.47%	1.07
wp-before-template (p90)	31.75	32.51	2.39%	0.76
wp-template (p10)	33.38	34.07	2.07%	0.69
wp-template (p25)	33.56	34.29	2.18%	0.73
wp-template (p50)	33.93	34.69	2.24%	0.76
wp-template (p75)	34.84	35.68	2.41%	0.84
wp-template (p90)	35.42	37.51	5.90%	2.09
wp-total (p10)	63.5	64.74	1.95%	1.24
wp-total (p25)	63.81	65.14	2.08%	1.33
wp-total (p50)	64.72	66.08	2.10%	1.36
wp-total (p75)	65.52	67.27	2.67%	1.75
wp-total (p90)	67.34	69.87	3.76%	2.53

ramonjd commented 1 month ago

Again ran the benchmark for TT4

Thanks for running it again. I think I still need some clarification on how to interpret these results.

Based on the tables, the server load time increases when the changeset in question is reverted.

Unless a higher number is better 😅 ? My assumption is that a lower server load time number is preferable.

Anyway, I'll keep testing and report back if I find something significant.

It's probably worth looking into caching the output of the method as well.

Thank you for your help!

ramonjd commented 1 month ago

I've run tests both locally and on CI and cannot find any conclusive evidence that https://github.com/WordPress/wordpress-develop/pull/6836 causes a statistically significant regression in performance.

In fact, I'm more confounded than anything. Some reports suggest that trunk is faster than the reverted branch, other times the needle falls in the other direction by between 1-4ms.

Example from a complete revert on CI ("After" is the revert):

Locally the benchmark tests reveal nothing significant for me as mentioned above. I've been running all variations for a couple of hours and am yet to find any reliable results.

Furthermore I added some relative paths to TT4 and tried caching the results of WP_Theme_JSON_Resolver::resolve_theme_file_uris() and WP_Theme_JSON_Resolver::get_resolved_theme_uris(), both of which yielded inconclusive results.

All that said, I do believe we need a caching strategy for these resolved URIs however.

I think perhaps caching them in WP_Theme_JSON_Resolver as is done with theme_json might work, using get_stylesheet() or get_stylesheet_directory() as a key. I'll add it to my list of things to look into.

Edit: This will improve things in general perhaps:

https://github.com/WordPress/wordpress-develop/pull/6879

aaronrobertshaw commented 1 month ago

I've run tests both locally and on CI and cannot find any conclusive evidence that https://github.com/WordPress/wordpress-develop/pull/6836 causes a statistically significant regression in performance.

I've also followed the steps outlined in the docs, using a completely fresh site locally for testing. Every test run appears to yield inconsistent results for me.

This was the latest set of results for me, running a batch of 10000 requests, with PR being a branch reverting the changes in question:

In fact, I'm more confounded than anything.

💯 in the same boat.

I've been running all variations for a couple of hours and am yet to find any reliable results.

+1

mukeshpanchal27 commented 1 month ago

Have you both checked for TT4?

ramonjd commented 1 month ago

Have you both checked for TT4?

Yes.

joemcgill commented 1 month ago

Just committed r59256, which shows pretty good improvements for TT4

ramonjd commented 1 month ago

@joemcgill does r59256 require backporting to Gutenberg? Happy to take care of it if you don't have time.

aaronrobertshaw commented 1 month ago

does r59256 require backporting to Gutenberg?

I believe it does. I was also planning to help out there if needed 👍

Core's action calling wp_enqueue_global_styles is replaced in Gutenberg with its version. Which ultimately means gutenberg_add_global_styles_for_blocks is used instead of wp_add_global_styles_for_blocks.

ramonjd commented 1 month ago

I think perhaps caching them in WP_Theme_JSON_Resolver as is done with theme_json might work, using get_stylesheet() or get_stylesheet_directory() as a key.

Took a naive stab at this: https://github.com/WordPress/gutenberg/pull/66155

mukeshpanchal27 commented 1 month ago

WP 6.7 RC 1 Performance Benchmark Report:

With object caching disabled,

Classic themes (TT1) web vitals and server-timing (~2% improvement) remains relatively stable.
Block themes (TT3)
- LCP shows a ~3% improvement (~3ms)
- TTFB shows a ~2% improvement (~1ms)
- LCP-TTFB shows a ~5% improvement (~2ms)
- wp-template shows a ~4% improvement (1ms)
- wp-total shows a ~2% improvement (1ms)
Block themes (TT4)
- LCP shows a ~5% regression (~8ms)
- TTFB shows a ~8% regression (~6ms)
- wp-template shows a ~10% regression (6ms)
- wp-total shows a ~8% regression (6ms)

Here are the results of the test runs:

TT1: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11472667306 TT3: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11472757880 TT4: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11477476622

TT1 (twentytwentyone) benchmarks

Web Vitals

Metric	WP 6.6.2	WP 6.7 RC 1	Diff %	Diff abs.
FCP	100.1	99.6	-0.50%	-0.50
LCP	100.1	99.6	-0.50%	-0.50
TTFB	46.1	45.55	-1.19%	-0.55
LCP-TTFB	53.8	53.95	0.28%	0.15

Server-Timing

Metric	WP 6.6.2	WP 6.7 RC 1	Diff %	Diff abs.
Response Time	39.54	38.7	-2.12%	-0.84
wp-before-template	11.35	11.26	-0.79%	-0.09
wp-template	27.19	26.48	-2.61%	-0.71
wp-total	38.61	37.77	-2.18%	-0.84

TT3 (twentytwentythree) benchmarks

Web Vitals

Metric	WP 6.6.2	WP 6.7 RC 1	Diff %	Diff abs.
FCP	97	93.4	-3.71%	-3.60
LCP	97	93.4	-3.71%	-3.60
TTFB	53.85	52.65	-2.23%	-1.20
LCP-TTFB	42.3	40.2	-4.96%	-2.10

Server-Timing

Metric	WP 6.6.2	WP 6.7 RC 1	Diff %	Diff abs.
Response Time	46.53	45.26	-2.73%	-1.27
wp-before-template	15.4	15.4	0.00%	0.00
wp-template	30.14	28.82	-4.38%	-1.32
wp-total	45.59	44.32	-2.79%	-1.27

TT4 (twentytwentyfour) benchmarks

Web Vitals

Metric	WP 6.6.2	WP 6.7 RC 1	Diff %	Diff abs.
FCP	155.9	164.15	5.29%	8.25
LCP	155.9	164.25	5.36%	8.35
TTFB	81.25	87.8	8.06%	6.55
LCP-TTFB	73.65	76.25	3.53%	2.60

Server-Timing

Metric	WP 6.6.2	WP 6.7 RC 1	Diff %	Diff abs.
Response Time	73.54	79.48	8.08%	5.94
wp-before-template	16.2	16.28	0.49%	0.08
wp-template	55.95	61.95	10.72%	6.00
wp-total	72.28	78.24	8.25%	5.96

mukeshpanchal27 commented 3 weeks ago

WP 6.7 RC 2 Performance Benchmark Report:

With object caching disabled,

Classic themes (TT1) web vitals and server-timing (~2% improvement) remains relatively stable.
Block themes (TT3)
- LCP shows a ~1% improvement (~1ms)
- TTFB shows a ~1% improvement (~1ms)
- LCP-TTFB shows a ~1% improvement (~0.5ms)
- wp-template shows a ~4% improvement (~1ms)
- wp-total shows a ~2% improvement (~1ms)
Block themes (TT4)
- LCP shows a ~1% regression (~1.5ms)
- TTFB shows a ~1.5% regression (~1ms)
- wp-template shows a ~1% regression (0.5ms)
- wp-total shows a ~0.5% regression (~0.5ms)

Here are the results of the test runs:

TT1: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11579114106 TT3: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11579298956 TT4: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11579469822

TT1 (twentytwentyone) benchmarks

Web Vitals

Metric	Before	After	Diff %	Diff abs.
FCP	100.2	99.65	-0.55%	-0.55
LCP	100.2	99.65	-0.55%	-0.55
TTFB	46.35	45.8	-1.19%	-0.55
LCP-TTFB	53.75	54	0.47%	0.25

Server-Timing

Metric	Before	After	Diff %	Diff abs.
Response Time	40.55	40.16	-0.96%	-0.39
wp-before-template	11.86	12.04	1.52%	0.18
wp-template	27.63	27.07	-2.03%	-0.56
wp-total	39.53	39.08	-1.14%	-0.45

TT3 (twentytwentythree) benchmarks

Web Vitals

Metric	Before	After	Diff %	Diff abs.
FCP	98.6	96.9	-1.72%	-1.70
LCP	98.6	96.9	-1.72%	-1.70
TTFB	55	54.2	-1.45%	-0.80
LCP-TTFB	42.85	42.4	-1.05%	-0.45

Server-Timing

Metric	Before	After	Diff %	Diff abs.
Response Time	48.23	46.9	-2.76%	-1.33
wp-before-template	16.25	16.22	-0.18%	-0.03
wp-template	30.82	29.6	-3.96%	-1.22
wp-total	47.1	45.82	-2.72%	-1.28

TT4 (twentytwentyfour) benchmarks

Web Vitals

Metric	Before	After	Diff %	Diff abs.
FCP	155.95	157.5	0.99%	1.55
LCP	155.95	157.5	0.99%	1.55
TTFB	80.85	82.1	1.55%	1.25
LCP-TTFB	74.5	74.3	-0.27%	-0.20

Server-Timing

Metric	Before	After	Diff %	Diff abs.
Response Time	72.95	73.39	0.60%	0.44
wp-before-template	15.79	15.78	-0.06%	-0.01
wp-template	55.94	56.51	1.02%	0.57
wp-total	71.82	72.28	0.64%	0.46

RC 2 shows improvement compared to RC 1 in block theme.

mukeshpanchal27 commented 2 weeks ago

I ran local benchmarks for TT4 on versions 6.6.2 and 6.7 RC2 and saw similar results. Core didn’t introduce any performance-related changes between RC1 and RC2, though the Editor packages were updated in the meantime.

Interestingly, the Code Vitals analysis didn’t show any improvement 🤔

mukeshpanchal27 commented 2 weeks ago

WP 6.7 RC3 Performance Benchmark Report:

With object caching disabled,

Classic themes (TT1)
- TTFB shows a ~2% improvement (~1ms)
- wp-template shows a ~2% improvement (~0.5ms)
Block themes (TT3)
- LCP shows a ~3% improvement (~3ms)
- TTFB shows a ~2% improvement (~1ms)
- LCP-TTFB shows a ~3.5% improvement (~1.5ms)
- wp-before-template shows a ~2% regression (~0.3ms)
- wp-template shows a ~3% improvement (~1ms)
- wp-total shows a ~1.5% improvement (~0.7ms)
Block themes (TT4)
- LCP shows a ~2.5% regression (~4ms)
- TTFB shows a ~2.7% regression (~2ms)
- LCP-TTFB shows a ~1.3% regression (~1ms)
- wp-template shows a ~3% regression (1.8ms)
- wp-total shows a ~2.7% regression (~2ms)

Here are the results of the test runs:

TT1: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11696995231 TT3: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11697098693 TT4: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11697201653

TT1 (twentytwentyone) benchmarks

Web Vitals

Metric	Before	After	Diff %	Diff abs.
FCP	100.6	100.55	-0.05%	-0.05
LCP	100.6	100.55	-0.05%	-0.05
TTFB	46.4	45.3	-2.37%	-1.10
LCP-TTFB	53.9	54.75	1.58%	0.85

Server-Timing

Metric	Before	After	Diff %	Diff abs.
Response Time	38.17	37.85	-0.84%	-0.32
wp-before-template	10.7	10.91	1.96%	0.21
wp-template	26.68	26.19	-1.84%	-0.49
wp-total	37.42	37.1	-0.86%	-0.32

TT3 (twentytwentythree) benchmarks

Web Vitals

Metric	Before	After	Diff %	Diff abs.
FCP	96.2	93.15	-3.17%	-3.05
LCP	96.2	93.15	-3.17%	-3.05
TTFB	53.5	52.5	-1.87%	-1.00
LCP-TTFB	41.7	40.25	-3.48%	-1.45

Server-Timing

Metric	Before	After	Diff %	Diff abs.
Response Time	45.83	45.04	-1.72%	-0.79
wp-before-template	15.1	15.39	1.92%	0.29
wp-template	29.66	28.69	-3.27%	-0.97
wp-total	44.86	44.11	-1.67%	-0.75

TT4 (twentytwentyfour) benchmarks

Web Vitals

Metric	Before	After	Diff %	Diff abs.
FCP	155.75	159.9	2.66%	4.15
LCP	155.75	159.9	2.66%	4.15
TTFB	81.2	83.4	2.71%	2.20
LCP-TTFB	74.55	75.55	1.34%	1.00

Server-Timing

Metric	Before	After	Diff %	Diff abs.
Response Time	75	77.06	2.75%	2.06
wp-before-template	16.49	16.7	1.27%	0.21
wp-template	57.28	59.09	3.16%	1.81
wp-total	73.75	75.79	2.77%	2.04

RC3 shows slight regression compared to RC2 in block theme.

mukeshpanchal27 commented 2 days ago

WP 6.7 Performance Benchmark Report:

With object caching disabled,

Classic themes (TT1)
- LCP and TTFB shows slight improvement
- wp-template and wp-total shows a ~3% improvement (~1ms)
Block themes (TT3)
- LCP shows a ~0.6% improvement (~0.6ms)
- TTFB shows a ~1.5% improvement (~0.8ms)
- LCP-TTFB shows slight regression
- wp-before-template shows a ~0.3% improvement (~0.06ms)
- wp-template shows a ~3.6% improvement (~1.1ms)
- wp-total shows a ~1.5% improvement (~0.7ms)
Block themes (TT4)
- LCP shows a ~3.6% regression (~5.7ms)
- TTFB shows a ~2.8% regression (~2.3ms)
- LCP-TTFB shows a ~3.7% regression (~2.7ms)
- wp-template and wp-total shows slight improvement

Here are the results of the test runs:

TT1: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11931170428 TT3: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11931351109 TT4: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11931556138

TT1 (twentytwentyone) benchmarks

Web Vitals

Metric	Before	After	Diff %	Diff abs.
FCP	100.5	100.3	-0.20%	-0.20
LCP	100.5	100.3	-0.20%	-0.20
TTFB	46.7	46.5	-0.43%	-0.20
LCP-TTFB	53.95	54.1	0.28%	0.15

Server-Timing

Metric	Before	After	Diff %	Diff abs.
Response Time	40.58	39.17	-3.47%	-1.41
wp-before-template	11.75	11.34	-3.49%	-0.41
wp-template	27.86	26.92	-3.37%	-0.94
wp-total	39.56	38.32	-3.13%	-1.24

TT3 (twentytwentythree) benchmarks

Web Vitals

Metric	Before	After	Diff %	Diff abs.
FCP	98.7	98.1	-0.61%	-0.60
LCP	98.7	98.1	-0.61%	-0.60
TTFB	55.5	54.65	-1.53%	-0.85
LCP-TTFB	42.8	42.9	0.23%	0.10

Server-Timing

Metric	Before	After	Diff %	Diff abs.
Response Time	48.99	47.97	-2.08%	-1.02
wp-before-template	16.58	16.52	-0.36%	-0.06
wp-template	31.32	30.19	-3.61%	-1.13
wp-total	47.8	46.8	-2.09%	-1.00

TT4 (twentytwentyfour) benchmarks

Web Vitals

Metric	Before	After	Diff %	Diff abs.
FCP	154.25	159.9	3.66%	5.65
LCP	154.35	160.05	3.69%	5.70
TTFB	81.2	83.5	2.83%	2.30
LCP-TTFB	73	75.7	3.70%	2.70

Server-Timing

Metric	Before	After	Diff %	Diff abs.
Response Time	74	73.35	-0.88%	-0.65
wp-before-template	16.22	15.89	-2.03%	-0.33
wp-template	56.47	56.34	-0.23%	-0.13
wp-total	72.68	72.22	-0.63%	-0.46

WordPress / performance