Start tracking performance

[kvark]

As we are approaching the full test coverage on Wrench and Servo WPT, we need to start looking into the behavior in dynamics: bugs appearing during scrolling, navigation, and of course - performance.

[dati91]

Is there any "official" tool (or way) to track those? I know that chrome uses telemetry for performance, but don't know if it can run with servo as well.

[kvark]

We have a debug overlay in WR that would be your first thing to track. Basically, enable the minimal overlay and visit a website, scroll back and forth and note the compositor timings. Another one is - running the MotionMark suite for scores.

[imiklos]

Here are our scores on MotionMark running with gecko using an NVIDIA 750 Ti graphics card. The scores are the average from 10 runs. If you interested all the runs : https://gist.github.com/imiklos/a23df038160362c4266d0af5c78f437f

MotionMark config:

• Test length: 30 sec each
• Tiles: Big Tiles
• Adjusting test complexity: Ramp
• Target frame rate: 60 FPS
• Time measurement method: performance.now()

OpenGL

Suites	Test name	AVG
HTML suite	CSS bouncing circles	314.813
	CSS bouncing clipped rects	237.741
	CSS bouncing gradient circles	306.049
	CSS bouncing blend circles	134.185
	CSS bouncing filter circles	186.271
	CSS bouncing SVG images	386.572
	CSS bouncing tagged images	350.765
	Leaves 2.0	218.781
	Focus 2.0	80.378
	DOM particles, SVG masks	240.986
	Composited Transform	201.583
	Total points	222.216

Vulkan WR (https://github.com/szeged/webrender/commit/0e55b4fbe871d35aca54b7282b58b98487fb64d7)

Suites	Test name	AVG	Diff from OpenGL
HTML suite	CSS bouncing circles	273.797	-41.016
	CSS bouncing clipped rects	126.548	-111.193
	CSS bouncing gradient circles	242.199	-63.85
	CSS bouncing blend circles	19.856	-114.329
	CSS bouncing filter circles	24.319	-161.952
	CSS bouncing SVG images	275.052	-111.52
	CSS bouncing tagged images	244.483	-106.282
	Leaves 2.0	163.867	-54.914
	Focus 2.0	25.915	-54.463
	DOM particles, SVG masks	133.058	-107.928
	Composited Transform	94.14	-107.443
	Total points	102.989	-119.227

Vulkan WR (https://github.com/szeged/webrender/commit/f60f89717d0a3da7d82612b82deae07416b23461)

Suites	Test name	AVG	Diff from prev. Vulkan	Diff from OpenGL
HTML suite	CSS bouncing circles	289.45	15.653	-25.363
	CSS bouncing clipped rects	194.614	68.066	-43.127
	CSS bouncing gradient circles	260.636	18.437	-45.413
	CSS bouncing blend circles	34.634	14.778	-99.551
	CSS bouncing filter circles	63.569	39.25	-122.702
	CSS bouncing SVG images	286.586	11.534	-99.986
	CSS bouncing tagged images	271.434	26.951	-79.331
	Leaves 2.0	188.934	25.067	-29.847
	Focus 2.0	39.753	13.838	-40.625
	DOM particles, SVG masks	152.816	19.758	-88.17
	Composited Transform	143.394	49.254	-58.189
	Total points	139.667	36.678	-82.549

Vulkan WR (https://github.com/szeged/webrender/commit/1bef6468ca63019aa78fd064e0527c50cb999816)

Suites	Test name	AVG	Diff from prev. Vulkan	Diff from OpenGL
HTML suite	CSS bouncing circles	295.849	6.399	-18.964
	CSS bouncing clipped rects	218.029	23.415	-19.712
	CSS bouncing gradient circles	265.206	4.57	-40.843
	CSS bouncing blend circles	35.866	1.232	-98.319
	CSS bouncing filter circles	62.935	-0.634	-123.336
	CSS bouncing SVG images	294.319	7.733	-92.253
	CSS bouncing tagged images	263.848	-7.586	-86.917
	Leaves 2.0	189.09	0.156	-29.691
	Focus 2.0	41.726	1.973	-38.652
	DOM particles, SVG masks	151.967	-0.849	-89.019
	Composited Transform	148.82	5.426	-52.763
	Total points	143.115	3.448	-79.101

[imiklos]

Here are some updates for the MotionMark scores: The scores are the average from 10 runs. If you interested all the runs : https://gist.github.com/imiklos/4876bdbe1e756496183c0bdf80c9a472

OpenGL (Baseline)

Suites	Test name	AVG
HTML suite	CSS bouncing circles	314.813
	CSS bouncing clipped rects	237.741
	CSS bouncing gradient circles	306.049
	CSS bouncing blend circles	134.185
	CSS bouncing filter circles	186.271
	CSS bouncing SVG images	386.572
	CSS bouncing tagged images	350.765
	Leaves 2.0	218.781
	Focus 2.0	80.378
	DOM particles, SVG masks	240.986
	Composited Transform	201.583
	Total points	222.216

Vulkan WR(https://github.com/szeged/webrender/commit/01cee09c73fdf0567836eec880b579d24c68bf13)

Suites	Test name	AVG of Vulkan	Diff from baseline
HTML suite	CSS bouncing circles	359.319	44.506
	CSS bouncing clipped rects	266.035	28.294
	CSS bouncing gradient circles	331.102	25.053
	CSS bouncing blend circles	265.641	131.456
	CSS bouncing filter circles	188.553	2.282
	CSS bouncing SVG images	419.094	32.522
	CSS bouncing tagged images	419.501	68.736
	Leaves 2.0	274.044	55.263
	Focus 2.0	73.917	-6.461
	DOM particles, SVG masks	274.032	33.046
	Composited Transform	236.954	35.371
	Total points	259.931	37.715

Vulkan WR(https://github.com/szeged/webrender/commit/37c8ee2cb0497ea51d7f51534e99282167b38c99)

Suites	Test name	AVG of Vulkan	Diff from baseline
HTML suite	CSS bouncing circles	358.922	44.109
	CSS bouncing clipped rects	253.972	16.231
	CSS bouncing gradient circles	332.296	26.247
	CSS bouncing blend circles	255.843	121.658
	CSS bouncing filter circles	185.904	-0.367
	CSS bouncing SVG images	416.996	30.424
	CSS bouncing tagged images	410.325	59.56
	Leaves 2.0	270.608	51.827
	Focus 2.0	72.489	-7.889
	DOM particles, SVG masks	259.507	18.521
	Composited Transform	237.539	35.956
	Total points	254.87	32.654

Vulkan WR(https://github.com/szeged/webrender/commit/daac8d4bcd419c8f06754141ffac9ecc0a7eb877)

Suites	Test name	AVG of Vulkan	Diff from baseline
HTML suite	CSS bouncing circles	356.253	41.44
	CSS bouncing clipped rects	248.915	11.174
	CSS bouncing gradient circles	334.081	28.032
	CSS bouncing blend circles	249.609	115.424
	CSS bouncing filter circles	185.896	-0.375
	CSS bouncing SVG images	414.184	27.612
	CSS bouncing tagged images	414.267	63.502
	Leaves 2.0	278.145	59.364
	Focus 2.0	71.306	-9.072
	DOM particles, SVG masks	258.16	17.174
	Composited Transform	237.878	36.295
	Total points	254.233	32.017

[zbraniecki]

Are the numbers between 37c8ee2 and daac8d4 really identical or is it a mistake?

[zakorgy]

@zbraniecki that was an unfortunate mistake, I've updated the numbers for 37c8ee2. Thanks for noticing it.