Game Performance on Cosmic/Gnome

[XV-02]

I've been benchmarking gaming performance in both Cosmic DE and Gnome recently.

While system configurations vary notably between individual systems - Discrete AMD desktop, compared to Nvidia discrete desktop, compared to Intel discrete desktop, compared to a hybrid graphics laptop - the run-to-run hardware configuration was the same within any given test series.

Between test-series, the ram configuration may be changed - as QA certifies desktop firmware upgrades for release, some ram configurations needs to be explicitly tested against.

[XV-02]

This first set of results shows Cyberpunk 2077 and Unigine Superposition benchmarks on a Thelio Spark B1 with an Nvidia RTX 3060Ti.

Also benchmarked in this battery were Xonotic and GLMark2, however, these tests did not seem to execute correctly in Cosmic, having results many times that of Gnome, and are being discarded as invalid at this moment. All benchmarks were run at 1920*1080, and were run at all available fidelity levels provided by the relevant phoronix-test-suite benchmark.

In these tests, the valid benchmarks are all within one percent of each other - and often within a single FPS of each other.

[ids1024]

Are these benchmarks running with or without vsync? I guess the one that's in the 200s at least must be without vsync.

Vsync is where certain issues may be more apparent. Also latency, but that requires some equipment to measure.

[XV-02]

This set of results shows Cyberpunk 2077 and Unigine Superposition benchmarks on a Thelio R3 with an AMD RX 6600 XT. I've formatted results as a table due to difficulties getting the phoromatic server to correctly compare test results.

Also benchmarked in this battery were Xonotic and GLMark2, however, these tests did not seem to execute correctly in Cosmic, having results several times that of Gnome, and are being discarded as invalid at this moment. All benchmarks were run at 1920*1080, and were run at all available fidelity levels provided by the relevant phoronix-test-suite benchmark.

In these tests, the valid benchmarks are effectively identical,

Test	Gnome	Cosmic	Performance% (Cosmic/Gnome)
Unigine Super - Low	274.9	273.8	99.6%
Unigine Super - Medium	134.2	133.7	99.6%
Unigine Super - High	95.1	95.0	99.9%
Unigine Super - Ultra	39.3	39.3	100%
CP2077 - Low	129.99	130.37	100%
CP2077 - Medium	100.16	100.47	100%
CP2077 - High	83.52	83.41	99.9%
CP2077 - Ultra	73.69	73.87	100%

[XV-02]

Okay, another round, with a few more titles. Right now, dedicated Arc graphics test are in the works. For these tables, I'm using a % +/- - positive values indicate better Cosmic performance, negative values indicate that Gnome has an edge.

As a summary: The Hybrid platform test were comparable, as were the dedicated AMD graphics tests. The dedicated Nvidia graphics results showed server regressions in the Unigine benchmarks compared with the last round of tests, though the actual game benchmarks showed a generally consistent results - slight leads in Cyberpunk 2077, slight trailing in Hitman 3 and DiRT 2. Of particular note to me is the variance in the Unigine benchmarks from test to test, as that wasn't exhibited in the game-specific benchmarks.

AMD

Benchmark	Cosmic DE	Gnome DE	% +/-
CP2077 - Low	130.26	129.51	+<1%
CP2077 - Medium	100.30	99.98	+<1%
CP2077 - High	83.42	83.06	+<1%
CP2077 - Ultra	73.59	73.20	+<1%
DiRT 2 - Low	190.99	191.25	-<1%
DiRT 2 - High	190.83	191.14	-<1%
DiRT 2 - Ultra	190.76	191.15	-<1%
Hitman 3 - Low	208.30	209.18	-<1%
Hitman 3 - Medium	205.05	206.09	-<1%
Hitman 3 - Ultra	196.51	197.42	-<1%
Unigine Super - Low	273.2	273.9	-<1%
Unigine Super - Medium	133.6	133.8	-<1%
Unigine Super - High	94.6	94.8	-<1%
Unigine Super - Ultra	39.2	39.2	0%
Unigine Valley	231.65	231.50	+<1%

Nvidia

The Nvidia system was not explicitly leveraging Vsync on any of the benchmarks. DiRT 2 - Low was excerpted due to crashing on Gnome.

Benchmark	Cosmic DE	Gnome DE	% +/-
CP2077 - Low	115.15	114.01	+<1%
CP2077 - Medium	94.81	94.32	+<1%
CP2077 - High	81.26	80.02	+1%
CP2077 - Ultra	71.42	70.89	+<1%
DiRT 2 - Low	excerpted		N/A
DiRT 2 - High	110.67	114.44	-3%
DiRT 2 - Ultra	110.70	114.58	-3%
Hitman 3 - Low	169.46	176.69	-4%
Hitman 3 - Medium	168.78	175.17	-3%
Hitman 3 - Ultra	162.46	168.60	-3%
Unigine Super - Low	266.2	271.7	-2%
Unigine Super - Medium	139.1	294.0	-111%
Unigine Super - High	103.4	128.5	-24%
Unigine Super - Ultra	45.8	93.2	-103%
Unigine Valley	217.37	263.30	-21%

Hybrid Graphics Laptop

A Serw13 with Nvidia 4070 RTX and Intel i9-13900HX was used.

Benchmark	Cosmic DE	Gnome DE	% +/-
CP2077 - Low	80.74	78.04	+3%
CP2077 - Medium	77.40	76.28	+1%
CP2077 - High	70.19	71.03	-1%
CP2077 - Ultra	70.30	70.68	-<1%
DiRT 2 - Low	127.15	127.16	-<1%
DiRT 2 - High	127.05	127.16	-<1%
DiRT 2 - Ultra	127.14	127.39	-<1%
Hitman 3 - Low	184.29	186.53	-1%
Hitman 3 - Medium	181.98	184.21	-1%
Hitman 3 - Ultra	171.82	171.01	-<1%
Unigine Super - Low	22.1	23.7	-6%
Unigine Super - Medium	11.9	12.3	-3%
Unigine Super - High	8.6	8.8	-2%
Unigine Super - Ultra	3.9	3.9	0%
Unigine Valley	21.08	22.47	-6%

Next steps

First, I am going to immediately verify the unigine results on Nvidia with more runs. Next I'll move into the next round of benchmarks, which should include Arc graphics and 4k benchmarks.

[XV-02]

After re-running the Unigine benchmarks several times, it looks like the regressions were the result of spurious runs. I'm uncertain of the cause, but subsequent runs were extremely consistent. The same hardware was used, as were the same package versions. Unigine Superposition tests conducted on Ultra were omitted as they crashed before completion 2 out of 3 times on both gnome and cosmic.

Benchmark	Cosmic DE	Gnome DE	% +/-
Unigine Super - Low	268.8	270.9	-<1%
Unigine Super - Low	269.7	271.6	-<1%
Unigine Super - Low	269.1	271.5	-<1%
Unigine Super - Medium	139.2	140.0	-<1%
Unigine Super - Medium	139.3	140.1	-<1%
Unigine Super - Medium	138.9	140.2	-<1%
Unigine Super - High	103.4	104.1	-<1%
Unigine Super - High	103.5	104.2	-<1%
Unigine Super - High	103.7	104.3	-<1%
Unigine Valley	218.89	220.11	-<1%
Unigine Valley	219.34	220.59	-<1%
Unigine Valley	219.33	220.88	-<1%

I'll keep an eye out on the next test run, which will have packages updated to their latest as of 11th April.