Benchmarks results

[shmerl]

Just for visibility, this can be a meta issue for referencing various posted benchmarks.

[shmerl]

The Witcher 3 (GOG GOTY), run in Wine/esync + dxvk. Around 3.6% improvement in framerate for me.

[Snektron]

I used this project on the implementation of my bachelor's thesis, which is about ray tracing volumetric models with Vulkan (https://github.com/Snektron/Xenodon). Im happy to report that performance more than doubles (on the single dataset i've tested). From some viewing angles my RX 580 even performs better than a GTX Titan X (Pascal)! Amazing work, im very impressed.

The output of my benchmark can be found here: https://gist.github.com/Snektron/0c601b89cc09bcfe29a93734d47d3482

[TheAquabat]

with Middle Earth Shadow of War, medium settings (DXVK):

71 fps with RADV Mesa 19.1.1 73 fps with Mesa-aco master.

[TheAquabat]

Shadow of The Tomb Raider, medium settings (DXVK):

39 fps AMDVLK git from last week 32 fps RADV, Mesa 19.1.1 34 fps RADV/ACO master git.

[SR-dude]

Just what version of Mesa should we compare ACO against? Is it 19.1.X or 19.2-git ?

[doitsujin]

@SR-dude you can disable ACO by setting RADV_PERFTEST=llvm without changing your Mesa build.

[SR-dude]

My question was when did ACO fork? 19.1 or 19.2? For clean performance testing, it would be unfair to compare ACO against an old mesa version.

[pendingchaos]

ACO is currently based on 6cc213b3c123a636baf07511c8e17ca32fb5da5a.

[grinceur]

Quake Champion, Ultra presets, render scale 100% (DXVK), on a rx580: with ACO: with RADV_PERFTEST=llvm :

Also an issue with this game is heavy stutters when first time encountering a shader: ACO in the menu when loading bots with random skin (longer frees 1.2 seconds): with RADV_PERFTEST=llvm (longer frees 2.4 seconds):

[grinceur]

The Talos Principle, Ultra settings: 69.1 fps with ACO 66.3 fps with RADV_PERFTEST=llvm

[dennis1248]

Middle-Earth: Shadow of War, mix of high-ultra, 3440x1440, low filters. Using the in-game benchmark. Each test ran 4 times since I noticed performance would increase after each run. GPU is a Vega 64, Proton 4.2-9.

llvm: 82fps aco: 85fps

[debiangamer]

The Rise of the Tomb Raider benchmark: with and without ACO 47 fps at 1080p medium quality settings. Using RX560 2GB, Ryzen 5 1600 and Debian Sid Xfce. When you install ACO mesa to /usr/local to not to mess your current mesa, add the following lines into SteamLibrary/steamapps/common/Rise of the Tomb Raider/config/extra-environment.sh: LIBGL_DRIVERS_PATH="/usr/local/lib/x86_64-linux-gnu:/usr/local/lib/x86_64-linux-gnu/dri/" LD_LIBRARY_PATH="/usr/local/lib/x86_64-linux-gnu:${LD_LIBRARY_PATH}" VK_ICD_FILENAMES="/usr/share/vulkan/icd.d/radeon_icd.x86_64.json

If you want to see the ACO is in use warning message, add --log-to-file to the launch parameters with the Steam client. The log file is in the ~/.local/share/feral-interactive/Rise of the Tomb Raider/ folder.

[shmerl]

ACO is currently based on 6cc213b.

@pendingchaos @daniel-schuermann: Can you please rebase on the recent Mesa master, to make use of the latest fix for llvm 9?

[aqxa1]

Another RoTTR benchmark (built in benchmark, Vega 56, 1440p, mostly high settings):

LLVM git (with LLVM9 fix): 85.53 fps ACO git: 99.33 fps RADV_PERFTEST=llvm (with bugged graphics): 92.67 fps

I'm not sure if my build is slower for some reason, or if the bugged build is faster due to missing effects. But looks to be 7-16% increase from ACO over the LLVM builds.

[ntropy83]

Star Citizen with Steam Remote Play:

llvm: 40 - 55 fps Aco: 45 - 60 fps

Ryzen 2700X, soon to be tested with Ryzen 3900X

more importantly, shader stutter is cut in half, tho the game can still die, if you have to compile a heavy one. Correction: the game does not die, it only hangs for 20 seconds, but then you are done with it. Happened two times on the most graphic intense places. But after that with aco shader the game becomes playable. Barely no stuttering at all. :)

https://www.youtube.com/watch?v=z7_kU6CQpvI shitty fps due to steam remote play

[Tvviztid]

From Vulkan examples and demos https://github.com/SaschaWillems/Vulkan Ubuntu 19.04 Kernel 5.1.16, Ryzen 3 1200 + RX 550

Compute Raytracing RADV_PERFTEST=llvm 54fps ACO 112fps

Compute Cloth RADV_PERFTEST=llvm 720-735fps ACO 620-650

A few others have seen minor fps increases and decreases. Haven't tested all of them.

Edit 1: Bloodstained Ritual of the Night GOG, WineHQ staging, DXVK 1.2.3 1920x1080 all Epic settings, standing still at very first room RADV_PERFTEST=llvm 76fps ACO 79fps

[aqxa1]

Nier: Automata (4K, AA off, blur off, Vega 56) opening shmup sequence until it shifts horizontal. Approximate numbers (just monitoring DXVK_HUD while playing). The averages are likely even higher for ACO, since the game is limited to 60 fps. LLVM rarely gets near 60 fps.

ACO: AVG = 57 fps, MIN = 45 fps LLVM: AVG = 44 fps, MIN = 37 fps

[daniel-schuermann]

@pendingchaos @daniel-schuermann: Can you please rebase on the recent Mesa master, to make use of the latest fix for llvm 9?

We will definitely do so in the next couple of days, but it needs some changes to ACO, which is why we didn't do it yet.

[daniel-schuermann]

Star Citizen with Steam Remote Play: Correction: the game does not die, it only hangs for 20 seconds, but then you are done with it. Happened two times on the most graphic intense places. But after that with aco shader the game becomes playable. Barely no stuttering at all. :)

Thx, could you elaborate if this extreme freezes also happen with llvm with an empty shader cache? 20s for a single shader sounds a bit crazy even if it hits some worst cases in our algorithms.

[ntropy83]

Star Citizen with Steam Remote Play: Correction: the game does not die, it only hangs for 20 seconds, but then you are done with it. Happened two times on the most graphic intense places. But after that with aco shader the game becomes playable. Barely no stuttering at all. :)

Thx, could you elaborate if this extreme freezes also happen with llvm with an empty shader cache? 20s for a single shader sounds a bit crazy even if it hits some worst cases in our algorithms.

They do but not that long, but more often. It is the moment when you go from Riker Memorial Station by train to Area 18, just before the train will go around the last corner. LLVM does have lots of heavy stuttering here, sometimes 5 - 10 seconds long. ACO has way less stuttering just that one bigtime one. It took so long, I was already on the train going the way back, when it finished.

If I can help you with debugging, let me know what variables to set.

[dennis1248]

Now that Dishonored 2 is fixed in the latest release I did a quick and simple test where I compared the performance of ACO to that of LLVM in a certain scene during the training level (Standing on the edge of the roof you spawn on during the stealth training while looking at the generator). This game profits lots from ACO.

ACO: 79fps LLVM: 70fps

Also the average FPS while playing through the game is also significantly better, LLVM gets 60-80fps while ACO gets 70-90. Very impressive.

The game however is still a unplayable stuttery laggy mess while playing through a level for the first time.

[AwesamLinux]

I have tried some games at 2560x1600 maxed settings on a Vega 56 nano, and have seen about 1-3fps gain across the board on the games I have tried so far. There is also a notable reduction in stuttering in certain games (that is hard to measure but you see it), especially in games like Tekken 7 and Hellblade.

Hellblade ACO: 55.4 LLVM: 54.1

A Plague Tale ACO: 50.1 LLVM: 48.8

The Surge ACO: 83.5 LLVM: 82.5

Witch Hunt ACO: 30.2 LLVM: 27.2

Redout ACO: 101.0 LLVM: 99.9

I have not encountered any games yet that performed worse or had graphical issues or crashes.

EDIT: System Specs: system_info.txt

[SpookySkeletons]

1080p Dark souls 3 Intel HD 4800MQ, AMD WX 7100. ACO doesn't seem to add much to the average FPS but I noticed something very fun after bringing up my dual boot. ACO/ LLVM w/ Mesa: 46-47 FPS Windows 10: 40 FPS

[shmerl]

Just FYI, mesa-aco is still hanging when built with latest llvm 9 and run with RADV_PERFTEST=llvm.

[daniel-schuermann]

We are actually waiting on https://github.com/daniel-schuermann/mesa/issues/64 to be sure to not introduce new regressions when updating.

[shmerl]

@daniel-schuermann: I see it was fixed! I'm not sure how to track rebases, since the log shows latest commits. Can you please post here once you rebase on newest Mesa? Thanks!

[daniel-schuermann]

@shmerl: it's not the very newest mesa, because we like to have a bit of testing ourselves, but it's done :) Using RADV_PERFTEST=llvm shouldn't expose any rtld bugs anymore on master.

[shmerl]

Supraland:

llvm:

aco:

Not much of a difference, and frametimes jumping all over the place in both cases. Something is off with Unreal Engine Vulkan usage.

[shadywack]

Elite Dangerous Ryzen 1800X and Vega 64 running on Manjaro testing (5.2.1 kernel), installed mesa-aco-git 19.2.0_devel.113859.5031ab774a6-1 from the AUR.

Tested: fuel scooping, planetary landings, and docking at Explorer's Anchorage near Sagittarius A. Explorer's anchorage has a moderate amount of traffic.

Running at 3840x2160 with Variable Refresh set to true in the xorg conf.

Comparing to Windows version, running natively I get 60fps (monitor refresh cap, didn't test vsync off) and 60fps under Proton 4.2-9 and RADV-ACO in space while fuel scooping.

Docking at Explorer's Anchorage framerates average 52 natively in Windows, RADV-ACO averages 49.

Planetary Landing, went to Stuemeae FG-Y d7561 ABCD 1 a and picked random spots to land, averaged 45fps natively in Windows, RADV-ACO averages 43.

Mesa 19.2 and ACO appears to have reduced the performance delta from 15% to 5% since I last compared framerates 3 months ago.

[shadywack]

Rise of the Tomb Raider - Ryzen 1800X and Vega64, 3840x2160, AF set to 8x, AO on, textures very high, all other options turned up to max detail. Using mesa-aco-git 19.2.0_devel.113859.5031ab774a6-1

Windows 10 DX12 Mountain Peak 56.07 (min: 19.14, max: 127.74) Syria 49.15 (min: 30.46, max: 66.88) Geothermal Valley 51.51 (min: 35.65, max: 60.86) Overall 52.42

Manjaro using Feral Vulkan port: Mountain Peak 53.86 (min: 33.81, max: 80.81) Syria 50.05 (min: 24.37, max: 82.56) Geothermal Valley 49.31 (min: 35.10, max 73.41) Overall 51.13

You folks in the Valve Open Source Graphics Group are wonderful, this is the best performance I've seen my Vega 64 pull out yet. This is a great improvement not just for DXVK games, but across the board. Thank you very much, as well as the outstanding Mesa devs. If this isn't "Fine Wine" idk what is.

[aqxa1]

Prey (2017)

Vega 56

76 fps -> 89 fps (+17%)

LLVM:

ACO:

[aqxa1]

@shadywack Although the RoTTR results look great you might want to compare with D3D11 on Windows for more of a like-for-like comparison, as I remember the D3D12 version didn't always perform well on Windows

[doitsujin]

as I remember the D3D12 version didn't always perform well on Windows

Pretty sure that was limited to systems with Nvidia GPUs. D3D12 outperforms D3D11 on my system on Windows even when the game is fully GPU-bound.

@aqxa1 regarding those Prey screenshots, if you compare the two, it looks like specular lighting is broken in one of them. Not sure which one is correct though.

[aqxa1]

@doitsuijin Fair enough on D3D12, that is even more impressive if that is the case.

I'll take a look at Prey again, as I do see the difference now that you point it out.

EDIT: Looks to just be a timing issue in the screenshot. The highlighting flickers in and out so the player can find useful items

Here's another ACO screenshot in the same state as the LLVM shot:

[doitsujin]

Fair enough on D3D12, that is even more impressive if that is the case.

To be fair, the Windows version also features HBAO+ which the Feral port lacks, and it does cost quite a bit of performance.

[shadywack]

@shadywack Although the RoTTR results look great you might want to compare with D3D11 on Windows for more of a like-for-like comparison, as I remember the D3D12 version didn't always perform well on Windows

As others stated, on AMD the DX12 renderer outperforms the DX11 renderer, but I will run some more benchmarks for comparison later on. There will be a marginal drop, the thing is that it's not that big a deal for the Vulkan port of Feral's to be as performant as it is, it's just a big deal for it to be that performant on AMD hardware in Linux, as last year the Mesa Vulkan driver was just not in the state it is today. The leaps and bounds the Mesa team has made with the ACO complimenting the improvements is staggering in terms of Vulkan performance alone, that ontop of the DXVK performance improvements is rapidly putting AMD hardware into a great value proposition where $ per frames per second is now swinging majorly in AMD's favor.

[ryao]

Would people posting numbers also include their system configurations? In specific, CPU, GPU, RAM and kernel version. It would help put the numbers into context.

[dlove67]

@ryao Also the settings used (if a preset) so it can be compared easily to windows performance.

[AwesamLinux]

ryao good point: System specs: system_info.txt GPU driver version: 19.1.0+aco+git1563884390-17d9490cef1-1bionic1

Anyway things are looking good so far, slight improvements in FPS and stuttering is noticeably reduced.

The Surge (2560x1600 resolution, maxed settings, AA set to FXAA) ACO: 69.8 LLVM: 65.8

Visage (2560x1600 resolution, maxed settings, AA quality disabled) ACO: 141.1 LLVM: 136.4

The Park (2560x1600 resolution, maxed settings, AA set to 6x) ACO: 57.5 LLVM: 56.3

(Edited for accuracy of the AA settings)

[daniel-schuermann]

@aqxa1

Prey (2017)

Would you be able to retest with current master for #85 to be sure it's APU-related?

[ntropy83]

I tried a game called "Remember Me" with the actual master build and it is an amazing improvement. Stutter is nearly gone and way less framedrops than on llvm. I will try make a split screen video of it on the weekend, if I figure out how that is done :).

[ntropy83]

Interstellar Prime alpha demo on backend_issue_70 branch. Master branch is not working for me atm, games always detect a missing DX11 renderer upon startup. Maybe I miscompiled it ?

https://www.youtube.com/watch?v=PPknO-zC8wA

No stutter at all

[random2324]

Vega 56 i7 4980HQ @ 4.5Ghz

Dota2 using PTS 1920x1080 LLVM 114,7 fps ACO 121,8fps

Dirt 3 using custom Proton build 1920x1080 Maximum Details 2x MSAA LLVM 72,5 fps ACO 206,3 fps

Looks like LLVM is horribly broken with this game.

[nottux]

Battlefield 4 over lutris:

With ACO:

With llvm:

Couple updates ago game on ACO would cause ring errors and heavy performance degradation (0-1 fps), as of today its not an issue any more and games works about 8-10 fps faster under ACO compiler, also less stuttery.

For testing i manually set gpu dpm to high and cpu modeset to performance, before all test i always removed bf4.dxvk-cache file

output of DRI_PRIME=1 glxinfo -B:

utku3@pop-os:~$ DRI_PRIME=1 glxinfo -B
name of display: :1
display: :1  screen: 0
direct rendering: Yes
Extended renderer info (GLX_MESA_query_renderer):
    Vendor: X.Org (0x1002)
    Device: Radeon 500 Series (POLARIS12, DRM 3.33.0, 5.3.0-050300rc3-generic, LLVM 8.0.0) (0x699f)
    Version: 19.2.0
    Accelerated: yes
    Video memory: 2048MB
    Unified memory: no
    Preferred profile: core (0x1)
    Max core profile version: 4.5
    Max compat profile version: 4.5
    Max GLES1 profile version: 1.1
    Max GLES[23] profile version: 3.2
Memory info (GL_ATI_meminfo):
    VBO free memory - total: 2036 MB, largest block: 2036 MB
    VBO free aux. memory - total: 3044 MB, largest block: 3044 MB
    Texture free memory - total: 2036 MB, largest block: 2036 MB
    Texture free aux. memory - total: 3044 MB, largest block: 3044 MB
    Renderbuffer free memory - total: 2036 MB, largest block: 2036 MB
    Renderbuffer free aux. memory - total: 3044 MB, largest block: 3044 MB
Memory info (GL_NVX_gpu_memory_info):
    Dedicated video memory: 2048 MB
    Total available memory: 5120 MB
    Currently available dedicated video memory: 2036 MB
OpenGL vendor string: X.Org
OpenGL renderer string: Radeon 500 Series (POLARIS12, DRM 3.33.0, 5.3.0-050300rc3-generic, LLVM 8.0.0)
OpenGL core profile version string: 4.5 (Core Profile) Mesa 19.2.0-devel
OpenGL core profile shading language version string: 4.50
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile

OpenGL version string: 4.5 (Compatibility Profile) Mesa 19.2.0-devel
OpenGL shading language version string: 4.50
OpenGL context flags: (none)
OpenGL profile mask: compatibility profile

OpenGL ES profile version string: OpenGL ES 3.2 Mesa 19.2.0-devel
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20

for anyone interested in my dpm scripts: https://gist.github.com/tuxutku/f59fe53ae9eacd1c8c09b21202df638d

[daniel-schuermann]

Vega 56 i7 4980HQ @ 4.5Ghz

Dota2 using PTS 1920x1080 LLVM 114,7 fps ACO 121,8fps

Dirt 3 using custom Proton build 1920x1080 Maximum Details 2x MSAA LLVM 72,5 fps ACO 206,3 fps

Looks like LLVM is horribly broken with this game.

Thx for reporting these nice results. Did you spot any visual difference in Dirt3? Because this difference looks a bit too good to be true ;) Maybe missing shadows, particles or the like?

[random2324]

Thx for reporting these nice results. Did you spot any visual difference in Dirt3? Because this difference looks a bit too good to be true ;) Maybe missing shadows, particles or the like?

I didnt spot any problems with ACO.

My theory is, that LLVM has problems with the EGO Engine V2.0 used by DIRT 3 and DIRT: Showdown. I mention Showdown because I reported https://bugs.freedesktop.org/show_bug.cgi?id=100069 years ago. Basically a specific graphic option slows down the game from cpu limited frame rates to a slide show. Of course this is just a corner case in an old game. Its still a little bit disappointing that no one investigated the problem over the years.

However, last year I tested DIRT3 on windows vs linux still with my old Radeon HD 7970 and to some surprise, I found a similar problem: On Windows, setting shadows to very high just cost 2-3 fps, on Linux it made the game unplayable. As we use Vulkan with DIRT3 and OpenGL 4.5 with Showdown, there must be something both share...

So this year, I tested ACO vs LLVM and bingo! ACO hardly takes a fps hit from that option but LLVM slows down a lot. Which makes me think, that this is a bug in the LLVM shader compiler.

On a last thing, I like to thank you guys for your work. I maintain the list https://www.gamingonlinux.com/wiki/Games_broken_on_Mesa to 90 % of the time. So I got a good overview of mesa problems. LLVM bugs are definitely some of the biggest problems the mesa project has. I mean, its not only a problem fixing those bugs which often takes a lot of time, its actually a joke that most of those fixes don't get packported to stable releases. Some distros run LLVM 7 some 8 some 9 and in the end you are forced to use git snapshots. For the user its like playing roulette.

I really hope ACO gets the default for Vulkan and OpenGL sooner than later.

[ntropy83]

In "Homefront: The Revolution" fps with mesa-aco is 15 - 180 fps on ultra 1080p, Vega 64/Ryzen 3900X. On llvm it is 10 - 120 fps. On mesa-aco all shader stutter is either completly gone or so small that I dont notice. The game tho has issues with volumetric lightning. I f I turn that off the framedrops happening wont be so harsh, yet then the game looks ugly. On AMDVLK it doesnt start at all.

[pingubot]

Just tested amdgpu-pro 19.30 vs an aco build a952333 in Wolfenstein 2.

GPU: Amd Vega 56 CPU: Intel Core i5 3570k@4.2ghz

Settings: WQHD, Mein Leben

Amdgpu-pro 1930:

Menu: 380fps In-Game test spot: 150fps

Amdvlk v-2019.Q3.4:

Menu: 356fps In-Game test spot: 146fps

ACO:

Menu: 315fps In-Game test spot: 124fps

LLVM:

Menu: 299fps In-Game test spot: 95fps

All fully gpu bound.

[daniel-schuermann]

Thx for sharing! ACO doesn't expose 16bit instructions yet, so this might be part of the difference. Did you also test against radv/llvm and amdvlk? I'm curious

[pingubot]

@daniel-schuermann Added llvm, amdvlk will follow tomorrow or saturday.