Install Reshade for a Vulkan game

[rylydou]

I am installing Reshade into Cemu which uses Vulkan. I know that on Windows you can install Reshade on a Vulkan game. But I don't know what dll name to use or if it is even possible.

I have tried:

• dxgi - The suggested option
• opengl32 - Because Cemu has an option to switch to that but I would prefer to use Vulkan for performance
• vulkan

[kevinlekiller]

I'm not sure, never used Cemu.

While doing a quick search I found this: https://i.imgur.com/o8nwDkd.jpg linked here https://teddit.net/r/CemuPiracy/comments/ey6ien/is_there_any_tutorial_for_reshade_on_cemu_botw/

Does that mean the games need patches for Reshade to function?

Looking at this video at 4m50s, there's no dll file in this Cemu folder, I'm guessing Reshade does something different to hook into games using Vulkan?

[kevinlekiller]

ReShade does not support DLL wrapping in Vulkan. Vulkan has a layering mechanism that ReShade has to use to be spec conform. And that requires ReShade to be in a central place, because those layers are configured via system-wide registry keys. You are best off using the setup tool to set up ReShade, it knows what to do to make it work.

https://reshade.me/forum/troubleshooting/6153-reshade-and-vulkan

I'm guessing if we figure out which registrey keys are added, then we could try to do that programatically?

[kevinlekiller]

For now, you could use vkBasalt which supports most Reshade shaders, only issue is there's no in game GUI.

[kevinlekiller]

This post explains how the installer sets up the registry:

https://reshade.me/forum/troubleshooting/6153-reshade-and-vulkan#41351

[rylydou]

Thanks, I'll give vkBasalt with goverlay a try. Thanks for the help!

[kevinlekiller]

Not tested in a game, but I've got something that might work if you want to test.

The WINEPREFIX has to be set for the game in particular and the ReShade64.json needs to be changded to ReShade32.json if the program is 32 bits, the reg:64 also has to be changed to reg:32 if it's a 32 bit prefix.

WINEPREFIX="$HOME/Games/windows/lutris/Test/wineprefix" wine reg ADD HKLM\\SOFTWARE\\Khronos\\Vulkan\\ImplicitLayers /d 0 /t REG_DWORD /v "Z:\\home\\$USER\\.reshade\\reshade\\ReShade64.json" -f /reg:64

Here's regedit in a test WINEPREFIX: https://i.imgur.com/80KiVki.png

[rylydou]

I've have tried doing the registry edit but I don't have either of the ReShade dlls in ~/.reshade/reshade. I do have the json files that point to them though.

[kevinlekiller]

That's a good point, the dxgi.dll would have to be linked to ReShade64.dll and the d3d9.dll to ReShade32.dll

I'll add those things to the script, will have to find a Vulkan game to test on.

[kevinlekiller]

Half life 2 is 32 bit and has Vulkan support : https://www.pcgamingwiki.com/wiki/Half-Life_2#API

Doom 2016 64 bit / Vulkan : https://www.pcgamingwiki.com/wiki/Doom_(2016)#API

Will test these this week and update the script.

[kevinlekiller]

I've been testing on Doom, but not having any success, the registry entry is correctly added, but the game doesn't launch with Reshade.

Added the changes to a different branch:

https://github.com/kevinlekiller/reshade-steam-proton/commit/8275417a3e0d4c18ee1042d5c0c8675b33b13fac

[kevinlekiller]

In a Windows VM I installed Reshade to make sure the Registry matches what's in wine:

So the issue is somewhere else.

[kevinlekiller]

The issue might be with Reshade itself:

https://reshade.me/forum/suggestions/7074-linux-vulkan-opengl-version#40598

This unlikely to happen anytime soon, since large parts of the ReShade code base are Windows specific (especially the hooking code). Only thing that could potentially be separated out would be the Vulkan layer implementation, but that is already partially covered by vkBasalt so it doesn't feel worth the effort right now.

Edit: Although, (in our case) it's through Wine, but perhaps there's some Windows specific Vulkan stuff that's not available on Linux so Wine can't do anything about it?

[kevinlekiller]

In ReShade 5.0, there are some changes to how the json files are handled.

Here's the relevant section:

https://github.com/crosire/reshade/blob/67f4f718614800b426f20291b2fc7d7ca6644f37/setup/MainWindow.xaml.cs#L711

[canceralp]

Would it be possible to install Reshade 4.9.1 with global Vulkan enabling option? This is better than nothing for now.

[kevinlekiller]

Not sure, from what I can remember it doesn't function at all. Busy with work and other stuff at the moment, so not enough time to test, unfortunately.

[canceralp]

I tried 4.9.1, 5.0.2 and 5.2.1. None works. I tried them on Valheim and RDR 2. Valheim can also work on Dx11 so it is possible to make the installation for it but RDR 2 has only Vulkan for Linux (Dx12 freezes the PC).

I have read somewhere Wine runs all the games in administrator mode. I also know that Reshade Vulkan doesn't load when the game is run with admin privileges. Could that be the problem?

[canceralp]

It is not about Vulkan games but there is good news from Crosire. He fixed a problem which prevented newer Reshade versions to work with DirectX 12 games. I hope he might look into Vulkan problem as well.

[SirSAC]

Hello, try vulkan-1.dll and let me know if worked, bye. Edit: Also download the ReShade executable and extract it with 7Zip and see if is there any DLLs, if is some named ReShade32.dll and/or ReShade64.dll then rename one of them as vulkan-1.dll the JSON file is useless or it should be, otherwise if is needed just edit the JSON file and replace the ReShade(32/64).dll with vulkan-1.dll and this should work.

[canceralp]

Hello, try vulkan-1.dll and let me know if worked, bye. Edit: Also download the ReShade executable and extract it with 7Zip and see if is there any DLLs, if is some named ReShade32.dll and/or ReShade64.dll then rename one of them as vulkan-1.dll the JSON file is useless or it should be, otherwise if is needed just edit the JSON file and replace the ReShade(32/64).dll with vulkan-1.dll and this should work.

I won't have access to a Linux PC couple more days. If anyone tried it, could they please report if it works?

[screwylightbulb]

Hello, try vulkan-1.dll and let me know if worked, bye. Edit: Also download the ReShade executable and extract it with 7Zip and see if is there any DLLs, if is some named ReShade32.dll and/or ReShade64.dll then rename one of them as vulkan-1.dll the JSON file is useless or it should be, otherwise if is needed just edit the JSON file and replace the ReShade(32/64).dll with vulkan-1.dll and this should work.

I won't have access to a Linux PC couple more days. If anyone tried it, could they please report if it works?

I tried this on Baldur's Gate 3 on my Linux machine where I do have Reshade working, but it did not work.

[SirSAC]

I tried too for Borderland 2 (DX9) and TerraTech (DX11) and won't work, so sorry, it was just an idea, but works only if I run the setup and select the game executable and telling the setup is running on Vulkan, is creating "ReShade32.dll" or "ReShade64.dll" but not the JSON file in the game folder and something else is creating in "C:\ProgramData" that contain the original JSON file and maybe something in registry but I'm not able yet to track what is it or if is something added/changed. My test PC with Linux died :( sorry if I'm not able to provide more details, I just give what I remember from the Linux machine.

PS: try to run the setup of ReShade and give the game executable and select Vulkan for runtime, maybe this will work if setup takes the details correctely.

[EndlesslyFlowering]

took a while to figure out: it works both normally and through gamescope. really tired now. will post how to do this later :)

[canceralp]

This is early new year:) I am (and many more, I'm sure) looking forward to instructions.

[Deus-nsf]

it works both normally and through gamescope. really tired now. will post how to do this later :)

I will gladly use those instructions! After you're well rested of course :) As I managed thanks to your preview DLL and some help from the community to have an HDR signal in System Shock 2! This might be the path towards universal HDR support on SteamOS!

[EndlesslyFlowering]

took a while to figure out: it works both normally and through gamescope. really tired now. will post how to do this later :)

short update: the thing I am doing is not a good solution and should be avoided. there is a proper (not recommended) way to do it though. currently seeking help with that :)

edit: this may take a long while...

[Deus-nsf]

this may take a long while...

I'm not in a hurry, it was great to have a proof of concept so quick already :) Better take the time and do things properly :)

[beckkake]

took a while to figure out: it works both normally and through gamescope. really tired now. will post how to do this later :)

short update: the thing I am doing is not a good solution and should be avoided. there is a proper (not recommended) way to do it though. currently seeking help with that :)

edit: this may take a long while...

follow-up?

[Dark-Thoughts]

+1 for an update. I tried to use it for Enshrouded to fix its awful bloom / brightness issues but as far as I can tell it is using Vulkan exclusively. Unfortunately Goverlay currently also has VKBasalt support removed in their UI overhaul and I don't think there's any other UI helper for it, let alone an in-game UI, which makes it hard to work with too.

[EndlesslyFlowering]

Current working on other projects which should be done this week. Then I can get to this :)

[Kirit29]

Current working on other projects which should be done this week. Then I can get to this :)

any update on this?

[canceralp]

Any news? I am looking forward to jumping back to Linux for all my Vulkan games:)

[EndlesslyFlowering]

I am switching between 5 projects to not get burned out 🙃 I want go finish some other stuff first before getting to finalize this.

I have it working in Vulkan but only in Proton 8. Not sure what the issue is in other Proton versions. Also the DirectX part of the script is not working atm (I rewrote it already but haven't tested it yet). I also haven't looked into Wine at all yet.

[EndlesslyFlowering]

you can find the current state here: https://github.com/EndlesslyFlowering/reshade-steam-proton

currently I am only focusing on the reshade-steam-proton script:

• you need to be on Proton 8.0 or later! I have only tested official Proton versions from Valve (Proton 8.0-x and Proton 9.0-x) and not GE variants. I have had reports of GE not working, so take that as is
• you need protontricks either via flatpak or normally installed (the script checks if it's available)
• the script is also incomplete because I added some features and did not implement them yet for DirectX games (I actually implemented them already but did not yet test them...)

[EndlesslyFlowering]

DirectX games should work again. I did some testing and everything works.

[Kuviot]

Were you able to get Reshade working for linux/proton games on steam that are vulkan games? I'm trying to do it on Doom Eternal but I don't know where to start

[EndlesslyFlowering]

Were you able to get Reshade working for linux/proton games on steam that are vulkan games? I'm trying to do it on Doom Eternal but I don't know where to start

Yes Proton only for now. Script asks you if it is a Vulkan game.

[d82mc]

I got Reshade to work on Cemu through Lutris by modifying the script a bit.

[Ro-Ly]

d82mc

Hey mate, could you share the instructions please?

[d82mc]

@Ro-Ly

Oi mate,

I went ahead and reworked the script to work more intuitively for Lutris/Wine/Vulkan games like Cemu.

https://github.com/d82mc/reshade-lutris/blob/main/reshade-lutris.sh

Instructions are in the description block in that script.

It will even support depth buffers for things like SSR and DisplayDepth. It should "just work", but let me know if you run into issues.

[Ro-Ly]

@d82mc Thank you so much mate. Unfortunately didn't got it to work yet. Tried to install it with Lutris runner and Steam runner, in both cases installation gets stuck on the step of the Vulkan installation and staying there with no progress.

[unhappy-ending]

It's actually pretty easy to do this manually.

1. Download and install VulkanSDK from https://vulkan.lunarg.com/sdk/home.
2. Download the Vulkan Runtime from https://vulkan.lunarg.com/sdk/home. It's right below the SDK link. You can choose the installer or a stand alone zip file. IMO zip file is easier because you can extract vulkan-1.dll files to a folder and later symlink to your game.exe folders, and makes for easy updates in the future.
3. Download and unzip ReShade to $HOME/.reshade. I like to make a 32 bit and 64 bit folder and separate the different architectures so I can create symlinks to different reshade .dlls, but you do what you like. Copy the appropriate versions to each.
4. Run this command: WINEPREFIX=/$HOME/.local/share/Steam/steamapps/compatdata/$SteamAppId/pfx wine reg add "HKLM\Software\Khronos\Vulkan\ImplicitLayers" /v Z:\\home\\$USER\\.reshade\\64\\ReShade64.json /t REG_DWORD /d 0 /reg:64
5. Run this command: WINEPREFIX=/$HOME/.local/share/Steam/steamapps/compatdata/$SteamAppId/pfx wine reg add "HKLM\Software\Khronos\Vulkan\ImplicitLayers" /v Z:\\home\\$USER\\.reshade\\32\\ReShade32.json /t REG_DWORD /d 0 /reg:32
6. Add a vulkan-1.dll override to wine. Do it via winecfg or a better way is to add it to a user env VARIABLE via $HOME/.profile. This allows the override to be used for all wine prefixes on the system so you don't have to do each one individually. echo 'export WINEDLLOVERRIDES="vulkan-1=n,b"' >> $HOME/.profile
7. Copy the vulkan-1.dll files downloaded during step 2 to your game's directory that contains the game.exe. Make sure you have a 32 bit version for 32 bit games, and a 64 bit version for 64 bit games. Do not try renaming ReShade64.dll to vulkan-1.dll, it's the same as having to supply a copy of d3dcompiler_47.dll to the game's folder when using wine/proton.
8. Copy your ReShade.ini into the folder with vulkan-1.dll.

That should be all. The only PITA is having to add the registry entry to each game prefix. I wish Steam would allow a single prefix for all games to make this kind of stuff easier. Oh right, and of course, make sure you use the right location and SteamAppId for the WINEPREFIX. That's on you to provide.

If you are using regular wine for non-Steam games, then you only have to do it once and never touch it again.

Happy Vulkan!

[EndlesslyFlowering]

It's actually pretty easy to do this manually.

1. Download and install VulkanSDK from https://vulkan.lunarg.com/sdk/home

2. Download and `unzip` ReShade to $HOME/.reshade. I like to make a 32 bit and 64 bit folder and separate the different architectures so I can create symlinks to different reshade .dlls, but you do what you like. Copy the appropriate versions to each.

3. Run this command: `WINEPREFIX=/$HOME/.local/share/Steam/steamapps/compatdata/$SteamAppId wine reg add "HKLM\Software\Khronos\Vulkan\ImplicitLayers" /v Z:\\home\\$USER\\.reshade\\64\\ReShade64.json /t REG_DWORD /d 0 /reg:64`

4. Run this command: `WINEPREFIX=/$HOME/.local/share/Steam/steamapps/compatdata/$SteamAppId wine reg add "HKLM\Software\Khronos\Vulkan\ImplicitLayers" /v Z:\\home\\$USER\\.reshade\\32\\ReShade32.json /t REG_DWORD /d 0 /reg:32`

5. Add a `vulkan-1.dll` override to wine. Do it via `winecfg` or a better way is to add it to a user `env VARIABLE` via `$HOME/.profile`. This allows the override to be used for all wine prefixes on the system so you don't have to do each one individually. `echo 'export WINEDLLOVERRIDES="vulkan-1=n,b"' >> $HOME/.profile`

6. Find a copy of `vulkan-1.dll`. You can get it from dll-files.com. Make sure you copy it to your game's directory that contains the game.exe. Make sure you have a 32 bit version for 32 bit games, and a 64 bit version for 64 bit games. _Do not try renaming `ReShade64.dll` to `vulkan-1.dll`_, it's the same as having to supply a copy of d3dcompiler_47.dll to the game's folder when using wine/proton.

7. Copy your `ReShade.ini` into the folder with `vulkan-1.dll`.

That should be all. The only PITA is having to add the registry entry to each game prefix. I wish Steam would allow a single prefix for all games to make this kind of stuff easier. Oh right, and of course, make sure you use the right location and SteamAppId for the WINEPREFIX. That's on you to provide.

If you are using regular wine for non-Steam games, then you only have to do it once and never touch it again.

Happy Vulkan!

About 6 and 7: You should install the latest Vulkan Runtime from here: https://vulkan.lunarg.com/sdk/home to not cause issues with random dlls downloaded from the internet which are likely to be outdated or are malware. The Vulkan loader from Wine has layer loading disabled since layer loading should be done on the host side. Installing the normal runtime will give you layer loading on the guest (Wine/Proton) side back.

You can see how I do it in the script here: https://github.com/EndlesslyFlowering/reshade-steam-proton/blob/02306c89c7fd71350c78e2a09389159fe3ddc915/reshade-steam-proton.sh#L442 note the "execution order matters" comment which took a while to figure out why I could not always get it to work

[unhappy-ending]

I already had the latest VulkanSDK in the very first step?

Does your script pull in vulkan-1.dll from a source? The VulkanSDK doesn't have it (thought it does provide vulkan-1.lib) and I can't get ReShade to work on Vulkan games without the vulkan-1.dll file in the game directory.

Edit: I'm derpin'. The runtime package on the SDK page has vulkan-1.dll, and it gets installed to your WINEPREFIX when you run it. How did I miss that?

[d82mc]

@d82mc Thank you so much mate. Unfortunately didn't got it to work yet. Tried to install it with Lutris runner and Steam runner, in both cases installation gets stuck on the step of the Vulkan installation and staying there with no progress.

@Ro-Ly

Seems like a problem in your wine prefix or runner. Did you use specifically Lutris-7.2-2-x86_64 and a new wine prefix? Try disabling all the extra flags in your lutris configuration

Haven't been able to reproduce with a new wine prefix on any runner.

Anyway you just need a runner that can handle the two installer exes for vulkan and reshade.

[Ro-Ly]

@d82mc Hey mate. I tried two kinds of runners, Proton 8-4 and latest Wine. Maybe the problem is in wineprefix which was generated by Lutris, but it was generated specifically for the runner I selected for the installer of reshade.

I got the idea now. If Reshade doesn't work because it requires the Vulkan, maybe it's a way to install the Reshade using old installer, then download the Vulkan dll manually and drop it inside the exe folder and add manually the winedlloverrides variable into the Steam/Lutris shortcut? Do I need to install Vulkan fully in this way or DLL would be enough?

[EndlesslyFlowering]

I already had the latest VulkanSDK in the very first step?

Does your script pull in vulkan-1.dll from a source? The VulkanSDK doesn't have it (thought it does provide vulkan-1.lib) and I can't get ReShade to work on Vulkan games without the vulkan-1.dll file in the game directory.

Edit: I'm derpin'. The runtime package on the SDK page has vulkan-1.dll, and it gets installed to your WINEPREFIX when you run it. How did I miss that?

I should have mentioned that the Vulkan loader is vulkan-1.dll :)

[unhappy-ending]

I got the idea now. If Reshade doesn't work because it requires the Vulkan, maybe it's a way to install the Reshade using old installer, then download the Vulkan dll manually and drop it inside the exe folder and add manually the winedlloverrides variable into the Steam/Lutris shortcut? Do I need to install Vulkan fully in this way or DLL would be enough?

You need the VulkanSDK installed to your WINEPREFIX and a proper 32/64bit copy of vulkan-1.dll in the game's folder. They're both needed at the moment.

For WINEDLLOVERRIDES you can either use them via .profile like I posted earlier or you can add to the game's command line WINEDLLLOVERRIDES=vulkan-1=n,b %command% and it will launch the game with the proper variable. You want it there and not when launching Steam itself.

Also, you should be trying this on vulkan versions of games. I can't get it to work on DXVK so if games are using d3dx you need to rename reshade to the proper dll and drop it in the game folder with a copy of d3dcompiler_47.dll. You'll also want WINEDLLOVERRIDES=d3dcompiler_47=n,b. This is not necessary for a Vulkan game like Doom 2016, just things that use DXVK.

[Ro-Ly]

unhappy-ending Is there an possibility to install the Vulkan SDK into the default Steam prefix to have enable it globally for all the non-Steam apps?

[unhappy-ending]

I already had the latest VulkanSDK in the very first step? Does your script pull in vulkan-1.dll from a source? The VulkanSDK doesn't have it (thought it does provide vulkan-1.lib) and I can't get ReShade to work on Vulkan games without the vulkan-1.dll file in the game directory. Edit: I'm derpin'. The runtime package on the SDK page has vulkan-1.dll, and it gets installed to your WINEPREFIX when you run it. How did I miss that?

I should have mentioned that the Vulkan loader is vulkan-1.dll :)

lol yeah I figured that out. I appreciate your script because it got me started, the thing I was missing from it was the runtime distinction from the SDK. Unfortunately it looks like I also need some stuff from the SDK, the explicit layers? I'll have to try to pinpoint it so I don't need to install the SDK on everything.

I noticed on Windows it seems you can use DXVK and then Vulkan ReShade on top of it, but that doesn't seem to work on Linux. I'm guessing because on Windows it's going DX > Windows Vulkan > Vulkan ReShade and on wine it's going DX > Linux Vulkan so it can't use Vulkan ReShade? Am I wrong? I'd be ecstatic if I could use Vulkan ReShade on top of DXVK.

[unhappy-ending]

unhappy-ending Is there an possibility to install the Vulkan SDK into the default Steam prefix to have enable it globally for all the non-Steam apps?

Nope :(

Edit: Oh, non-Steam apps? I don't know, I'd have to test it. I tested a Steam app but not a non-Steam app.

I just tested it. I forgot, Steam creates new WINEPREFIX for non-steam games and they're in $HOME/.local/share/Steam/steamapps/compatdata/ with a much longer string of numbers, for example 3406853318. So no, no global Proton options.

Yet another reason I wish I could just use my system wine globally through the Steam launcher.