kevinlekiller / reshade-steam-proton

Easy setup and updating of ReShade on Linux for games using wine or proton.
GNU General Public License v2.0
361 stars 19 forks source link

Install Reshade for a Vulkan game #6

Open rylydou opened 2 years ago

rylydou commented 2 years ago

I am installing Reshade into Cemu which uses Vulkan. I know that on Windows you can install Reshade on a Vulkan game. But I don't know what dll name to use or if it is even possible.

I have tried:

kevinlekiller commented 2 years ago

I'm not sure, never used Cemu.

While doing a quick search I found this: https://i.imgur.com/o8nwDkd.jpg linked here https://teddit.net/r/CemuPiracy/comments/ey6ien/is_there_any_tutorial_for_reshade_on_cemu_botw/

Does that mean the games need patches for Reshade to function?

Looking at this video at 4m50s, there's no dll file in this Cemu folder, I'm guessing Reshade does something different to hook into games using Vulkan?

kevinlekiller commented 2 years ago

ReShade does not support DLL wrapping in Vulkan. Vulkan has a layering mechanism that ReShade has to use to be spec conform. And that requires ReShade to be in a central place, because those layers are configured via system-wide registry keys. You are best off using the setup tool to set up ReShade, it knows what to do to make it work.

https://reshade.me/forum/troubleshooting/6153-reshade-and-vulkan

I'm guessing if we figure out which registrey keys are added, then we could try to do that programatically?

kevinlekiller commented 2 years ago

For now, you could use vkBasalt which supports most Reshade shaders, only issue is there's no in game GUI.

kevinlekiller commented 2 years ago

This post explains how the installer sets up the registry:

https://reshade.me/forum/troubleshooting/6153-reshade-and-vulkan#41351

rylydou commented 2 years ago

Thanks, I'll give vkBasalt with goverlay a try. Thanks for the help!

kevinlekiller commented 2 years ago

Not tested in a game, but I've got something that might work if you want to test.

The WINEPREFIX has to be set for the game in particular and the ReShade64.json needs to be changded to ReShade32.json if the program is 32 bits, the reg:64 also has to be changed to reg:32 if it's a 32 bit prefix.

WINEPREFIX="$HOME/Games/windows/lutris/Test/wineprefix" wine reg ADD HKLM\\SOFTWARE\\Khronos\\Vulkan\\ImplicitLayers /d 0 /t REG_DWORD /v "Z:\\home\\$USER\\.reshade\\reshade\\ReShade64.json" -f /reg:64

Here's regedit in a test WINEPREFIX: https://i.imgur.com/80KiVki.png

rylydou commented 2 years ago

I've have tried doing the registry edit but I don't have either of the ReShade dlls in ~/.reshade/reshade. I do have the json files that point to them though.

kevinlekiller commented 2 years ago

That's a good point, the dxgi.dll would have to be linked to ReShade64.dll and the d3d9.dll to ReShade32.dll

I'll add those things to the script, will have to find a Vulkan game to test on.

kevinlekiller commented 2 years ago

Half life 2 is 32 bit and has Vulkan support : https://www.pcgamingwiki.com/wiki/Half-Life_2#API

Doom 2016 64 bit / Vulkan : https://www.pcgamingwiki.com/wiki/Doom_(2016)#API

Will test these this week and update the script.

kevinlekiller commented 2 years ago

I've been testing on Doom, but not having any success, the registry entry is correctly added, but the game doesn't launch with Reshade.

Added the changes to a different branch:

https://github.com/kevinlekiller/reshade-steam-proton/commit/8275417a3e0d4c18ee1042d5c0c8675b33b13fac

kevinlekiller commented 2 years ago

In a Windows VM I installed Reshade to make sure the Registry matches what's in wine:

Screenshot_20220118_155049

So the issue is somewhere else.

kevinlekiller commented 2 years ago

The issue might be with Reshade itself:

https://reshade.me/forum/suggestions/7074-linux-vulkan-opengl-version#40598

This unlikely to happen anytime soon, since large parts of the ReShade code base are Windows specific (especially the hooking code). Only thing that could potentially be separated out would be the Vulkan layer implementation, but that is already partially covered by vkBasalt so it doesn't feel worth the effort right now.

Edit: Although, (in our case) it's through Wine, but perhaps there's some Windows specific Vulkan stuff that's not available on Linux so Wine can't do anything about it?

kevinlekiller commented 2 years ago

In ReShade 5.0, there are some changes to how the json files are handled.

Here's the relevant section:

https://github.com/crosire/reshade/blob/67f4f718614800b426f20291b2fc7d7ca6644f37/setup/MainWindow.xaml.cs#L711

canceralp commented 2 years ago

Would it be possible to install Reshade 4.9.1 with global Vulkan enabling option? This is better than nothing for now.

kevinlekiller commented 2 years ago

Not sure, from what I can remember it doesn't function at all. Busy with work and other stuff at the moment, so not enough time to test, unfortunately.

canceralp commented 2 years ago

I tried 4.9.1, 5.0.2 and 5.2.1. None works. I tried them on Valheim and RDR 2. Valheim can also work on Dx11 so it is possible to make the installation for it but RDR 2 has only Vulkan for Linux (Dx12 freezes the PC).

I have read somewhere Wine runs all the games in administrator mode. I also know that Reshade Vulkan doesn't load when the game is run with admin privileges. Could that be the problem?

canceralp commented 1 year ago

It is not about Vulkan games but there is good news from Crosire. He fixed a problem which prevented newer Reshade versions to work with DirectX 12 games. I hope he might look into Vulkan problem as well.

SirSAC commented 10 months ago

Hello, try vulkan-1.dll and let me know if worked, bye. Edit: Also download the ReShade executable and extract it with 7Zip and see if is there any DLLs, if is some named ReShade32.dll and/or ReShade64.dll then rename one of them as vulkan-1.dll the JSON file is useless or it should be, otherwise if is needed just edit the JSON file and replace the ReShade(32/64).dll with vulkan-1.dll and this should work.

canceralp commented 9 months ago

Hello, try vulkan-1.dll and let me know if worked, bye. Edit: Also download the ReShade executable and extract it with 7Zip and see if is there any DLLs, if is some named ReShade32.dll and/or ReShade64.dll then rename one of them as vulkan-1.dll the JSON file is useless or it should be, otherwise if is needed just edit the JSON file and replace the ReShade(32/64).dll with vulkan-1.dll and this should work.

I won't have access to a Linux PC couple more days. If anyone tried it, could they please report if it works?

screwylightbulb commented 9 months ago

Hello, try vulkan-1.dll and let me know if worked, bye. Edit: Also download the ReShade executable and extract it with 7Zip and see if is there any DLLs, if is some named ReShade32.dll and/or ReShade64.dll then rename one of them as vulkan-1.dll the JSON file is useless or it should be, otherwise if is needed just edit the JSON file and replace the ReShade(32/64).dll with vulkan-1.dll and this should work.

I won't have access to a Linux PC couple more days. If anyone tried it, could they please report if it works?

I tried this on Baldur's Gate 3 on my Linux machine where I do have Reshade working, but it did not work.

SirSAC commented 9 months ago

I tried too for Borderland 2 (DX9) and TerraTech (DX11) and won't work, so sorry, it was just an idea, but works only if I run the setup and select the game executable and telling the setup is running on Vulkan, is creating "ReShade32.dll" or "ReShade64.dll" but not the JSON file in the game folder and something else is creating in "C:\ProgramData" that contain the original JSON file and maybe something in registry but I'm not able yet to track what is it or if is something added/changed. My test PC with Linux died :( sorry if I'm not able to provide more details, I just give what I remember from the Linux machine.

PS: try to run the setup of ReShade and give the game executable and select Vulkan for runtime, maybe this will work if setup takes the details correctely.

EndlesslyFlowering commented 6 months ago

took a while to figure out: doom_reshade it works both normally and through gamescope. really tired now. will post how to do this later :)

canceralp commented 6 months ago

This is early new year:) I am (and many more, I'm sure) looking forward to instructions.

Deus-nsf commented 6 months ago

it works both normally and through gamescope. really tired now. will post how to do this later :)

I will gladly use those instructions! After you're well rested of course :) As I managed thanks to your preview DLL and some help from the community to have an HDR signal in System Shock 2! This might be the path towards universal HDR support on SteamOS!

EndlesslyFlowering commented 6 months ago

took a while to figure out: doom_reshade it works both normally and through gamescope. really tired now. will post how to do this later :)

short update: the thing I am doing is not a good solution and should be avoided. there is a proper (not recommended) way to do it though. currently seeking help with that :)

edit: this may take a long while...

Deus-nsf commented 6 months ago

this may take a long while...

I'm not in a hurry, it was great to have a proof of concept so quick already :) Better take the time and do things properly :)

beckkake commented 4 months ago

took a while to figure out: doom_reshade it works both normally and through gamescope. really tired now. will post how to do this later :)

short update: the thing I am doing is not a good solution and should be avoided. there is a proper (not recommended) way to do it though. currently seeking help with that :)

edit: this may take a long while...

follow-up?

NaturalHate commented 4 months ago

+1 for an update. I tried to use it for Enshrouded to fix its awful bloom / brightness issues but as far as I can tell it is using Vulkan exclusively. Unfortunately Goverlay currently also has VKBasalt support removed in their UI overhaul and I don't think there's any other UI helper for it, let alone an in-game UI, which makes it hard to work with too.

EndlesslyFlowering commented 4 months ago

Current working on other projects which should be done this week. Then I can get to this :)

Kirit29 commented 2 months ago

Current working on other projects which should be done this week. Then I can get to this :)

any update on this?

canceralp commented 1 month ago

Any news? I am looking forward to jumping back to Linux for all my Vulkan games:)

EndlesslyFlowering commented 1 month ago

I am switching between 5 projects to not get burned out 🙃 I want go finish some other stuff first before getting to finalize this.

I have it working in Vulkan but only in Proton 8. Not sure what the issue is in other Proton versions. Also the DirectX part of the script is not working atm (I rewrote it already but haven't tested it yet). I also haven't looked into Wine at all yet.

EndlesslyFlowering commented 1 month ago

you can find the current state here: https://github.com/EndlesslyFlowering/reshade-steam-proton

currently I am only focusing on the reshade-steam-proton script:

EndlesslyFlowering commented 1 month ago

DirectX games should work again. I did some testing and everything works.

Kuviot commented 2 weeks ago

Were you able to get Reshade working for linux/proton games on steam that are vulkan games? I'm trying to do it on Doom Eternal but I don't know where to start

EndlesslyFlowering commented 6 days ago

Were you able to get Reshade working for linux/proton games on steam that are vulkan games? I'm trying to do it on Doom Eternal but I don't know where to start

Yes Proton only for now. Script asks you if it is a Vulkan game.