search

ValveSoftware / Proton

Compatibility tool for Steam Play based on Wine and additional components
Other
23.85k stars 1.04k forks source link

DualSense advanced features compatibility #5900

Open ClearlyClaire opened 2 years ago

ClearlyClaire commented 2 years ago

DualSense feature compatibility

EDIT: the patches at https://github.com/ClearlyClaire/wine/commits/proton-wine-9.0-1%2Bdualsense/ (those marked as “merged” have been merged in the development version of Wine, I don't know if they have reached any version of proton yet) allow using DualSense-specific features (Adaptive triggers, speaker, VCM-based haptics) in at least:

All features require the DualSense to be plugged through USB before starting the game and Steam Input to be disabled for Playstation controllers (or simply Steam Input being disabled for the specific game). I have not found hid-playstation to cause any issue (apart from registering additional events on the touchpad which could interfere in some games).

Technical details

All DualSense features are implemented directly in the game application, with no additional driver. However, the games' implementation rely on low-level APIs that may not be implemented in Wine or details that may be different on a typical Linux environment.

Adaptive triggers work through custom HID reports, so this will work out of the box as long as Steam Input is disabled.

Speaker and advanced haptics rely on a 4-channel audio device, and the difficulties lie in games finding and selecting that audio output. Various games use various means to do it, but from what I have seen, it mainly boils down to one or more of these:

I have written a sample test application to sum up these findings and test Wine/Proton against those behaviors without requiring the games: https://github.com/ClearlyClaire/dualsense-games-compat-check

ClearlyClaire commented 2 years ago

I had an informal conversation with someone at Audiokinetic, and they have helpfully confirmed that most games using Wwise will either search for a device called “Wireless Controller” or use the containerId obtained from a handle to the DualSense gamepad, and finding an IMMDevice with a matching containerId (via the IMMDevice's property store). That can happen inside a Wwise library or in game-specific code (or both).

They also pointed out that some games with earlier implementation of DualSense support might behave differently, so it could be worth confirming if the changes I have discussed so far are enough to get them working properly:

Finally, while the person at Audiokinetic has been kind enough to confirm those details about the current implementation in Wwise, it does not mean that Audiokinetic will actively put any effort into getting games to work with Proton/Wine or commit to providing any kind of support.

ClearlyClaire commented 2 years ago

In other news, I have submitted some of my patches upstream and have some idea on how to properly implement the containerId stuff but that would still require a lot of work. Basically, one would need to get the sysfs path for the devices and walk up to find a “removable” device (see https://docs.microsoft.com/en-us/windows-hardware/drivers/install/how-container-ids-are-generated-from-the-removable-device-capability and https://www.kernel.org/doc/Documentation/ABI/testing/sysfs-devices-removable)

Finally, I have been informed that my solution for FF7R and FF14 won't currently work if you are using pipewire-pulse because the exposed properties are not exposed quite in the same way: https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2485

Hadrianneue commented 2 years ago

Interesting. I'm surprised haptics would work at all without any patch or tweaks, but maybe it uses the compatibility mode rather than advanced haptics? You could check if it's outputting anything to the DualSense audio device.

EDIT: and i'm wrong yet again, nothing on pavumeter....

@ClearlyClaire

In other news, my patch to rename the audio device to “Wireless Controller” has been accepted in Wine.

that is awesome :D

i did build proton with your patches, problem is that whenever i build it myself, be it proton, proton-ge, wine-ge, etc, it always forces xinput on for whatever reason, both Death Stranding/Ghostwire: Tokyo only sees a xinput device, and because of that even the adaptive triggers won't work, so i can't test it on my end :(

original post: F1 2021 has both adaptive triggers and haptic feedback working without any tweaking whatsoever besides disabling steam input (apparently using libscepad), however F1 Series also have a 'Radio Output Device' option to output the race engineer comms, it does let you set it to the wireless controller audio output but it goes through my headset nonetheless... following forum reports however that seems to be a common issues with previous F1 installments...

ClearlyClaire commented 2 years ago

Interesting. I'm surprised haptics would work at all without any patch or tweaks, but maybe it uses the compatibility mode rather than advanced haptics? You could check if it's outputting anything to the DualSense audio device.

In other news, my patch to rename the audio device to “Wireless Controller” has been accepted in Wine. The one I submitted for the appropriate mix format to be returned is pending, and I'm still trying to write a proper one for the containerId thing (this is way more involved than the other patches, and I think I will only be able to deliver something that works when using udev in combination with either winepulse.drv or winealsa.drv as audio backend).

Bitwolfies commented 2 years ago

Replying to https://github.com/ValveSoftware/Proton/issues/5900#issuecomment-1172038570

Fantastic to hear, and it looks like the pw team is looking into fixing pulse too. Though im surprised wine accepted it, don't they not really like specific hacks?

Bitwolfies commented 2 years ago

Looks like the PW fix was merged, those guys are fast

ClearlyClaire commented 2 years ago

I'm still working on proper support for the ContainerID stuff (required by Ghostwire: Tokyo and most probably most of the other games, see https://github.com/ValveSoftware/Proton/issues/5900#issuecomment-1170775313), but I think I have figured it out, and I have started submitting parts of it to Wine.

When all the patches are merged upstream, I will backport them for latest proton, but for the time being, the patches in https://github.com/ValveSoftware/Proton/issues/5900#issuecomment-1167700297 should do the trick as long as you use a single DualSense controller at a time.

i did build proton with your patches, problem is that whenever i build it myself, be it proton, proton-ge, wine-ge, etc, it always forces xinput on for whatever reason, both Death Stranding/Ghostwire: Tokyo only sees a xinput device, and because of that even the adaptive triggers won't work, so i can't test it on my end :(

I am sorry I'm unable to help you with that :/

and it looks like the pw team is looking into fixing pulse too

Indeed, when I started discussing my plans for DualSense support on the Wine development channel, someone mentioned possible incompatibilities with pipewire-pulse, and when we identified them, someone quickly stepped up to actually fix them!

ClearlyClaire commented 2 years ago

Took advantage of the sales to buy Deathloop, as it was listed by the Audiokinetic person as one of the games using an earlier implementation of DualSense support in Wwise. It uses the containerId to find the audio device, and my patches are enough to get it to use the audio-based haptic feedback perfectly.

However, the controller's speaker isn't used at all, while on Windows it is used for various sound effects and radio chatter. I'm not sure why that is. It's not a GetMixFormat issue, as the Windows box I tried it on returns exactly the same formats and timings as my patched Proton. I suspect more SetupApi is involved, but I haven't investigated much yet.

None of my new patches has been merged upstream so far, but some of them got improvements. In any case, I'm going to regularly rebase them on proton's fork of Wine here: https://github.com/ClearlyClaire/wine/tree/features/dualsense

Bitwolfies commented 2 years ago

Hopefully valve pays attention to your work, would be a nice thing to see DS5 features supported in the next patch notes.

ClearlyClaire commented 2 years ago

Progress update: all patches necessary for FF7R and FF14 have been merged in upstream Wine! Those necessary for Ghostwire: Tokyo have not been merged, but I have slightly improved them.

As for Deathloop's use of the speaker, I believe I have figured everything that was missing from Wine, but unfortunately this will be a lot of work, and I have no patch to show for this. Basically it involves:

Excerpt of Deathloop trace regarding speaker audio initialization: https://gist.github.com/ClearlyClaire/2a185be39755dd3f30058fd8ee405230

I also have started writing a sample program that attempts to replicate the various games' logic regarding DualSense audio selection in order to more easily test them without needing to buy, install or run large games (a DualSense is still required though): https://github.com/ClearlyClaire/dualsense-games-compat-check

ClearlyClaire commented 2 years ago

I ended up figuring out a way to make Deathloop use the speaker properly, not by adding code but by… disabling XAudio2. When the xaudio2_8 and xaudio2_9 DLLs are disabled (or set to “native”, for some reason it does not make a difference, even though they are in the game's prefix), Deathloop happily uses audio anyway, and it does not need the audio devices to be present in SetupApi. For the speaker, it uses the audio device it has detected using ContainerID.

I'm not sure this workaround would work for other games, but at least it works for Deathloop for some reason.

Bitwolfies commented 2 years ago

I ended up figuring out a way to make Deathloop use the speaker properly, not by adding code but by… disabling XAudio2. When the xaudio2_8 and xaudio2_9 DLLs are disabled (or set to “native”, for some reason it does not make a difference, even though they are in the game's prefix), Deathloop happily uses audio anyway, and it does not need the audio devices to be present in SetupApi. For the speaker, it uses the audio device it has detected using ContainerID.

I'm not sure this workaround would work for other games, but at least it works for Deathloop for some reason.

What are the other games that use the deathloop style of detection?

ClearlyClaire commented 2 years ago

I don't know. It's the only one I encountered so far, but for obvious reasons I haven't tested every game with DualSense features out there.

Hopefully there aren't a lot of games working like this as implementing it correctly sound like a massive amount of work.

Mutcholoko commented 1 year ago

Finally got it to work! Thanks for the patch files :)

ClearlyClaire commented 1 year ago

Sorry, I should probably edit the first post to make clear that the latest and most complete patchset is available at https://github.com/ClearlyClaire/wine/tree/features/dualsense. I have been busy with other stuff lately (and I still am), so I haven't been able to rebase them on latest proton, nor rewrite the last patches for inclusion upstream.

As far as I know, this should work with PipeWire indeed, as long as it's a recent enough version, but I haven't tried.

ClearlyClaire commented 1 year ago

I have rebased the changes on proton 7.0-4, there were no conflicts: https://github.com/ClearlyClaire/wine/tree/proton-wine-7.0-4+dualsense ; everything still works in at least FF7R, Deathloop and Ghostwire: Tokyo.

I have also bought Death Stranding Director's Cut, but could not get the haptics to work. The game enumerates the audio devices and queries both the FriendlyName and the ContainerId properties for every device. It does not stop the enumeration at the DualSense output and does not seem to attempt to select and open any of these outputs for haptics feedback. I am unsure why. Possibly of interest, it enumerates HID devices right after enumerating audio devices, and the following message appears, that does not appear with other games:

013c:fixme:combase:RoGetActivationFactory (L"Windows.Gaming.Input.Gamepad", {8bbce529-d49c-39e9-9560-e47dde96b7c8}, 000000000041EA68): semi-stub
Bitwolfies commented 1 year ago

I'm no expert on this but its trying to query Windows.Gaming.Input of all things? Valve added support for it not too long ago, but that only applies to Xbox One/One S/Series X controllers. So I somehow don't think it would be related to the DS5 haptics.

In theory it could be trying to work with the impulse triggers on the Xbox controllers, they were never added to Xinput, only Windows.Gaming.Input. PCGamingWiki says the game does in fact use them.

ClearlyClaire commented 1 year ago

I don't know this API but I agree with you, this does not seem to be directly related. Still, I don't understand yet why the game does not attempt to open the DualSense audio output (or any second audio output for that matter). I'm also not sure where the game is supposed to use adaptive triggers, so maybe it doesn't decide to use DualSense features at all for some unknown reason, but it at least does reset them on startup.

Bitwolfies commented 1 year ago

I don't know this API but I agree with you, this does not seem to be directly related. Still, I don't understand yet why the game does not attempt to open the DualSense audio output (or any second audio output for that matter). I'm also not sure where the game is supposed to use adaptive triggers, so maybe it doesn't decide to use DualSense features at all for some unknown reason, but it at least does reset them on startup.

Triggers should be used nearly every time you adjust your cargo, so basically all the time, Steam Input must be disabled and ect for it to work.

As for audio, the game does have a feature where you can send BB audio to any speaker, though its meant specifically for PS controllers, as all the audio BB has in the PS versions comes out of it. You have to manually enable it, but in theory its unrelated to the haptic speakers.

ClearlyClaire commented 1 year ago

Triggers should be used nearly every time you adjust your cargo, so basically all the time, Steam Input must be disabled and ect for it to work.

ok, I have only played a few minutes so I might just not have picked up heavy enough cargo, but I haven't noticed any use of the adaptive triggers when picking up the first few items or balancing myself in the river. So I guess the game may decide to not use the DualSense features at all for some reason? But Steam Input is disabled, and in addition to resetting the adaptive triggers on startup, the game is properly handling input (including different actions for the left and right side of the touchpad) and displaying proper button labels, so I'm not sure what may be going on.

EDIT: Got to a point of the game where you have heavier cargo, and the adaptive triggers are definitely used.

As for audio, the game does have a feature where you can send BB audio to any speaker, though its meant specifically for PS controllers, as all the audio BB has in the PS versions comes out of it. You have to manually enable it, but in theory its unrelated to the haptic speakers.

Yeah I'd expect haptics to work without changing the setting for the speaker output (even though they would technically be on the same output device). I tried with default audio out and with the DualSense out specifically, but neither results in haptics feedback. Without selecting the DualSense audio output for the BB audio, the game doesn't open the DualSense audio output at all.

Hadrianneue commented 1 year ago

013c:fixme:combase:RoGetActivationFactory (L"Windows.Gaming.Input.Gamepad", {8bbce529-d49c-39e9-9560-e47dde96b7c8}, 000000000041EA68): semi-stub

CodeWeaver's Rémi Bernon (and yourself) are the only ones who submitted patches for this api/dualsense support, probably just some stubbed function (or even just UUID check failling).

ClearlyClaire commented 1 year ago

I've switched away from PulseAudio to PipeWire, and noticed another discrepancy. I updated the patches in https://github.com/ClearlyClaire/wine/tree/proton-wine-7.0-4+dualsense and https://github.com/ClearlyClaire/wine/commits/features/dualsense to account for that.

I have made no progress with regards to Death Stranding Director's Cut.

ClearlyClaire commented 1 year ago

Tried “Sackboy: A Big Adventure”, and the audio-based haptics don't work. What's odd is that whenever a DualSense is plugged in, I can see the game enumerating the audio devices in a loop, including querying their FriendlyName, but not their container ID. I'm not sure what is going on.

Bitwolfies commented 1 year ago

Just to clarify, when you say does not work, are we referring just to the rumble, or the triggers as well?

ClearlyClaire commented 1 year ago

Just to clarify, when you say does not work, are we referring just to the rumble, or the triggers as well?

VCM-based rumble, as usual. Everything else is going through the HID device, and while I haven't seen the triggers yet I'm confident it'd work, as gyro works.

EDIT: Played a bit to get to the point where adaptive triggers are used (throwing an acorn), and I can confirm they are properly used.

Mutcholoko commented 1 year ago

Just wanna say that I've tested your patches and both Ghostwire: Tokyo and Uncharted 4 worked perfectly. I have even created a repo with a build of Proton including those patches (and also a guide on how to build Proton with your patches, Claire). You are doing an amazing job, and if you want someone to test this and get this further investigated, you can count on me :)

ClearlyClaire commented 1 year ago

Just an update: I am very busy and have very little time to work on this. I have made no progress whatsoever about Death Stranding or Sackboy.

I have tried Marvel's Spider-Man remastered, though, and my patches make the game crash whenever a DualSense is plugged in, because it tries to use a SetupAPI function that is not even stubbed in Wine: SetupDiGetDeviceInterfacePropertyW.

Stubbing this function makes the game fall back to more classic rumble. The method is used to query at least DEVPKEY_Device_InstanceId. I haven't tried to implement it yet.

EDIT: I implemented SetupDiGetDeviceInterfacePropertyW for DEVPKEY_Device_InstanceId specifically (which as I understand may be a more modern alternative to SetupDiGetDeviceInstanceIdW) to see if it got me further, but it's still not using the DualSense haptics.

ClearlyClaire commented 1 year ago

I'm happy to to report that the patches work flawlessly for Returnal! (though Returnal itself has an issue with hanging during shaders precompilation for some reason, see https://github.com/ValveSoftware/Proton/issues/6538)

Hadrianneue commented 1 year ago

I'm happy to to report that the patches work flawlessly for Returnal! (though Returnal itself has an issue with hanging during shaders precompilation for some reason, see #6538)

it works beautifully, i have a question however...

your patches seems to work only with proton and within steam, building against wine-ge, wine, or even trying to use proton on lutris simply doesn't enable haptic feedback...

the issue being that most if not all current dualsense supported games are available on other stores...

can you advice me on which change should i do when building with, for instance wine-ge, to make it work without steam?

ClearlyClaire commented 1 year ago

IIRC upstream Wine and Proton differ in that the former will try to use SDL for gamepad input unless instructed otherwise, and my patches do not implement everything needed when the gamepad is controlled through SDL. That distinction shouldn't apply to the Proton on Lutris configuration though. I'm not sure what else would cause issues.

(To disable SDL for gamepad input, search for “Enable SDL” in https://wiki.winehq.org/Useful_Registry_Keys)

Bitwolfies commented 1 year ago

Im curious, what's the status of your patches being merged upstream (and then presumably proton after a rebase)? They sadly seem to have flown under the wine teams radar.

ClearlyClaire commented 1 year ago

There hasn't been much progress on that, my patches are functionally ok but some of them are far from ideal. Unfortunately I'm not knowledgeable enough about wine's internals and windows APIs to really come up with something cleaner, and I'm a bit busy with my dayjob as well.

On February 16, 2023 10:38:22 AM GMT+01:00, Bitwolf @.***> wrote:

Im curious, what's the status of your patches being merged upstream (and then presumably proton after a rebase)? They sadly seem to have flown under the wine teams radar.

-- Reply to this email directly or view it on GitHub: https://github.com/ValveSoftware/Proton/issues/5900#issuecomment-1432792749 You are receiving this because you were mentioned.

Message ID: @.***>

MicaelJarniac commented 1 year ago

Could these patches be added to https://github.com/GloriousEggroll/proton-ge-custom in the meanwhile? I'm pretty clueless about how it all works, but as far as I can tell, proton-ge is basically a customized version of Proton with some additional features, so I'd imagine it could be a nice place to add these patches.

Also, what's the progress with Death Stranding Director's Cut? That's the main game I wanted to play with the haptics working.

ClearlyClaire commented 1 year ago

I'm not sure what ProtonGE's policy is for patches, but I can't see why they couldn't use them.

Also, what's the progress with Death Stranding Director's Cut? That's the main game I wanted to play with the haptics working.

Unfortunately, no. I have no idea what's missing, I'm not sure where to even start looking, and I haven't got any help with it either.

Roman-AE commented 1 year ago

@ClearlyClaire patches works great with Last of Us. Applied it to ProtonGE.

Bitwolfies commented 1 year ago

@ClearlyClaire patches works great with Last of Us. Applied it to ProtonGE.

You actually managed to get the port to work? Shits pegged at 30fps for me with the worlds slowest shader compile ever.

Roman-AE commented 1 year ago

@ClearlyClaire patches works great with Last of Us. Applied it to ProtonGE.

You actually managed to get the port to work? Shits pegged at 30fps for me with the worlds slowest shader compile ever.

Yes, it works out of the box. Slowest shader compilation ever (before for me it was Detroit).

ClearlyClaire commented 1 year ago

Quick update: with the release of Ratchet & Clank: Rift Apart, I wanted to give it a try. Seems like Proton 7.0 is unable to run it, so I had to rebase my work on Proton 8.0. Link to the build: https://github.com/ClearlyClaire/wine/releases/tag/experimental-wine-bleeding-edge-8.0-50838-20230727-p70694a-w9303d4-d4ed147-vdb6089%2Bdualsense

The good news is that the Proton 8.0 build appears to work on all games where my Proton 7.0 build worked.

The bad news is that my patches are not enough to have proper DualSense audio-based features to work in Ratchet & Clank: Rift Apart. Like other Nixxes-ported games, the game fails to use audio-based haptics and gracefully falls back to the “classic” vibrations.

A note on upstreaming: I have made no further effort to upstream my pending patches. Stock Wine / Proton 8.0 includes a number of my patches already, but those are not enough to bring proper support to any game. Furthermore, while some of those patches should have been enough for some games (FF7R and FFXIV), a bug in Wine 8.0 rendered it pointless. That bug is already fixed in the development version of wine (see https://gitlab.winehq.org/wine/wine/-/merge_requests/3045).

Bitwolfies commented 1 year ago

Replying to https://github.com/ValveSoftware/Proton/issues/5900#issuecomment-1656718656

Do you happen to have a list of which games do and dont work, and why they dont work? Just curious to see the progress made, thanks!

ClearlyClaire commented 1 year ago

Not really. The last few I tested are:

Bitwolfies commented 12 months ago

Interestingly, on the decks dev branch, with steam input disabled, the triggers work wirelessly. Tested with both ratchet and Clank and the last of us. Wonder if that's valves or Sony's doing.

pinkflames commented 12 months ago

Interestingly, on the decks dev branch, with steam input disabled, the triggers work wirelessly. Tested with both ratchet and Clank and the last of us. Wonder if that's valves or Sony's doing.

Odd. I do not believe it's supposed to be possible on PC when using the standard Bluetooth protocols. As such there's nothing Valve or the game could do about it without kernel level support for this, since the driver would need to expose something be it a custom HID or another interface.

Bitwolfies commented 12 months ago

Interestingly, on the decks dev branch, with steam input disabled, the triggers work wirelessly. Tested with both ratchet and Clank and the last of us. Wonder if that's valves or Sony's doing.

Odd. I do not believe it's supposed to be possible on PC when using the standard Bluetooth protocols. As such there's nothing Valve or the game could do about it without kernel level support for this, since the driver would need to expose something be it a custom HID or another interface.

They could very well be using the HID files now, I know Valve supposedly has it working with Steam input but turning that on made the wireless triggers stop working. We know Valve and Sony have some sorta contact what with the new dualsense tag for games and the updater being built into steam.

Bitwolfies commented 12 months ago

Update: manged to get it to connect to my arch machine, can confirm the triggers work there. Not sure if this works across all dualsense games, but it sure works there with ratchet and clank.

Pcgamingwiki mentions wireless compatibility on Ratchet and Clank, so this might be nothing, ill try it with ghostwire.

Bitwolfies commented 12 months ago

Ghostwire didnt even respond in wireless mode so I tried bugsnax. Sadly triggers only worked in wired mode, so this its not driver related and sony found a way to get the feature working wirelessly with new games only on the current driver.

I do hope Sony or a Third party can rework the dualsense driver on Linux to allow backwards compatable wireless functions, but I sadly doubt that's gonna happen and we'll be stuck with this growing pain of the games that came before.

ClearlyClaire commented 12 months ago

Triggers can be used over bluetooth, although using a slightly different format for the HID reports. You could rewrite the HID reports on the fly, which is actually what I did in https://github.com/ValveSoftware/Proton/issues/5900#issuecomment-1156822244, but that won't cover audio-based features. I don't know of any PC game or software that use the audio-based features wirelessly, and I don't know what protocol the PS5 uses for that. I am not actually sure this is doable over bluetooth, and it's possible you'd need specific hardware adapters for that.

pinkflames commented 12 months ago

For regulatory reasons it's almost certain that Sony is using the existing IEEE 802.15.1 (or 802.11.2?) PHY and MAC layers, since otherwise the controller would not even implement them. As such any proprietary magic must be in the higher layers such as the network or transport protocol.

Whether an existing Bluetooth adapter can be made to fully support the DualSense is not known to me but perhaps it's not as hopeless as might seem at first. At worst it would require BT adapter firmware changes but perhaps only kernel level changes could suffice, with the understanding that doing so might make that device incompatible with the standard Bluetooth ecosystem.

Bitwolfies commented 12 months ago

Triggers can be used over bluetooth, although using a slightly different format for the HID reports. You could rewrite the HID reports on the fly, which is actually what I did in #5900 (comment), but that won't cover audio-based features. I don't know of any PC game or software that use the audio-based features wirelessly, and I don't know what protocol the PS5 uses for that. I am not actually sure this is doable over bluetooth, and it's possible you'd need specific hardware adapters for that.

Well, I'm glad to hear that its therortically possible to make a backwards compatible trigger driver. Hope it's a reality one day.

Question, is it possible to have the dualsense speaker work over Bluetooth with a driver update? Or does that go beyond the stand bt stack.

Hadrianneue commented 10 months ago

@ClearlyClaire seems to work on Alan Wake 2

Hadrianneue commented 8 months ago

Cyberpunk 2077 2.1 update also has haptic feedback working with your Proton 8.0 build @Mutcholoko but not on ClearlyClaire's latest release (same happens with other games iirc, like TLOU) also it seems to work at random, not sure what is happening there.