Abstract: Maintainance of eSync patchset

[Kreyren]

Based on available informations i've understand that esync patches for WINE has the ability to increase the game performance in a scenario where wineserver is pegging a core. (seemingly case for OverWatch and possible other wineapps)

I was told that ZF (Maintainer of esync patchset) is not willing to maintain the patchset excluding rebasing, but that he's aware of a fixes to make the patchset working. (NOT VERIFIED, WILL BE VERIFIED BASED ON RESPONSE)

If that's the case are we willing to help with the maintainance? In theory Lutris and Winehackers might be also interested in this cooperation.

Patchset is fork of wine written in C. https://github.com/zfigura/wine

Relevant: https://bugs.gentoo.org/674552

Relevant: https://github.com/bobwya/bobwya/tree/master/app-emulation (Has esync patches, was told that it breaks lots of things)

[Kreyren]

WIll try bobwya's wine with esync patches on overwatch, this comment will be updated with results.

RESEARCH: Emerging app-emulation/wine-staging-4.0_rc4::bobwya

Emerge successful wine is eselected on bobwya's

Installing OverWatch

Crashes with https://paste.pound-python.org/show/ZW03S9PglUhVhmdevUqz/

Had simmilar issue on Dishonored 2, investigating

Installed corefonts, vcrun2008, vcrun2017, dxvk

Installing OverWatch

Killed, needs win7

Installing OverWatch

The window can be dragged with no issues -> Still suspect phoenicis issue with installer

Started https://youtu.be/QmXOCecSLkg

01d6:fixme:nvapi:unimplemented_stub function 0xe3640a56 is unimplemented! 01d7:fixme:nvapi:unimplemented_stub function 0xae457190 is unimplemented! Reason for lutris to disable it ?

CPU spiked on 100% investigating

OBS is eating 50% of system resources optimizing..

Invoked WINEESYNC="1" DXVK_HUD="devinfo,fps,frametimes,drawcalls,pipelines,memory,version" wine Overwatch.exe

Failed to authetificate

invoked WINEESYNC="1" DXVK_HUD="devinfo,fps,frametimes,drawcalls,pipelines,memory,version" wine Overwatch\ Launcher.exe

Has issues, investigating

added nvapi and nvapi64 in libs -> woudn't start -> set nvapi and nvapi64 on builtin then native -> game starts

Game spikes on 100% CPU and starts lagging, disablig nvapi + nvapi64

Game works on 250FPS +- but spikes on 100% if game is joined, investigating

spams with 0137:fixme:ntdll:NtQueryDirectoryObject multiple entries not implemented 0137:fixme:ntdll:NtQueryDirectoryObject multiple entries not implemented

set ntdll on native then builtin, opens game spikes on 100% later

set ntdll on builtin then native, game opens and runs on +-60 FPS has freezes

disable ntdll, Game opens and runs on +-60FPS with freezes (more playeble)

disable ntdll64, Game open and runs above 55 FPS without freezes, has some minor issues that i wasn't able to identify -> GOLD

Using DXVK_STATE_CACHE="/tmp/OW" WINEDEBUG="-all" WINEESYNC="1" DXVK_HUD="devinfo,fps,frametimes,drawcalls,pipelines,memory,version" wine Overwatch\ Launcher.exe

Still complains about 0132:fixme:ntdll:NtQueryDirectoryObject multiple entries not implemented 0132:fixme:ntdll:NtQueryDirectoryObject multiple entries not implemented

Video: https://youtu.be/HlvGlfJjldU

Trying to play, spiked on 100% on numbani -> silver

"glitched" on busan

glitches on other maps crashes.

Trying crazy wine from bobwya -> same issues ( [2] wine-staging:9999 * Fri Jan 4 19:01:08 2019 +0100 54e4d51fdbda08333012040c0683394853a097e9 )

Trying 3.19_p1-R5 based on info from @Tk-Glitch

Emerging app-emulation/wine-staging-3.19_p1-r5::bobwya

Game is more stable and playeble, but has freezes, Crashes ware fixed.

Removed disbled on ntdll + ntdll64 and keeping nvapi + nvapi64 disabled -> Game was performing on gold without esync with freezes (i would say better in comparison to previous attempt)

trying esync, game performs much better lags are less frequent.

===

Game is working quite good without esync on gold with freezes and outputs:

018b:fixme:wininet:InternetSetOptionW Option 77 STUB 018b:fixme:wininet:InternetSetOptionW INTERNET_OPTION_SEND/RECEIVE_TIMEOUT/DATA_SEND_TIMEOUT 15000 018c:fixme:wininet:InternetSetOptionW Option 77 STUB 018c:fixme:wininet:InternetSetOptionW INTERNET_OPTION_SEND/RECEIVE_TIMEOUT/DATA_SEND_TIMEOUT 15000 0122:fixme:ntdll:NtQueryDirectoryObject multiple entries not implemented 0122:fixme:ntdll:NtQueryDirectoryObject multiple entries not implemented 0122:fixme:ntdll:NtQueryDirectoryObject multiple entries not implemented 0122:fixme:ntdll:NtQueryDirectoryObject multiple entries not implemented

[plata]

So gold with and without esync?

[qparis]

It is fine if we have a maintened repository.

My question is: what are the different versions available and how to select them. For example, how do I checkout wine 3.17 with esync?

[qparis]

Moving this issue to phoenicis-winebuild

[ImperatorS79]

AFAIK, the esync patcheset comes originally from https://github.com/zfigura/wine/tree/esync. There is no tags associated with different wine-version in this repository.

Additionnaly, there is https://github.com/Tk-Glitch/PKGBUILDS/tree/master/wine-tkg-git/wine-tkg-patches (a lot used by the community), which contains patch to apply esync on top of wine-staging. However, you have to "manually" check which patch you need to apply on top of which wine-staging version :/ (r1, r2, r3, ...)(and I do not know which parts in this are required).

[qparis]

So what do we decide to do finally? We create two distributions and we'll have small number of builds inside them? @plata @madoar I'd like to have your opinion here

[madoar]

I'm not very knowledgeable about wine. I just would suggest to not add too many different distributions. When looking at our current engines tab it is already confusing with the current number of tabs.

[qparis]

This is a UI problem, not a number of distribution problem I guess.

[madoar]

I don't think so. I agree it's nice for developers to have a huge number of distributions available, but it is annoying for normal users who just want to install an engine and have no in-depth knowledge about the details of the different categories.

If we can't solve the problem differently (for example through a verb), then I have no problem with adding an additional engine category. We should just be sure that it is absolutely required to prevent cluttering (my opinion).

[qparis]

Maybe we could create a phoenicis distribution that contains different patchs then

[ImperatorS79]

I think it is really a UI problem. You only have: -upstream -staging -proton (basically upstream (not latest) + esync + DXVK + other patches) -dosbox -cx (is it really needed for a normal user ?)

[qparis]

Do we need to support esync AND tkg?

[ImperatorS79]

tkg ?

For esync, the only interesting way to support it would be to add the esync patch on top of wine-staging (because proton is not at the latest wine version). But this is details.

[qparis]

These ones: https://github.com/Tk-Glitch/PKGBUILDS/tree/master/wine-tkg-git/wine-tkg-patches

[ImperatorS79]

This is just a repo full of a lot of patchsets which are not in wine/wine-staging. A lot of them are outdated ; and/or interesting like e.g. PBA (which seems, for some games and high OpenGL version, to increase performance of the d3d11->opengl), but it is really unmaintained by his author (and full of bugs), and meaningless now that there is DXVK. I have only mentioned it because it contains the esync patchset as a .patch .

[qparis]

Do we want to maintain a repository that contains patchset, like wine staging does?

[Kreyren]

Please note that the gold was changed on silver due issues in quick play i believe that it's due using crazy wine version which build directly from master i'm still testing.

But i do suggest using esync patches since we can always disable the function and the wine is practically working the same way without it. (Needs verification, working on it)

• it would probably support the development of esync patches in the future.

[qparis]

But i do suggest using esync patches since we can always disable the function and the wine is practically working the same way without it. (Needs verification, working on it)

This is not as simple as this

[Kreyren]

@qparis it is just by seting WINEESYNC variable. (still testing)

[qparis]

I know, but this does not solve our problem at all

[Kreyren]

meaning?

[qparis]

As I said before, we need to find a reliable way to build it on the long run, without multiplying the number of distributions

[Kreyren]

Alike i can submit it for installers that i'm working on no problem.. o.o It's building fast on my system, but the issue is compatibility due usage of useflags which shoudn't be impossible to solve either.. And then you can compile them from bobwya's

OR you can make a system which checks the wine on rolling release and request the end-user to recompile the wine if needed

[qparis]

Thank you for the proposition but these solutions are not acceptable

[ImperatorS79]

No, because phoenicis is intended so that users, I assume, do not have to deal with wine things. (for normal users)

[Kreyren]

@qparis why is it not acceptable to build esync from bobwya's or practically you can build them from ZF assuming that bobwya didn't do any black witchery to his ebuilds

[qparis]

The problem is not to build wine, the problem is to make phoenicis-winebuild able to build it for any version of wine (or decide that we don't want to build it for all version of wine). The other problem is to decide about the criteria to accept to add a new supported wine distribution

[ImperatorS79]

We could provide a wine-staging + esync, in place of wine-staging (since you still need WINEESYNC=1 to use the esync patchset, it will make no difference for regular use). The only remaining problem is to rebase the esync patch for each wine version (as .patch like in tkg, so that you apply it on top of wine-staging).

[qparis]

I don't think we should stop providing wine-staging alone. This distribution is accepted and maintained by the wine team and the release cycle is very reliable

[ImperatorS79]

Yes, but adding esync on top of it will not change that, will prevent you to store another distribution, and will not change anything for regular use since you need "WINEESYNC=1" to activate the patchset.

[qparis]

It will, because we won't be able to release wine-staging every two weak if we do that

[ImperatorS79]

You just need to ensure that the esync patchset apply cleanly.

[qparis]

There is a large chance that it is not compatible with every release.

[ImperatorS79]

It seems @Tk-Glitch is really fast at doing the rebase. If the patch does not cleanly apply, you can still build wine without it.

But I agree everything would be simpler if the esync patchset was in wine-staging directly (which I do not understand ^^).

Again this is really details since esync is in Proton. this is just to have a combination of esync + wine more up-to-date than in Proton.

[qparis]

Yes, @Tk-Glitch is probably doing amazing job, but we cannot make our process rely on one person.

But I agree everything would be simpler if the esync patchset was in wine-staging directly (which I do not understand ^^).

This is one more reason not to add in our staging builds. Wine team must have a good reason.

I would suggest to create one last ehanced distribution with all the patch required. @madoar What do you think?

[ImperatorS79]

https://github.com/zfigura/wine/issues/18 Seems to be more a reason of coding style than something really problematic. Again since you need an environment variable to use it, there is no worries.

Adding a whole new distribution just for something "cosmetic" like this seems quite bad for me. If the wine-staging + esync way I proposed is not an option (which I understand perfectly), I think the best way is to not do anything.

[qparis]

If you read the comment, this is not just cosmetic.

There are some aspects of the design of these patches (namely, use of shared memory) and of their functionality (namely, that PulseEvent() and wait-all can't quite work right) that won't be acceptable upstream. As such I am afraid the patchset will always have to remain out-of-tree.

[ImperatorS79]

If you read it, you see it is not upstreamable because what is doing this patchset is not an acceptable solution in the way wine devs want solutions. But this not something like "IF you add this patchset, there is a chance everything will explode!" ^^. Remember wine still use pure ANSI C.

EDIT: Remember this is used by Valve in Proton! I mean cosmetic for the fact that Proton has esync, and the whole point here is : "Should we provide this outside Proton ?"

[qparis]

Wine is one of the most outstanding example of app in term of development process, quality of code / product and reverse engineering that I know. I really trust their point of view in that matter. In addition, when people will install POL staging versions, it will not really be a "wine staging", so it can be misleading for them when they receive bugs from our users.

On the other hand I believe that this patch is very useful. This is why I would suggest to have a separate distribution

[ImperatorS79]

I perfectly/absolutely understand that.

[qparis]

We just need to circle back with @madoar and we are good to go

[Kreyren]

Too much text for me to read through atm, lets wait for the results of my research first or try to verify for yourself so that we have more info to base the decision on.

EDIT: afaik ZF rebases the esync for each wine release which shouldn't affect the maintainance.

[ImperatorS79]

Yes, but the repo of zfigura does not have proper tags :/ so that it is difficult to have a specific wine version.

[Kreyren]

@ImperatorS79 bobwya has proper tags afaik https://github.com/bobwya/bobwya/tree/master/app-emulation

well at least to gentoo's standarts

[Tk-Glitch]

I'd like to say I'm fast at rebasing also thanks to ZF's direct help and reactivity on bugfixing. Since I'm providing wine builds sources for Lutris, receiving active support from him is a game changer for the "safety" side of things. But yeah, there is no guarantee that will last forever.

Also PBA patchset is encountering huge regression since 3.20 and would require a rewrite that, so far, nobody has done.

[Kreyren]

@Tk-Glitch Can you recommend an esync wine build for overwatch for me to test in theory anything under 3.20 should work?

btw. thanks for your feedback it's appreciated ^^

EDIT: More info on huge regression since 3.20 In case someone from phoenicis community is willing to contrib?

EDIT2: Does the regression also affects DXVK in some way? In theory relevant to https://github.com/ValveSoftware/Proton/issues/823 ?

[Tk-Glitch]

@Kreyren I don't think there's a need for anything fancy regarding OW, so the basic "tkg" build available through Lutris would probably be my default pick. If you want a bit fancier, the protonified build will give you additional fullscreen hacks for scaling and compositor auto-turnoff. Also, 4.0rc's have been slightly slower than 3.21 (probably one of the reason why Strider didn't yet build 4.0rc based -tkg builds). If you got an Arch/Arch-based system around, building an optimized build from my PKGBUILD would give an additional perf boost.

About PBA regression, I can point to https://github.com/Tk-Glitch/PKGBUILDS/issues/49 as a base of work. I didn't dive into the code since november and I have to admit I have lost track of the issues with it as they were stacking and snowballing. There has been quite a bit of refactoring upstream and the first problems came with 3.17. Bobwya has made an additional patch and cleaned up the patchset for 3.17 & 3.18 but anything past that is basically broken. Firerat also opened an issue for it : https://github.com/Firerat/wine-pba/issues/3

[Kreyren]

@Tk-Glitch

tkg" build available through Lutris would probably be my default pick.

Found patches on https://github.com/Tk-Glitch/PKGBUILDS/tree/master/wine-tkg-git/wine-tkg-patches will compile myself. (not that i don't believe strider, but i don't :D ) Any instructions to compile them?

EDIT: found bobwya's repo https://github.com/bobwya/gentoo-wine-pba, no need for instructions

From https://github.com/Tk-Glitch/PKGBUILDS/issues/49 a pretty significant regression in DX9 games. Does it affect DXVK games? (DirectX10/11) or vkd3d (DirectX 12) ? Gallium9 affected?

[Tk-Glitch]

I'm not using Strider builds either tbh :smile:

The regression only affects wined3d. DXVK, vkd3d and Gallium9 are unaffected.

[Kreyren]

@Tk-Glitch Noted, DXVK + esync + staging + WINE 3.19-p1 still has freezes, but the performance is much better in comparison to esync less wine. Trying to tweak dlls now will try PBA after. ^^