[d3d8] Old EverQuest client (takproject.net) still needs dgVoodoo

[kai4785]

My working system looks like this: Ubuntu 24.04 nvidia GeForce RTX 2060 driver 535.183.01 (from Ubuntu) libvulkan1-1.3.275.0-1build1 (from Ubuntu) lutris-GE-Proton8-26-x86_64 winecfg set native,builtin for all dxvk libraries: d3d10core.dll, d3d11.dll, d3d8.dll, d3d9.dll, dxgi.dll d3d8.dll from dgVoodoo2_8 (direct link) d3d11.dll and dxgi.dll from dxvk-2.4

This works for the old EverQuest client for The Al'Kabor Project server.

If I copy the remaining dxvk libraries into place (d3d10core.dll, d3d8.dll, d3d9.dll), the game crashes after character select (which coincides with calling into d3d8).

Ideally I want to switch away from dgVoodoo and go with just dxvk now that there's d3d8 support.

I have tried looking at the diff of the log between the working environment and the broken one with export WINEDEBUG=+loaddll,+d3d, but nothing jumps out to me. I do notice that dxvk's d3d8.dll causes d3d9.dll to load when it does not when using dgVoodoo's version. But anything that looks like an error when it fails seems to also be in the logs when it is playable. Those logs are ~22MiB. Should I upload those?

Here's logs with just export WINEDEBUG=+loaddll. In the zip, good.log is with dgVoodoo, bad.log is with all dxvk libraries. logs.zip

[WinterSnowfall]

You'll need both the d3d8 and d3d9 dlls from dxvk to use with d3d8 games, as descried in the readme. If a d3d8 game doesn't also properly load dxvk's d3d9.dll, it will crash (that can be an issue when games also force load dlls from system32/syswow64). You do not need d3d10core.dll or any of the rest - that may actually cause additional issues in this sort of situation. Please try only with d3d8 and d3d9 dxvk dlls for starters and use WineD3D/dgVoodoo for the rest, even if this is not your intended final setup.

If it's not that then please stick to the issue template and upload dxvk logs and an apitrace of the problem.

P.S.: From your logs it appears d3d8 dxvk never even gets called? Not much we can do about it in this case. We'd appreciate logs that isolate only d3d8/d3d9 dxvk log messages and don't contain other noise.

A successful initialization of a d3d8 device would output the following line in the logs: The D3D9 device is now operating in D3D8 compatibility mode.

[kai4785]

I'm sorry, somehow I managed to click the link in the fine print instead of clicking the big green "Create a new bug" button. Here is a better bug report.

Software information

Old EverQuest client for The Al'Kabor Project server. No setting changes required. I wrote those instructions, and I'm currently looking to clean them up. I'd like to eliminate dgVoodoo if possible.

System information

• GPU: NVIDIA GeForce RTX 2060
• Driver: 535.183.01
• Wine version: lutris-GE-Proton8-26-x86_64
• DXVK version: 2.4

Apitrace file(s)

eqgame.tar.gz

Log files

Log file with apitrace enabled, no WINEDEBUG set. wine-mer4785.log

More testing

Thanks for the links, and that confirms what I was wondering. So I went back and copied just d3d8.dll and d3d9.dll, since dgVoodoo clearly hops straight from d3d8->d3d11. Here is a log with dxvk-2.4 d3d8.dll and d3d9.dll libraries (not the wrappers) with +loaddll enabled. I confirmed that this log includes the The D3D9 device is now operating in D3D8 compatibility mode. log line. wine-mer4785.log

Thank you for your time.

[WinterSnowfall]

Thank you, this is way more relevant to us. Having a look over the trace though I can see nothing out of place. I notice it detects 3 adapters on your system and picks the Nvidia GPU, so if any of this sounds unexpected, do let me know. Otherwise, it's creating a device, enumerating modes, querying format support (all standard and expected stuff)... and then, doing nothing. There's no crash or exception on dxvk side in d3d8 or d3d9.

Could you also attach a d3d8 trace captured while using dgVoodoo please (basically your working setup)? I can then compare the two and see if anything is amiss, such as cases where we should be erroring out some calls and we're currently not perhaps.

[kai4785]

I may need assistance setting up the wrappers.

From my working setup with the overrides, I backed up d3d8.dll, d3d11.dll, and dxgi.dll in C:\windows\system32.

../../wineprefix/eggroll-lutris-GE-Proton8-26-x86_64/drive_c/windows/system32/d3d11.dll.back
../../wineprefix/eggroll-lutris-GE-Proton8-26-x86_64/drive_c/windows/system32/d3d8.dll.back
../../wineprefix/eggroll-lutris-GE-Proton8-26-x86_64/drive_c/windows/system32/dxgi.dll.back

Then I copied dgVoodoo's d3d8.dll and dxvk's d3d11.dll, and dxgi.dll in their place. In this configuration the game launches just fine.

When I copy the wrapper files for d3d8.dll, d3d11.dll, dxgi.dll and dxgitrace.dll into the game directory, the game crashes immediately after server selection. This is the same place it crashes when I use dxvk-2.4's d3d8 and d3d9 libraries alone. It generated 2 trace files. eqgame.tar.gz

Removing the wrapper dlls restores the game's functionality.

[WinterSnowfall]

By wrapper dlls, I assume you mean apitrace :wink: . Sadly, some games just do that and crash when attempting to capture an apitrace in certain situations.

Out of the two you've captured one is d3d11 specific, so not particularly useful here. The second one is indeed d3d8 and is almost identical to the one captured with dxvk, except for the final device creation, which crashes. So it is in fact NOT crashing in the same place, because with apitrace the breakage is clearly at d3d8 level, whereas dxvk passes that point without issues.

Unfortunately this doesn't help us very much. Can you please try to only use the d3d8.dll from apitrace in the game directory and nothing else, see if that fares any better? Again, we're not really interested in any of the other dlls at this point.

[kai4785]

Ok, once more with just d3d8.dll apitrace wrapper! Still crashes unfortunately. eqgame.tar.gz

[WinterSnowfall]

Still crashes unfortunately.

Interesting. The latest trace indicates that the d3d8 device creation is successful, much like it was with dxvk, so this went a bit better let's say. It still doesn't capture anything beyond that point though, as in the game doesn't appear to make any d3d8 calls after creating the device, so while the crash may be indeed related to getting d3d8 dxvk/apitrace into the picture, it occurs outside of our code. Which makes it very difficult to debug, if at all possible.

The only notable difference I could see between the dxvk trace and the dgVoodoo trace was that dgVoodoo advertises D32 support and the game indeed picks that for the auto depth-stencil during device creation, but I doubt that's relevant, since D32 is widely unsupported anyway even on Windows.

[WinterSnowfall]

One last thing to try, I guess. I strongly recommend you take a backup of your Wine prefix before doing this though.

Put:

• apitrace d3d8.dll in the game directory
• dgVoodoo2 d3d8.dll in system32 (or in syswow64 if you're on a 64-bit prefix)

All and any other d3d dlls: Wine defaults. Make sure you also remove any native Wine overrides for any d3d dlls except d3d8.

If in doubt about what is native and what is Wine default in system32/syswow64, simply delete them and then run: wineboot -u to restore the missing files to Wine defaults.

[kai4785]

Reaching for adjacent things that might be useful. I am using dgVoodoo v2.8, which is unfortunately not available to download at the moment (404). I have a cached copy locally. The next oldest version available to download is v2.82.2 (which is fewer updates than it seems) and the latest version is 2.83.1. Both of those versions of dgVodoo crash around the same time as dxvk.

[kai4785]

Ok, clean wineprefix

wine reg add 'HKEY_CURRENT_USER\Software\Wine\DllOverrides' /v d3d8 /d native /f >/dev/null 2>&1
mv ../../wineprefix/eggroll-lutris-GE-Proton8-26-x86_64/drive_c/windows/system32/d3d8.dll{,.back}
cp -a ../../cache/dgVoodoo2_8/MS/x86/D3D8.dll ../../wineprefix/eggroll-lutris-GE-Proton8-26-x86_64/drive_c/windows/system32/d3d8.dll
cp -a ../../cache/apitrace-12.0-win32/lib/wrappers/d3d8.dll .

Expected crash after server select. eqgame.tar.gz

[WinterSnowfall]

Thank you for trying. The trace looks identical to the ones captured before.

[kai4785]

All the more reason to move to d3d8 from dxvk I tried replicating all the tests above using the wine-stable version 9.0 from winehq. Everything seemed literally the same. Any other thoughts I can try out? Do you want to see any specific traces from stock wine, rather than from lutris-GE?

[Blisto91]

We have poked at bunch at this, but haven't been able to figure out why the game doesn't like dxvk d3d8.

[kai4785]

dxwrapper does work! And there certainly are performance problems. With dgVoodo->d3d11, GPU sits at 5% utilization.

The fullscreen/windowed shenanigans may be due to the eqw code that got baked into the TAKP client. The client is old enough to be from the time where "Full Screen" was used as anti-cheat!

I'll try to get an api trace with dxwrapper instead of dgvoodoo and see how that goes.

[WinterSnowfall]

And there certainly are performance problems. With dgVoodo->d3d11, GPU sits at 5% utilization.

Not saying there aren't performance issues (that frametime chart does indeed look stuttery), but the 100% GPU utilization is very likely because:

• you are on Nvidia
• you have V-Sync enabled

It's basically a known issue that will cause GPU utilization to show up as peaked, but I assure you it's nowhere near 100%.

[kai4785]

Ok, here are 3 traces. Hopefully I didn't trim them down too far to be useful. eqgame-trim.tar.gz

• dxvk2.4/eqgame-trim.trace
  • d3d8.dll from dxvk-2.4
  • d3d9.dll from dxvk-2.4
  • crashes after server select
• dxwrapper-dxvk2.2/eqgame-trim.trace
  • d3d8.dll from the provided dxwrapper zip above
  • d3d9.dll from dxvk-2.2
  • works reasonably well (despite reported 100% gpu utilization)
• dxwrapper-dxvk2.4/eqgame-trim.trace
  • d3d8.dll from dxwrapper
  • d3d9.dll from dxvk-2.4
  • has performance issues

[kai4785]

I also went through all three of the same scenarios above, and tried setting WINE_D3D_CONFIG="renderer=no3d" with no change in behavior. I tried adding wine explorer /desktop=EQ,1920x1080 before eqgame.exe, and it crashes with dxvk-2.4, and renders just a black screen with dxwrapper. When I tried all 3 scenarios with both WINE_D3D_CONFIG and /desktop, it crashed 2.4 and black screened dxwrapper.

[WinterSnowfall]

As @dungeon007 has said, this seems to be more of a problem with something crashing inside game code (or game code causing a crash somewhere) rather than with what any wrappers/translation layers are doing. I don't think there's a reasonable approach for us to solve this, and would classify it as a game bug. I guess it's just one of those situations where problematic behavior manages to sneak through and work on Windows d3d.

[K0bin]

@dungeon007 I don't really understand your last comment, could you please clarify what you meant?

Is there a performance problem with DXVK in that game?

[kai4785]

I can confirm that d3d8 works just fine if I cut out the eqw code. As for an explanation, this client is so old that it doesn't support running in windowed mode. As far as I know, eqw was written a long time ago (I can't find a source for it's origin) and was written to provide windowing capability before it was officially added to the client later. I'm now certain it's eqw's fault that I'm crashing. However, d3d8 is performing poorly. CPU is at 100% when it's usually closer to 40%. I don't know much, but it sounds like maybe it's rendering things in software that could be done on hardware? I wouldn't begin to know how to track that down though.

[WinterSnowfall]

However, d3d8 is performing poorly. CPU is at 100% when it's usually closer to 40%. I don't know much, but it sounds like maybe it's rendering things in software that could be done on hardware? I wouldn't begin to know how to track that down though.

Obligatory "does setting d3d9.cachedDynamicBuffers = True help?" question :).

[Blisto91]

@kai4785 What eqw code?

[kai4785]

Re: EQW @Blisto91 On the getting started page, there's a link to download the unmodified eqgame.exe file with a companion eqw.exe program (the purple link) I don't really have anymore information, and will go looking for it. However, this allows me to run eqgame.exe with out eqw in the way, and that launches for me just fine. If I launch eqw.exe directly from this environment, it crashes like my original post. Re: d3d9.cachedDynamicBuffers = True Unfortunately not.

$ grep cachedDynamicBuffers ../logs/lutris-GE-Proton8-26-x86_64/wine.log
info:    d3d9.cachedDynamicBuffers = True
info:    d3d9.cachedDynamicBuffers = True

Re: Forced full screen Since it forces full screen mode, I tried explorer /desktop=EQ,1920x1080 again, and couldn't get it to work. It renders the login server off screen, but if I blindly hit the right keys, it launches the game and then turns the window black and doesn't do anything else.

[kai4785]

Ok, I think we officially make this ticket a performance issue, like @dungeon007 suggests. I'm increasingly confident that the eqw program that we so heavily rely on is the source of all my problems, while simultaneously being indispensable by forcing the game into a window. I will take my adventure to find a replacement "Run it in a window" exercise somewhere else (like maybe dxwnd).

Since I have a new entry point to work with with out eqw, I have discovered that the game runs great with out of the box settings in lutris-GE-Proton8-26-x86_64; with out any help from dxwrapper, dxvk, or dgVoodoo. (I do get some flickering on the login server, where @dungeon007 says it's using DDRAW, but it's usable.)

So, let's reset a little bit and say:

This old EverQuest Client runs great on lutris-GE-Proton8-26-x86_64, and has performance issues when trying to use dxvk's d3d8.

(edit) Oh, it runs great because a single client is burning 200% cpu in top. Compared to dgVoodoo's d3d8 -> dxvk's d3d11, which idles nicely at 25-30% cpu utilization.

[Blisto91]

I dunno. I think it is fine to keep this about the original issue since it works fine on native Windows so ideally should with dxvk too. It has also gotten quite messy so i would personally prefer if the performance problem got its own separate issue

[K0bin]

@dungeon007 Try #4274

[kai4785]

I dunno. I think it is fine to keep this about the original issue since it works fine on native Windows so ideally should with dxvk too. It has also gotten quite messy so i would personally prefer if the performance problem got its own separate issue

Well, if you're interested in trying to get eqw to work properly, I think we may be enlightened by the dgVoodoo author's post a while back, when another player from this server asked him about a different bug.

If I use a d3d8.dll that's emitted directly by the compiler then I can't even log in into the character selection screen because it crashes right after switching into d3d8 mode. This is because the game seems to hook D3D8 and does this by overwriting the virtual function table of the D3D8 COM object(s). TBH it's a very bad practice because the compiler puts vft's into a read-only memory section (that's why it crashes) and it's only by chance that it works with dgVoodoo releases (the executable packer makes those pages writable). Native D3D8 puts the vft into a dynamically allocated (writable) area that's why this hook method works with it.

This sound like exactly the problem I started with. His diagnosis is that eqw is scribbling in memory in order to hook d3d, and in some cases that memory is read-only. That's possibly what is happening to dxvk as well.

[kai4785]

@dungeon007 I'm using the 2.0 client and then I installed freethemouse version eqgame_dll-v3.5.3.2c-for-ftm over the top.

All versions contain the same game files and all versions contain the same eqw.dll library. They vary only in 2 ways:

• which version of freethemouse
• presence or absence of dgVoodoo

I have not tried using the 2.1 or 2.2 clients because they purported to resolve issues I didn't have, or were already solving (e.g. the addition of dgVoodoo).

For the purposes of this thread, I would suggest using the 2.0 client because really it boils down to figuring out if eqw.dll is doing something dxvk wants to support (overwriting the virtual function table?).

[kai4785]

Had a friend divine what dege was saying about dgVoodoo crashing. Apparently if d3d8.dll is compressed, it will be decompressed into a writable location in memory. I got it to work using UPX to compress d3d8.dll.

kai@overmind:~/.takp/drive_e/TAKP$ sha256sum d3d*.dll
b6b9801a5eea87c061f974f9c182cbb601ee624e90b8ab84f8a842f808a1d661  d3d8.dll
b2c451064705afa824b6b24e71b7bfa5dccbce92c326665a30482908abd6ec5c  d3d9.dll
kai@overmind:~/.takp/drive_e/TAKP$ upx d3d8.dll
                       Ultimate Packer for eXecutables
                          Copyright (C) 1996 - 2024
UPX 4.2.2       Markus Oberhumer, Laszlo Molnar & John Reiser    Jan 3rd 2024

        File size         Ratio      Format      Name
   --------------------   ------   -----------   -----------
   1646606 ->    489486   29.73%    win32/pe     d3d8.dll                      

Packed 1 file.
kai@overmind:~/.takp/drive_e/TAKP$ sha256sum d3d*.dll
e32de843e9c795579bd27177f67ef4e4ab2cef0ba305fa69c27955fff19379d7  d3d8.dll
b2c451064705afa824b6b24e71b7bfa5dccbce92c326665a30482908abd6ec5c  d3d9.dll

Now dxvk-2.4 gets past the crash originally reported in this bug. Very interesting!

Here's a screenshot with d3d8/9 with eqw.dll doing it's glorious work.

[WinterSnowfall]

Had a friend divine what dege was saying about dgVoodoo crashing. Apparently if d3d8.dll is compressed, it will be decompressed into a writable location in memory. I got it to work using UPX to compress d3d8.dll.

Sigh. Well, as I said before, a clear case of game code doing(/expecting) unreasonable things. We can mark the original issue as a game(/loader)-specific bug, whereas the performance problems are in the process of being addressed by @K0bin.

cos of redesign some potentional regressions on a HWVP path... that could be or might not 👯

K0bin's latest optimizations target SWVP only devices, so the likelihood of regressions outside of that is minimal. Of course it could regress other known SWVP-only games like The Sims 2, but those will be checked.

Overall I think it's looking good. Thanks to everyone on this thread for their contributions in figuring it all out. As soon as the performance optimizations get merged we can close this issue.

P.S.: On a side-note, the game also creates the d3d8/9 device with D3DCREATE_MULTITHREADED, which we were not handling on d3d8 side. I've added proper support for it with #4270, but of course that wasn't the problem at hand here.

[kai4785]

@dungeon007
I updated the wiki to enumerate a few more of the changes, but I think it comes down to the freethemouse code that emu devs created. Among other things, it adds a framerate limiter, and the different versions of the client have different default values for the framerate limiter. I have preferred using DXVK_FRAME_RATE=60 on my machine. Sometimes 380 fps isn't worth the power consumption over 60fps when you have a 60hz monitor. : )

@WinterSnowfall It makes sense that dxvk wouldn't want to be required to provide a work around for eqw poking holes in read-only memory, and it makes sense that old programs like eqw took advantage of the lack of these protections. I'd be interested to find a way to something like ask WINE to disable some of these modern security measures, especially if it could be done for a single dll. In lieu of something insane like that, I have a work around to compress the d3d8.dll file from DXVK that I'm happy to carry around for a while.

[kai4785]

@dungeon007 I would be happy to test a build, but I'm not sure how to download builds from his branch (this link gives me a 404), and I haven't setup a build environment yet. I may have time later, I'm eager to see if it works well for me.

[kai4785]

No upxing -- crash with upxing -- same poor performance. Even with this 8 FPS, CPU utilization is at ~40%. When I look at a wall: CPU utilization jumps to ~80% with good framerate.

[Blisto91]

The perf fix isn't merged so the master download linked above doesn't contain it. You can download it from here https://github.com/doitsujin/dxvk/actions/runs/10944430419

[kai4785]

Much better. Still have to upx it though.

However, it doesn't seem to respect DXVK_FRAME_RATE=60, so I get a smooth 100 FPS looking around in the bazaar (lots of character models and geometry) and 300 FPS looking at the wall. Is that a known issue for d3d8?

If I use the frame rate limiter hacked into the client, it burns about 20% cpu looking at the wall and 80% cpu walking around.

Using the same build, but with dgVoodoo -> d3d11, DXVK_FRAME_RATE=60 works properly, and the same experiment looking at the wall vs walking around, I get the same 20% cpu utilization looking at the wall and 60% utilization looking around.

I think it's more likely that dxvk's better at limiting framerate than the hacks in the client.

I am optimistic that the next release of dxvk will be everything I need it to be! I'm gonna run d3d8+d3d9 on this build for the next few play sessions. Thank you all so much!

[kai4785]

While DXVK_FRAME_RATE=60 does not work, d3d9.maxFrameRate = 60 does work.

[Blisto91]

How are you using the first one

[WinterSnowfall]

While DXVK_FRAME_RATE=60 does not work, d3d9.maxFrameRate = 60 does work.

What @Blisto91 is trying to say is that DXVK_FRAME_RATE=60 is an env variable intended for command line use (or in .bashrc and the like), whereas d3d9.maxFrameRate = 60 goes in dxvk.conf. Both work when used appropriately.

[kai4785]

When using dgVoodoo d3d8 -> dxvk d3d11+dxgi, both the environment variable DXVK_FRAME_RATE=60 works as well as setting dxgi.maxFrameRate = 60 in dxvk.conf.

However, when using dxvk's d3d8->d3d9, setting the environment variable DXVK_FRAME_RATE=60 does not limit frame rate, but setting d3d9.maxFrameRate = 60 in dxvk.conf does work.

I'm suggesting that the environment variable is broken somehow when using d3d8+d3d9.

[WinterSnowfall]

I'm suggesting that the environment variable is broken somehow when using d3d8+d3d9.

Works just fine over here on dxvk master with d3d8 games, so I'm not sure why you're having issues with it.

[kai4785]

@dungeon007 I understand that using upx isn't a desirable solution. I only do it experiment with different ways to get this old game I like to play to work. I would be pleased if there was a way for dxvk to behave like Windows does, as you reported, and allow the eqw hack to hook d3d8 the way it does. As far as stupid things happening, after a few hours of playing, the game operates just as well, and often better, than using dgVoodoo's d3d8 and dxvk's d3d11+dxgi. If there's an interest in changing dxvk's d3d8 to let eqw hack it's vtable in the way dege describes, I'd love to use it. If not, I'll probably continue to use upx on dxvk's d3d8 rather than mix and match with dgVoodoo.

I do really appreciate everyone's help on the multiple issues that came up in this thread. I realize just how edge case my problem is; an old 2002 video game with a third party hack to force the full-screen only game into windowed mode.

I've learned a lot since I created this thread. Including how to disable eqw and run the game with out it (which does not require upx!). I would be very interested to learn of any other, more supported, ways to force the game into a window. Thus far, adding explorer /desktop=EQ,1920x1080 has not worked. If that's something that's supposed to work reasonably well, let me know, and I'll try to produce some more test results.

[kai4785]

I just dunno, how you can be supported if that become reason of some possible problems, so thats it 🤣

I totally understand. If I ever come back with an issue running this game, I will report it with out upxing, and with instructions on how to replicate the problem with out using eqw at all.

I have a working development environment for dxvk and will try my hand at figuring out exactly what eqw is doing, and perhaps discover why it works for some d3d8.dll files (like Windows 7 and some dgVoodoo) but not others (like dxvk and other wrappers).

[WinterSnowfall]

@kai4785 Performance should be fine with the game now on latest master. Please have a go and let us know if it behaves as expected. As for the upx-ing discussion, I would imagine that part (memory access shenanigans) is more in the realm of Wine and not something we (or other wrappers) should consider working around.

I think we can close this issue once you confirm it's good with latest master.

[kai4785]

Ya, I think the latest master is running well. If you have any pointers on where to go digging on the upx issue, I'd gladly take them. Thanks!

[WinterSnowfall]

If you have any pointers on where to go digging on the upx issue, I'd gladly take them. Thanks!

Kind of hard to tell unless there's at least hints or code pertaining to what eqw might be doing. I suspect it might try to inject itself into d3d8, which would technically trigger a memory access violation. Upx-ing gets around that most likely because it unpacks the dll in a separate memory area (which is also writable?). I don't have anything beside these speculations, I'm afraid.

[K0bin]

On 2.4.1 release, our benchmark is saying that it is like 10% slower than what was i testing 4 days ago.

That must be margin of error then. The code hasn't changed since.

[kai4785]

release 2.4.1 "works fo rme" (with upx).

I'm not having much luck tracking down precisely what eqw is doing, but I strongly suspect that it's calling Direct3DCreate8, and then poking at the returned structure. I'm trying to catch it in the act and replicate it, but my windows debugging skills (especially with out debug symbols and source code) are insufficient.