search

narzoul / DDrawCompat

DirectDraw and Direct3D 1-7 compatibility, performance and visual enhancements for Windows Vista, 7, 8, 10 and 11
BSD Zero Clause License
907 stars 69 forks source link

Rainbow Six Rogue Spear Windows 7 Visual Bugs & FPS Drops #126

Closed DaLeX97 closed 1 year ago

DaLeX97 commented 2 years ago

Hi, @narzoul !

Recently I installed Rainbow Six Rogue Spear on my Windows 7 PC, and so far, the process was quite simple. The game installed normally, I was able to patch it without problems, and I can run the game. However, there are a few issues that I've been trying so hard to solve these past couple of days, and so far I had no luck.

I have written down my experience trying to run the game here: https://rentry.co/9af3q

You only need to focus your attention on the DXWRAPPERS section of the document, so feel free to ignore the other sections, except this part:

Running the game normally as an admin and under Win98 compatibility, and without any third-party tools ---- I get some weird graphical bugs like the main menu having some shadow overlay / ghosting issue. The screen in the game starts "shaking" / flickering. But on the bright side, the game does run smoothly with high FPS.

So, there are two problems that I'm facing in this game: Main menu shadow overlay / ghosting, and in-game flickering / glitching

So far, from the couple of wrappers that I tried, they either: fix both problems that I have, but induce FPS drops (like dgVoodoo), or fix one problem but the FPS is untouched, which is where DDrawCompat comes in.

I've noticed that when I use your wrapper, the in-game flickering / glitching is solved, and my game runs smoothly. However, I still have the main menu shadowing / ghosting issue that I mentioned before.

So, instead of talking more about this let me show you how the vanilla game looks like: https://youtu.be/FYF7GNzbTFo

(I'm sorry for sloppy recording, it's hard to play the game in one hand and record in the other)

Anyway, this is how the game looks like when run without any wrappers. If I try to record the game with OBS, it looks totally different, even tho what I recorded with my phone is exactly how I see it when I play the game. Here's a link to the OBS video: https://youtu.be/RGCQmaupe3c

As you can see when the game starts, there is that main menu issue I was talking about, if I click on any of the buttons in the main menu (Singleplayer, Multiplayer, Training, Options, etc.) and come back, the problem seems to be gone, although temporarily.

In-game, when I play you can see how the textures "flicker", how it looks like there is a double image, as can be seen with the shooting targets in the training map.

When I'm done playing and come back to the main menu, the same ghosting issues appear.

Now, when I use DDrawCompat, the results I get vary depending on which version I use. For example:

Now, strangely enough, there are a couple of your 'ddraw.dll' files that do work with the game on Windows 7, and they don't cause the game to CTD or to run on very low FPS. For example, I found this GOG thread from a user who posted a ddraw.dll link to MEGA here, and when I try to use it, the in-game flickering is fixed, my game runs at high FPS, but the main menu is still a problem. Here's the video: https://youtu.be/QDVwD3pi8L0

I think that file is one of your former experimental versions which you had before, but I'm not sure which version it was.

I did also find a v0.2.1 experimental version that works the same as the one above, it fixes the gameplay and FPS issue, but the main menu is still a problem. You can see the video here: https://youtu.be/r-8zWbwEFKs

As you can see, the v0.2.1 ddraw.dll causes CTD when I try to run the game, but the experimental .dll file produces the same result as the one above from the GOG thread.

And that's everything from DDrawCompat that I tried. The other wrappers tend to fix both problems, but the FPS drops to very low levels to be playable, for example, dgVoodoo2, which can be seen here https://youtu.be/cNH9mb7yhAo (sorry for not enabling speakers).

Oddly, the MSI Afterburner / Rivatuner works only with dgVoodoo, but not with any other DXWrapper, I don't know why that is.

And that's it from me. I apologize for such a long thread, I was hoping I could give you as much info as possible to help you manage to pinpoint where the problem lies. Nonetheless, thanks for reading this far, if you did :)

I've also saved the logs from each version of DDC I tested in the attached DDrawCompatLogs.zip file, so feel free to use them if they're of any value to you.

I've also uploaded version 0.2.1 with the experimental DLL here: https://gofile.io/d/HR0Fu1

Thanks for making this wrapper, and apologies again for such a long post.

Cheers!

narzoul commented 2 years ago

There are already a couple of open issues about performance issues on Windows 7. Please try the temp workaround release here: https://github.com/narzoul/DDrawCompat/issues/104#issuecomment-869168178

DaLeX97 commented 2 years ago

There are already a couple of open issues about performance issues on Windows 7. Please try the temp workaround release here: #104 (comment)

I tried the ddraw.zip you linked in that comment, but it still has the same results as the experimental ones I've shown above. Are there any other workarounds for this, or is it a lost cause?

Also, there are no performance issues in terms of FPS like with the latest v0.3+ versions, only graphical bugs, that ironically, v0.3.1 fixes. Even tho v0.3.1 sacrifices FPS, the previous ones don't fix the graphical bugs, but leave FPS untouched, lol! What even causes issues like that?

narzoul commented 2 years ago

Strange, I would have assumed it's the same issue and definitely should have been fixed by that quick fix. Maybe there is some other problem as well.

Try alt-tabbing from the main menu and increasing the CPU affinity to "All processors" for RogueSpear.exe (right click on it under the details tab and select "Set affinity" from the menu). Do this with the temp release in the linked comment.

If that doesn't help, please use the ReleaseWithDebugLogs build from the debug zip of v0.3.1. Move the mouse for 5-10 seconds in the main menu, then exit and attach the log file (compressed).

It's possible I'll have to make a debug build of the linked version instead, but let's first see if this reveals anything new.

DaLeX97 commented 2 years ago

Strange, I would have assumed it's the same issue and definitely should have been fixed by that quick fix. Maybe there is some other problem as well.

Try alt-tabbing from the main menu and increasing the CPU affinity to "All processors" for RogueSpear.exe (right click on it under the details tab and select "Set affinity" from the menu). Do this with the temp release in the linked comment.

If that doesn't help, please use the ReleaseWithDebugLogs build from the debug zip of v0.3.1. Move the mouse for 5-10 seconds in the main menu, then exit and attach the log file (compressed).

It's possible I'll have to make a debug build of the linked version instead, but let's first see if this reveals anything new.

Done!

When I tried to run the game with the temp release from the above linked comment, I get the same issue as I did before, even after modifying the CPU affinity. I attached the logs for that just in case you needed them in this file: W7QuickFix-AllCPUCores-DDrawCompat-RogueSpear.zip

After doing that, I downloaded the DDrawCompat-v0.3.1-debug.zip file, extracted both files from the ReleaseWithDebugLogs, ran the game, moved my mouse for a couple of seconds in the main menu screen, then exited the game as you asked.

You can check those logs here: v031-DebugWithRelease.zip

Additionally, after clearing the .log file, I ran the game again, this time I did the same thing as you told me, except I opened the training mission map, played a little, went back to the main menu, then exited the game. For those results, here is the file: v031-DWR-Gameplay.zip

narzoul commented 2 years ago

Thanks! Well indeed, going to need the debug version of the quick fix instead, because the main stalling I see here is the same issue as discussed in the other thread, so it's difficult to see what else is slow.

Here's a ReleaseWithDebugLogs build of the quick fix version (except without the linear blt hack, which is probably irrelevant anyway): ddraw.zip (diff.txt compared to v0.3.1)

Please repeat the main menu test with this one instead and post the new logs.

Another thing I noticed is that GetSystemMetrics(23) is getting called a lot more frequently for you than in my case. I think it's because your mouse polling rate is a lot higher than mine, though I'm not sure if this has any impact on performance. If you can lower the polling rate, it might be worth a try too.

DaLeX97 commented 2 years ago

Thanks! Well indeed, going to need the debug version of the quick fix instead, because the main stalling I see here is the same issue as discussed in the other thread, so it's difficult to see what else is slow.

Here's a ReleaseWithDebugLogs build of the quick fix version (except without the linear blt hack, which is probably irrelevant anyway): ddraw.zip (diff.txt compared to v0.3.1)

Please repeat the main menu test with this one instead and post the new logs.

Another thing I noticed is that GetSystemMetrics(23) is getting called a lot more frequently for you than in my case. I think it's because your mouse polling rate is a lot higher than mine, though I'm not sure if this has any impact on performance. If you can lower the polling rate, it might be worth a try too.

Ok, I'll test it out with the new debug build. I'd also like to know how exactly do I lower my mouse polling rate, and at what rate should I lower it to?

I'm going to test the new debug build with my current mouse polling rate, then when I find out how to lower the polling rate, I'll test it again, and upload both versions of the logs for you, just in case to see if it does impact any performance at all.

EDIT: Nevermind, I found a way on how to lower the polling rate of my mouse (Logitech G502), at what settings would you like me to set it up, and then try the debug process?

Here are the settings: Clipboard01

narzoul commented 2 years ago

Try it at 125 and see if it makes any difference. It probably won't. But just for comparing logs, it's better if you leave it at 125 anyway while collecting debug logs.

DaLeX97 commented 2 years ago

Try it at 125 and see if it makes any difference. It probably won't. But just for comparing logs, it's better if you leave it at 125 anyway while collecting debug logs.

Done!

Okay, so these are the settings I've used for my mouse when testing Clipboard01

I lowered the polling rate, plus sensitivity, just in case higher sensitivity causes any problems with the wrapper.

Next, I did the debugging, same as before, one time with only moving my mouse in the menu screen for a couple of seconds, the other time with a little gameplay, and here are the results:

First the two files with the "normal" 1000 polling rate (which I did before I find out how to change the PR on my mouse):

Next, are the requested lowered mouse polling rate debug sessions:

And, just for comparison sakes, I redid the v0.3.1 test, this time with the lower polling rate as well:

I should mention that I did experience FPS drops during gameplay with the Windows 7 quick fix ddraw file, but I dismissed it as just it being the debug version and the fact that it's writing down about hundreds of lines per second were causing those drops.

And, there it is, I hope these logs will prove useful to you.

Cheers!

narzoul commented 2 years ago

I think I misunderstood you earlier. Is there no performance issue with the release build of the quick fix? Because I can't see any performance problems in the debug logs for the main menu.

If you meant that the graphics glitches are back, then I don't understand why, because the quick fix is essentially identical to v0.3.1, which you said has no rendering issues. So if this is the problem, then I'm out of ideas for now. I'd suggest to wait for the next release, because there are significant rendering changes under the hood, which may or may not fix the problem. I can't reproduce the glitches myself so it's pretty much impossible for me to fix/debug it remotely.

PS: What GPU are you using? Are your drivers up to date?

narzoul commented 2 years ago

Also worth trying: FaultTolerantHeap fix from Microsoft Application Compatibility Toolkit. It fixes some issues with disappearing textures (https://github.com/narzoul/DDrawCompat/issues/2#issuecomment-280867054). Although that problem doesn't look the same as the one you're having, maybe it has some effect on it too.

Also, if you're applying some forced GPU driver settings (either globally or specifically for this application) like antialiasing, try changing all of them back to default. Close down all 3rd party tools that could be interfering as well (RivaTuner, MSI Afterburner, OBS, etc.)

Since you mentioned you have the same glitches without any wrappers too, I think it's most likely some driver issue, maybe related to synchronization. If the quick fix reintroduces those glitches, then maybe the only reason they weren't there in v0.3.1 in the first place is because it was running so slowly it didn't run into any sync issues.

DaLeX97 commented 2 years ago

Ok, sorry for any misunderstandings, let me help clear things up for you.

Is there no performance issue with the release build of the quick fix? Because I can't see any performance problems in the debug logs for the main menu.

That's correct! When I use Windows 7 quick fix release build, there are no performance issues (and by performance, I mean FPS, NOT the main menu visual glitches). There are no performance issues in the main menu, nor in the in-game gameplay, however, there are visual glitches in the main menu screen, exactly as they look like in this video: https://youtu.be/QDVwD3pi8L0 (ignore the fact that I used some other version because it's the same result with the Windows 7 quick fix)

These visual glitches in the main menu appear both in these "experimental" versions + the Windows 7 quick fix, (and are oddly temporarily fixed when I click on any of the main menu buttons, then go back to the main menu). On the other hand, the FPS is untouched, and the game runs smoothly.

In version 0.3.1, the main menu visual glitches ARE fixed, BUT, both the main menu screen and in-game gameplay has enormous lag spikes, and the FPS is basically like 1-4, as shown in this video: https://youtu.be/TyHNGKXKxMk

This is true for both the release AND debug versions of 0.3.1

You can also see how the training map in which I usually show you the gameplay takes about 2 minutes to load when usually it loads instantly.

In the debug release version of the Windows 7 quick fix, there is NO performance (FPS) issues in the main menu like you said, only the visual glitches appear.

I did experience FPS drops while playing the training map on the W7 quick fix debug version, but as I said before, I dismissed this simply due to it being the debug version which writes down about 100 lines per second within the log file, so ignore this, as it's not related to the release version of that quick fix.

If you meant that the graphics glitches are back, then I don't understand why, because the quick fix is essentially identical to v0.3.1, which you said has no rendering issues. So if this is the problem, then I'm out of ideas for now. I'd suggest to wait for the next release, because there are significant rendering changes under the hood, which may or may not fix the problem. I can't reproduce the glitches myself so it's pretty much impossible for me to fix/debug it remotely.

Yes, as I said above, the graphical glitches (in the main menu) are back, so if there's minimal difference in rendering between v0.3.1. and the quick fix, then I really don't know what could be the problem, I'm stumped as well :(

The only thing that I can mention again is that with the dgVoodoo2 wrapper, I get those visual glitches in the main menu screen fixed, AND the FPS in the main menu runs smoothly, BUT my FPS drops when in-game to around 30-ish / 40-ish, instead of 1-4 as with DDC v0.3.1. The maps also load instantly.

I use the latest dgVoodoo2 version, with default settings in the control panel, and the ddraw.dll + D3DImm.dll files from the "MS > x86" folder. You can see a video of my dgVoodoo2 recording here: https://youtu.be/cNH9mb7yhAo

As you can see the main menu is perfectly normal and fixed, but once I go in-game, the FPS begins to drop.

PS: What GPU are you using? Are your drivers up to date?

My GPU is GTX 980 Ti (specifically MSI variant), with the latest drivers (472.12 WHQL).

Also worth trying: FaultTolerantHeap fix from Microsoft Application Compatibility Toolkit. It fixes some issues with disappearing textures (#2 (comment)). Although that problem doesn't look the same as the one you're having, maybe it has some effect on it too.

Ok, I'll try Microsoft's ACT method, but you'll have to tell me what I need to do and where to download it. I never used that tool before, and all I'm seeing from Microsoft's website is that the toolkit can only be downloaded for Windows 10. If it's a Windows 10-only tool, then I'm afraid I can't help here.

Also, if you're applying some forced GPU driver settings (either globally or specifically for this application) like antialiasing, try changing all of them back to default. Close down all 3rd party tools that could be interfering as well (RivaTuner, MSI Afterburner, OBS, etc.)

No forced GPU driver settings or any custom profiles were exhibited for Rogue Spear, or any programs running in the background, both while I was testing various versions (including the W7 quick fix), and recording videos, with the only exception being the OBS videos, and the dgVoodoo2 running with MSI Afterburner / Rivatuner.

Since you mentioned you have the same glitches without any wrappers too, I think it's most likely some driver issue, maybe related to synchronization. If the quick fix reintroduces those glitches, then maybe the only reason they weren't there in v0.3.1 in the first place is because it was running so slowly it didn't run into any sync issues.

Ah, I guess that makes sense, but still, would there be any workaround for this, or is it a lost cause :( ?

Also, why is the dgVoodoo2 wrapper so different that it doesn't cause the same issues in the main menu screen with such high FPS (as can be seen from the video above)? Maybe you could get in contact with Dege, to help you fix the problem, but IDK, that's just one idea I could think of.

Thanks for trying so far. I hope this comment helped cleared some things up.

narzoul commented 2 years ago

dgVoodoo2 is a complete reimplementation of the DirectDraw/Direct3D APIs that runs on top of Direct3D 11/12, so it can completely avoid whatever issue is present in the Direct3D 9 drivers that the native ddraw implementation uses (which is also used by DDrawCompat). The bug is most likely specific to the Direct3D 9 driver.

It's strange that no one else has reported similar issues with NVIDIA GPUs, even though I've had reports from other people who were using NVIDIA with this game (e.g. GTX 1060 here: https://github.com/narzoul/DDrawCompat/issues/115). I thought that you had some very old graphics card maybe with outdated drivers or some such, but that's not the case either.

I have no idea how to fix this without being able to reproduce the issue on my end. Have you tried windowed mode? I know you mentioned it crashes natively, but with the quick fix version it might run fine.

Also, does "Force Software" in the graphics menu help with the textures in-game? Note that you can only use this with 640x480 resolution, otherwise it will cause other graphics glitches. You may also have to restart the game whenever you change this option.

ACT for Windows 7 is no longer offered on Microsoft's official sites I think, they probably removed it when support for Windows 7 ended. You can obtain it from 3rd party websites (use google) but I don't want to suggest any particular links since I don't know how safe they are either. Maybe you don't have to download it at all. If you open a command prompt and type "sdbinst", does that command work? (If it doesn't write any error messages like "sdbinst is not recognized...", then it probably does). If it's available, I can just upload my sdb file and help you install the shim via that.

narzoul commented 2 years ago

After re-reading your first post, I just noticed that there were some experimental versions which didn't have the in-game flickering issue. That does give some hope. Are both versions in https://gofile.io/d/HR0Fu1 link behaving the same way, i.e. good FPS without in-game flickering? (Ignore the menu flickering for now.)

DaLeX97 commented 2 years ago

I have no idea how to fix this without being able to reproduce the issue on my end. Have you tried windowed mode? I know you mentioned it crashes natively, but with the quick fix version it might run fine.

I tried to run it in windowed mode with W7 Quick fix and v0.3.1, and it crashes each time I try, same as it does natively, no matter if it's a release version or the debug version of either of those two.

Also, does "Force Software" in the graphics menu help with the textures in-game? Note that you can only use this with 640x480 resolution, otherwise it will cause other graphics glitches. You may also have to restart the game whenever you change this option.

With the textures in-game? You mean those texture flickering that I showed in the "Vanilla" video while in the training / shooting scenario map? No, it doesn't, the screen still "vibrates" / flickers when I run it in software mode @ 640x480, it's slightly less noticeable than leaving it unchecked, but I can record the video of it if you'd like. Still, I don't see the point, because as I said, both the experimental versions + Windows 7 quick fix versions do FIX the in-game texture flickering, but not the main menu "shadowing" / ghosting overlay issues, which is what we're having difficulties solving right now.

ACT for Windows 7 is no longer offered on Microsoft's official sites I think, they probably removed it when support for Windows 7 ended. You can obtain it from 3rd party websites (use google) but I don't want to suggest any particular links since I don't know how safe they are either. Maybe you don't have to download it at all. If you open a command prompt and type "sdbinst", does that command work? (If it doesn't write any error messages like "sdbinst is not recognized...", then it probably does). If it's available, I can just upload my sdb file and help you install the shim via that.

I typed in "sdbinst" in the CMD, and this is what I get: Clipboard01

I take it, it's already installed?

After re-reading your first post, I just noticed that there were some experimental versions which didn't have the in-game flickering issue. That does give some hope. Are both versions in https://gofile.io/d/HR0Fu1 link behaving the same way, i.e. good FPS without in-game flickering? (Ignore the menu flickering for now.)

No, the official v0.2.1 release crashes the game on startup, while the experimental version does fix the in-game flickering, and runs at high FPS. I posted the video of both versions tested here: https://youtu.be/r-8zWbwEFKs

If you meant Windows 7 quick fix, and that experimental version as that "both versions", then yeah, both versions fix the in-game flickering, while maintaining high FPS.

narzoul commented 2 years ago

So just to recap: the release version of the quick fix solves all issues except the main menu glitches? From your earlier replies I assumed that it still has the in-game "shimmering" too, but your previous reply says otherwise.

So if only the main menu visual bug remains, then that might be fixable. In that case the ACT fix is not relevant.

DaLeX97 commented 2 years ago

So just to recap: the release version of the quick fix solves all issues except the main menu glitches? From your earlier replies I assumed that it still has the in-game "shimmering" too, but your previous reply says otherwise.

So if only the main menu visual bug remains, then that might be fixable. In that case the ACT fix is not relevant.

That's correct! Remember, in my first post I wrote:

So, there are two problems that I'm facing in this game: Main menu shadow overlay / ghosting, and in-game flickering / glitching

The FPS issue was not part of the game's problems. It only came from the wrappers, dgVoodoo2, DDC v0.3.1, etc.

The in-game flickering has been resolved with all the wrappers I tried. The main menu visual bug on the other hand was either fixed in some but induced FPS spikes or the main menu visual bug wasn't fixed, but the FPS remained unchanged. Nonetheless, in all of these, the in-game shimmering / flickering was solved.

So, yes, only the main menu visual bug remains as the last and only problem left for the W7 quick fix, and the experimental version.

DaLeX97 commented 2 years ago

I tried the ddraw.zip you linked in that comment, but it still has the same results as the experimental ones I've shown above. Are there any other workarounds for this, or is it a lost cause?

Also, there are no performance issues in terms of FPS like with the latest v0.3+ versions, only graphical bugs, that ironically, v0.3.1 fixes. Even tho v0.3.1 sacrifices FPS, the previous ones don't fix the graphical bugs, but leave FPS untouched, lol! What even causes issues like that?

Yeah, I just noticed where your confusion came from. In my second comment, I wrote "previous ones (referring to the experimental version + W7 quick fix) don't fix the graphical bugs", so it's no wonder you were thinking this whole time that I was also having the in-game flickering issues, my apologies :(

English isn't my native language, and as you can see, I clearly need to work on my wording more :D

BEENNath58 commented 2 years ago

@DaLeX97 I have had some success with DxWnd on Windows 10/11 for running the game, where it doesn't work at all because of some compatibility shims. Here's a .dxw profile that you could try: RogueSpear.zip

You need to install DxWnd and import this profile. DDrawCompat may be or might not be required, what I experienced is it smoothens the game a little but has texture minor issues in mission. I am using a GF10 series NV GPU.

DaLeX97 commented 2 years ago

@DaLeX97 I have had some success with DxWnd on Windows 10/11 for running the game, where it doesn't work at all because of some compatibility shims. Here's a .dxw profile that you could try: RogueSpear.zip

You need to install DxWnd and import this profile. DDrawCompat may be or might not be required, what I experienced is it smoothens the game a little but has texture minor issues in mission. I am using a GF10 series NV GPU.

Hi, @BEENNath58 ! I tried running the game with DxWnd, and I didn't have any 'compatibility shims', or other problems.

When I used DxWnd, I had the same results as with dgVoodoo2. The game starts up normally and runs fine, there were no visual bugs (neither in the main menu, nor in the game), but the same FPS drops down to ~30-ish.

I will try out the .dwx profile you sent, but I'm having hope @narzoul will be able to fix the rendering issue that I and other W7 users experience with the menu screens.

narzoul commented 2 years ago

I fixed this issue before (or a very similar one) in 2017 (commit 5195a5e4b4803f53d5b7ac11421b5d5477b2d9ef). The fix seems to still work on Windows 10/11, maybe Windows 7 does something differently. Are you sure you don't have Desktop Composition disabled or something?

Anyway, we can check if it's a similar issue again. The old case was related to WM_ERASEBKGND handling. I just removed any painting code from there, plus added the temp Win 7 performance fix: ddraw.zip (diff.txt compared to v0.3.1)

Does this fix the issue? It's not a real fix, but at least it should confirm if the problem is still with WM_ERASEBKGND.

DaLeX97 commented 2 years ago

I fixed this issue before (or a very similar one) in 2017 (commit 5195a5e). The fix seems to still work on Windows 10/11, maybe Windows 7 does something differently. Are you sure you don't have Desktop Composition disabled or something?

No, desktop composition is not disabled neither in the game's compatibility settings, nor in Windows 7 Performance Options, but it does not even matter because...

Does this fix the issue? It's not a real fix, but at least it should confirm if the problem is still with WM_ERASEBKGND.

YES!. It did fix it!

When I booted up the game I no longer get flashing / ghosting problem in the main menu, the flickering of the in-game textures is not present (although I did say that before), and I get high FPS, meaning every problem I had has now been fixed :D

I recorded videos of me testing the uploaded ddraw.zip file, and here are the results:

As you can see from the first video, the "ghosting" / flickering problem in the main menu is gone. When I enter the training scenario, I wanted to show how there was no in-game flickering / glitching (you can view the original "Vanilla" video for comparison on that).

I then proceeded to return back to the main menu to confirm that the problem was fixed, and voila! it was! I even exited the game and opened it three times just to make sure that it wasn't some weird coincidence that there was no flickering while I was recording.

The second time I entered the game I wanted to show how the game is running smoothly on high FPS, but this is a problem since the game runs at 120+ FPS on a 144 Hz monitor, while my phone records @1080p30

Nonetheless, I entered the game that many times just to prove that everything is alright and I believe the problem has been fixed. I'll do further testing to try and see if the problem will reappear in the main menu again, but so far everything seems fine, I even did a high vs. low polling rate on the mouse, and there was no difference.

On the second video, I (sloppily) showed some gameplay footage with FRAPS running in the background to show how much FPS I'm getting in the game, and as you can see it runs constantly at maximum (125). The dgVoodoo2 on the other hand had a problem in that area, The FPS dropped down to below 20, whereas with DDC it runs smooth at 120+ FPS :)

Thanks for the fix man, I had hope it was fixable, and I'm glad you proved that. If there's any testing that needs to be done on Rogue Spear (+ other expansions when I install them), let me know. I'll be glad to help out!

Cheers!

P.S. What exactly was causing the problem? You mentioned that the old case was related to WM_ERASEBKGND handling, what exactly causes the conflict here? Windows 7? GPU Drivers?

DaLeX97 commented 2 years ago

Did a couple of more tests, nothing seems to break the main menu into giving me those same visual glitches, which does mean that it's definitely stable, at least, that's how I see it.

Also, you mentioned that this is not a real fix in your last comment:

Does this fix the issue? It's not a real fix, but at least it should confirm if the problem is still with WM_ERASEBKGND

So far, everything seems to be stable on my side, do you not consider it a real fix because it's just a small change in code or something else like I asked in the previous comment about what was the cause for it?

Nonetheless, thanks @narzoul again for the fix! Let me know if you need any additional testing done.

narzoul commented 2 years ago

It's not a real fix as in it removes some functionality, which would break other games. So I can't keep it in the release version as it is.

Anyway, after further debugging, I found that the reason you're receiving an extra WM_ERASEBKGND message is because your main window receives a WM_SYNCPAINT out of nowhere, after the game already began drawing the main menu background. The WM_ERASEBKGND was erasing that background to black, so the flickering was caused by page flipping between two primary surfaces (one erased to black by mistake, the one properly drawn by the game).

But the WM_ERASEBKGND message was also being triggered from WM_SYNCPAINT because of a bug in DDrawCompat, which I found and fixed properly now. I can no longer reproduce the menu flickering even after injecting a fake WM_SYNCPAINT message (which I was using to reproduce the issue on my end now).

Here is the proper fix, please test if it works correctly on your end too: ddraw.zip (diff.txt compared to v0.3.1)

As usual, it also includes the temp performance fix as well.

Hopefully this will fix the flickering in a way that I can also add to the main release. Thanks for all the tests!

DaLeX97 commented 2 years ago

Hey, @narzoul ! Just did the test of the new proper fix, and unfortunately, I have some bad news.

The new fix did not work, and the main menu screen rendering issue has reappeared. It behaves the same way as it did before, and it's "fixed" the same way by pressing the "OPTIONS", or any other buttons on the main menu screen, then returning back to it. Of course, the moment you return back to the main menu screen (for example; from a training map), the issue reappears.

The new fix did not cause any other problems as far as I've seen. The previous in-game texture flickering issue still remains fixed, and the FPS is still in the high 120s.

Here is a video of me testing out the new fix: https://youtu.be/xFp9nS6e1d0

As you can see, the problem looks to be the same as it was before. I re-downloaded and installed the previous version as well just for comparison's sake, and again, there seem to be no problems with that one, but if I try the new fix, the rendering issues begin to reappear.

If you can, please provide a debug version of this new proper fix. Not sure if it's going to make a difference, but I'm hoping there might be something to catch your eye again and help you resolve this issue.

Also, here's a log file from the recorded video. Although, I doubt it will be much useful compared to a debug version: DDrawCompat-RogueSpear.log

One more thing,

I can no longer reproduce the menu flickering even after injecting a fake WM_SYNCPAINT message (which I was using to reproduce the issue on my end now)

Just curious, but on what system were you performing your tests as well, and was it with the same game or one of its expansions (Urban Ops, Covert Ops, etc.)? Also was it Windows 7?

Because if it was the same setting as mine (Windows 7 x64, Rogue Spear patched to v2.05 + no expansions), then I think the problem may be something on my side :(

Anyway, if there's anything else I could do to help you solve this problem, let me know!

narzoul commented 2 years ago

I'm testing everything on Windows 11 as of a few days ago. I don't have any Windows 7 PCs since a couple of years now. Even at work everything was upgraded to Windows 10 a long time ago. I don't think I'll ever have Windows 7 again either, so no real testing was done by me for DDrawCompat on Windows 7 in the last couple years. I try to maintain backwards compatibility only with the Windows API (i.e. not using any functions that weren't already available on Vista), so that at least games should launch, but I can't guarantee that they'll work as well as on Windows 10/11. I was hoping more people would have upgraded by now, considering that even Microsoft has dropped support, but that's another topic.

I usually just use the demo version of RS for testing, but I don't think it makes any difference for the menu flickering. I used it to reproduce the issue (by injecting a WM_SYNCPAINT), and also to fix the issue years ago when I still had it myself (on Windows 7 probably). I tried the full version also, but on Windows 10/11 there was no flickering with it either.

Here's a debug version of the last fix attempt, with some extra added logs: ddraw.zip (diff.txt compared to v0.3.1)

I'd like to request a bit different testing now to make it easier to compare the logs. Please make sure you don't move the mouse at all. Use the keyboard to start the game, and Alt+F4 to exit. Let it run for about 10 seconds after the flickering starts.

Also, please repeat the same test with a renamed executable (e.g. RogueSpear2.exe) to make sure all compatibility modes are cleared. Attach the logs for both tests. Thanks!

DaLeX97 commented 2 years ago

Hey, @narzoul !

I did the tests just as you asked, and recorded the process here: https://youtu.be/6-bW1CQyTwQ

In the video I used my keyboard to start the game (as you can hear from the obnoxiously loud keystrokes of a mechanical keyboard), and didn't move my mouse at all during the menu screen, I exited the game using ALT+F4, and then did the same process with a renamed executable which had all compatibility modes disabled.

Here are the log files as I have packed them in the video:

I hope these logs prove useful.

Let me know if you need anything else!

narzoul commented 2 years ago

Thanks! Actually, the fix itself is working fine, it didn't trigger WM_ERASEBKGND from WM_SYNCPAINT anymore. But it got triggered later from WM_TIMER (with the magic timer ID 0x4242), which I think is something that DirectDraw uses to force the main window onto the top of the z-order, with a little delay after entering full screen mode. This seems to be what's causing the extra WM_ERASEBKGND now, with the whole window being marked as invalidated. (Actually, an extra WM_ERASEBKGND got triggered once before the timer too, with just the bottom 40 pixels of the screen being invalidated. I guess that corresponds to the task bar, which is exactly 40 pixels high on Windows 11, and maybe on Windows 7 too. That's why only the bottom of the screen flickers initially. But this likely has the same root cause as the second WM_ERASEBKGND, as explained below.)

The window getting invalidated at this point is what's suspicious. This doesn't happen on Windows 11, and it shouldn't happen when desktop composition is enabled. Luckily, I added some extra logs about the DWM composition state, which confirms my earlier suspicion that it's indeed disabled at this point. It seems to get disabled when entering full screen mode, and it's restored to enabled state when leaving full screen mode (when exiting the game). It's the same with the renamed executable, so I don't think it has anything to do with compatibility settings, I guess it's something that's done by default by DirectDraw.

Looking at the import table of Windows 7's ddraw.dll, it references two DWM functions that my system doesn't: DwmpDxgiDisableRedirection and DwmpDxgiEnableRedirection. I suspect that this is what it's using to toggle the composition state.

So the next attempt is to try to disable those function calls and see if it's enough to keep composition enabled, which will hopefully get rid of the unnecessary window invalidation that causes the flickering. Here's both a Release and ReleaseWithDebugLogs build of this added fix (on top of all the previous stuff): ddraw.zip (diff.txt compared to v0.3.1)

Please attach the logs from the ReleaseWithDebugLogs build, with the same test as before. The Release version is provided just for further testing, in case this fix finally happens to work. Thanks again in advance!

DaLeX97 commented 2 years ago

Please attach the logs from the ReleaseWithDebugLogs build, with the same test as before. The Release version is provided just for further testing, in case this fix finally happens to work. Thanks again in advance!

Hey, @narzoul ! It's good to hear you're making progress with this, and it's good to know that the fix itself is working correctly.

I did the tests with the new ddraw.dll from the ReleaseWithDebugLogs folder with the same method as before. No mouse movement, only keyboard was being used, both on normal compatibility modes and with a new executable file without any compatibility settings.

Unfortunately, the problem still exists, and with the same results as from the previous video.

Here is a video of me testing the newest release (sorry for the camera losing/gaining focus at times, no idea why that happens): https://youtu.be/Hs_zt67vE50

And, here are the log files:

I hope there's something that'll catch your eye this time as well.

Let me know how it goes.

P.S.

Actually, an extra WM_ERASEBKGND got triggered once before the timer too, with just the bottom 40 pixels of the screen being invalidated. I guess that corresponds to the task bar, which is exactly 40 pixels high on Windows 11, and maybe on Windows 7 too

I found this thread back from 2011 from a user asking a similar thing about Windows 7, and I think it is true, Windows 7's taskbar is 40px high as well.

narzoul commented 2 years ago

Right... I didn't realize those functions were exported by ordinal only, not by name, so the hooks didn't work. Anyway, they seem to be just wrappers around the public DwmEnableComposition function, which is exported by name, so let's try hooking that instead.

Here's the new test subject: ddraw.zip (diff.txt compared to v0.3.1)

DaLeX97 commented 2 years ago

Right... I didn't realize those functions were exported by ordinal only, not by name, so the hooks didn't work. Anyway, they seem to be just wrappers around the public DwmEnableComposition function, which is exported by name, so let's try hooking that instead.

Here's the new test subject: ddraw.zip (diff.txt compared to v0.3.1)

Well, I hope this will make you happy because finally, I have some good news.

I recorded the newest supplied DLL file, and the main menu problem seems to be fixed, just as before in the previous version which you said wasn't a real fix. Here, the results are the same, and to me, it looks perfect.

I recorded the ReleaseWithDebugLogs debug DLL file with the same method as before. You can see the video here: https://youtu.be/Ht9fKUdjJPE

Additionally, once I saw that everything was working fine, I tested it for the third time, as you can see in the video. I opened the game normally with all its compatibility settings, moved the mouse around the buttons within the main menu screen, opened the tutorial map, then went back to the main menu again just to see if it was still working, and what do you know, it works perfectly!

Here are all the log files as I have packed them in the video:

Additionally, I've recorded the Release version as well, along with FRAPS to show how the game was still running smoothly at high FPS. You can see that video here: https://youtu.be/fM_9eNl4vpM

As you can see from the video, there are still no issues with the main menu screen even with the release version. When I enter the training map, there is no in-game texture flickering, and the game runs at high FPS. Additionally, I showed some (sloppy) gameplay footage of a Killhouse training map from where the other wrappers usually experience a drop in FPS. Here DDC showed a constant high FPS of 120+!

You may have noticed that when one of my characters dies, the camera went a bit weird into an "out-of-map" zone with the "sky/cosmos" textures, also that there is some weird black bar over agent Price's face. This is nothing to worry about, as this is something I experienced in the vanilla game as well, so the bug is from the game's side, not DDC. This usually happens from time to time with or without wrappers.

The only thing odd that I noticed about this version is that once I exit the game, my Chrome window shrinks down as can be seen at the end of the gameplay video above. This happens with both the debug version and the release version each time I exit the game, I'm not entirely sure if this is DDC that's causing it, or if it's probably something on my side, but I thought it might be worth mentioning.

Nonetheless, thank you for all the work you've done so far, and all the patience you had in helping me in solving this, you're a champ @narzoul !

Can we consider this issue closed now, or would you like me to do further testing?

Also, will you be implementing these changes into the new version as well?

I'll report back in this thread (or create a new one if you'd like) if I encounter any issues with the Rogue Spear's expansions (Urban Ops, Covert Ops Essentials, etc.), but for now, I'll stay with Rogue Spear until you give the green light that no more testing needs to be done.

Cheers, and good luck man!

narzoul commented 2 years ago

Great! Thanks for the extensive testing!

The only thing odd that I noticed about this version is that once I exit the game, my Chrome window shrinks down as can be seen at the end of the gameplay video above. This happens with both the debug version and the release version each time I exit the game, I'm not entirely sure if this is DDC that's causing it, or if it's probably something on my side, but I thought it might be worth mentioning.

Doesn't this happen without DDrawCompat too? For me it's happening with every full-screen DirectDraw game that uses a non-native resolution (which is pretty much every game), with or without DDrawCompat. I always assumed it was a Windows problem, but maybe it's just a problem with the native DirectDraw implementation. Anyway, the next release will allow display scaling to be handled by the wrapper, which avoids the resolution change, so it also fixes this issue.

Can we consider this issue closed now, or would you like me to do further testing?

It can be put on hold until the next release. I usually don't close issues until they're verified in the main release also (which is likely still a couple months away at this point), but it's fine with me either way.

Also, will you be implementing these changes into the new version as well?

Of course, all needed fixes will be part of the next release.

DaLeX97 commented 2 years ago

Doesn't this happen without DDrawCompat too? For me it's happening with every full-screen DirectDraw game that uses a non-native resolution (which is pretty much every game), with or without DDrawCompat. I always assumed it was a Windows problem, but maybe it's just a problem with the native DirectDraw implementation. Anyway, the next release will allow display scaling to be handled by the wrapper, which avoids the resolution change, so it also fixes this issue.

Well, on Rogue Spear without DDC it doesn't happen. It didn't happen before with the previous version tests, but it's really not a bother to me anyway.

And yeah, I did experience something like this in other DirectDraw games when going fullscreen, so I guess it's either the DirectDraw or Windows that causes it.

It can be put on hold until the next release. I usually don't close issues until they're verified in the main release also (which is likely still a couple months away at this point), but it's fine with me either way.

Sure thing, we'll put it on hold for now. When the next version releases, I'll test it out.

In the meantime, is it safe to move on with this release and play the game normally, or will you do some more code changes to it?

I'd also like to test it out on the original Rainbow Six + Rogue Spear's expansions and see how it behaves there, both without wrappers and with this release, so let me know if I should wait or I'm free to go on.

narzoul commented 2 years ago

Sure, you can keep using the latest Release build attached here in the meantime. There are just additional debug logs in it, nothing that would really affect the Release build.

DaLeX97 commented 2 years ago

Hey, @narzoul !

Sorry for this delayed reply to the thread. After my last post, I went and played MGS V, SWAT, and a couple of other games, so I totally forgot about Rainbow Six and its problems. Guess I kept you waiting, huh :D

Anyway, back to the topic at hand.

Recently, I went ahead and installed Rainbow Six (1996) from GOG, and the Urban Ops, Covert Ops and Black Thorn expansions for the Rogue Spear, and these are the results of testing:

Rainbow Six

After installing the game from GOG, the game runs at full 144 FPS (monitors refresh rate) but experiences the same in-game texture flickering issues as the Rogue Spear did. However, there was no main menu shadowing / flickering as shown before.

As expected, installing the latest W7-DDC Fix you provided here solves the problem, but the game's FPS drops slightly to 125 and remains at that FPS with very, very small deviations. Of course, this is not a complaint as the game's still running smoothly, just thought I'd mention this.

Rogue Spear

After noticing the slight FPS drop from the original Rainbow Six game, I decided to test out if it's the same with Rogue Spears, and it seems like it is.

Without DDC, the game runs at full 144 FPS, but with all those glitches and bugs, but after installing DDC, the bugs and glitches are fixed, and the only thing that happens is that FPS slightly drops to a stable 125, so sorry I forgot to mention this before. I guess I was too focused on the dgVoodo's tremendous FPS drops that I didn't notice such a difference before.

Urban Ops, Covert Ops & Black Thorn

All the bugs and problems that we mentioned in this thread about Rogue Spear appear the same way on all of these expansions, that is, that main menu shadowing / ghosting problem and in-game texture flickering appear if I don't use that W7-DDC Fix.

The moment, I install DDC, is the moment every one of those problems disappears and are fixed just as we discussed before. The only new addition being this 125 FPS lock.

So, I guess, because they're expansions for Rogue Spears, probably means that they run on the same engine as RS does. The same problems will occur as with Rogue Spear, and naturally, the same fixes for Rogue Spear apply to them as well.

And that's it from all the testing I've done.

Hopefully, this is enough info that I've provided, but in case you wish me to do additional testing let me know, otherwise you can consider this issue fixed, and close it whenever you want.

Cheers!

narzoul commented 1 year ago

The 125 FPS limit was probably caused by how the temporary fix was implemented. It should be properly fixed now in v0.4.0 without any FPS limit.