nipkownix / re4_tweaks

Fixes and tweaks for the "UHD" port of Resident Evil 4
zlib License
341 stars 32 forks source link

Refactoring #366

Closed nipkownix closed 1 year ago

nipkownix commented 1 year ago

@emoose Could you give me your opinion on this PR draft? This makes some changes I've been meaning to make for some time. Biggest change being the wrappers, but I'm also trying to improve support for running from paths that contain UTF-8 chars.

While messing with the game on Linux, I noticed that the game crashes on start if the path it is running from contains special characters such as Japanese words. Before I could even dig into that issue, I realized something else: re4_tweaks itself was also having some trouble with special chars in paths. I've changed some of our stuff to use wstring instead, but some other issues remain: I couldn't get our .ini reader to read files inside a path with special chars. There's also the looseFilePath from our sideload feature, which I haven't tested with special chars yet, but I imagine it wouldn't work as is.

Of course, none of this special/UTF-8 chars stuff is really important right now unless we can make the game itself run from paths with those. (Do people even do that normally? Regardless, I imagine the game might also fail to load its own config.ini from the Documents path if the user's account name contains special chars..)

Do you think any of these things would be best left as is? The wrappers seem to be working nicely (from my tests). The wstring changes also aren't hurting, but they're pretty much useless right now, so no worries if we end up discarding this PR.

Ah, this should also fix the LNK4098 warning related to freetype. I noticed you pushed a workaround in 9bccd1ffbd3352c9f239cdc9cb6fde3bea981bf6 earlier. Turns out the problem was my fault for not including a debug version of the .lib as well. Sorry about that :p

emoose commented 1 year ago

Haven't been able to look through it much yet, from what I've read std::string should work fine with UTF-8 already though, other than doing doing things like string splits/regexes which might need extra handling, but don't think we really use those on file paths anyway.

Our std::string/c_str()/char* uses probably do end up calling some Win32 xxxA funcs though, on Windows those recently had UTF-8 support added in ~Win10 (but might require a manifest flag to activate), don't know if WINE/Pluton supports that at all yet, maybe those still only support ASCII there, if so then changing to wstring/wchar_t/xxxW is probably the better choice.

E: hm, using std::string to store UTF-8 does seem a little sketchy though, eg. using size() returns the number of bytes instead of actual character count, could imagine things breaking with that. Probably is better for us to move over to wstring/wchar instead.

E2: eh seems Silent's patches also use wstring/wchar too, so guess you can ignore what I said :P

Also it looks like he also fixed some game funcs by treating their strings as UTF-8 and converting them over to wchar_t with MultiByteToWideChar, for them to be used with xxxW: https://github.com/CookiePLMonster/SilentPatchMGR/blob/master/SilentPatchMGR/SilentPatchMGR.cpp#L322

I did try looking into one of the INI reading funcs in UHD before, seems they make use of std::ifstream to handle reading the file though, was hoping to patch it to std::wstring somehow but didn't seem like EXE had any std::ifstream ctor that would accept wstring, seemed like a pretty big task to add it too... maybe something similar to the UTF-8 method Silent used could work there though.

nipkownix commented 1 year ago

from what I've read std::string should work fine with UTF-8 already though

True, yeah. I decided to move try and move to wstring/wchar because of the weirdness you mentioned. Seems UTF8 with regular strings should work fine as long as whatever you're passing it to knows/expects that the string is going to be UTF8-encoded. The ini reader, for example, doesn't do that, apparently ._. I've read C++ 20 has better support for that, with stuff like char8_t, but moving to the w version of things seemed like a better idea (plus yeah, I've seen Silent's patches doing the same :p)

Edit: Latest commit makes use of namespaces for some things. Not sure about it, but I think it works/looks better this way? Let me know what you think. (I'll rebase whatever we choose to keep from here after we merge the trainer branch, so no worries with that :p)

nipkownix commented 1 year ago

Just rebased this onto the trainer branch to make it easier to test/make changes.

nipkownix commented 1 year ago

@emoose Just pushed the DXVK stuff. ~hopefully building it there works correctly.~ Just one issue: DXVK source files compile with tons of warnings on MSVC. Not sure if there's a way to suppress warnings for an entire folder or something like that. Do you happen to know of a way to do it?

It would be nice to also have a way to test if the user's system supports Vulkan, and fallback to d3d9 if it isn't available. Haven't looked into that yet.

Anyway, seems it does improve performance:

image

image

emoose commented 1 year ago

You can select multiple files and open properties window to change settings on them all, can maybe just add the warnings to "Disable Specific Warnings" for them.

Think that only needs to be applied to the .c/.cpp files of it, and maybe any of our .cpp that include dxvk headers too (unless we edit whatever headers they include)

Nice performance diff there, could try enabling DisableVsync & DisableFramelimiting in [DEBUG] section too and see how much FPS difference you get. (hm, some reason I had to set vsync 0 in games config.ini too, shouldn't DisableVsync have handled that? turning it back to vsync 1 still let it use more than 60FPS now, weird, not sure why it didn't let it before...)

(for 4080 on r101 I went from ~211FPS with DX9 to ~1400FPS with DXVK, not bad 😸 - both at 2560x1440, the 1400+FPS with DXVK did unfortunately break something and stopped me from being able to control chara after the radio call though...)

E: eh the results are a little weird actually, DX9 gets ~800FPS during the little "picking up radio" animation, but then after the radio call and I get chara control it drops to ~200FPS. Since DXVK seemed to freeze before I could get control it's not really fair to compare those, will try a different map...

E: ah okay, r103 farm gets ~257-270FPS with vulkan, and ~220-250FPS with dx9, so there is still an increase, but not as large as I thought it was :P (E2: hmm, #255 also gets around the same FPS as vulkan did too, maybe something else is bottlenecking it to that FPS...) (E3: seems like it's CPU limited due to something, at least reducing CPU clock did reduce both VK & #255 the same, wonder if we can get some kind of profiler to work with the fakepdb...)

emoose commented 1 year ago

Had a quick look through the code, the new namespacing seems pretty nice, re4t::init::XXX for the init funcs of each part makes sense, I guess for stuff like trainer where we have Trainer_Init & Trainer_Update etc we can just have an re4t::trainer namespace to handle it too.

There's also a few things I've been meaning to fix up, just some ideas to make things tidier, will probably try fixing them after we merge this refactor (but if anyone wants to look into them feel free):

nipkownix commented 1 year ago

You can select multiple files and open properties window to change settings on them all, can maybe just add the warnings to "Disable Specific Warnings" for them.

Ah cool. Had no idea we could do it per file like that. Seems it does get rid of the warnings. Thanks!

(E2: hmm, https://github.com/nipkownix/re4_tweaks/pull/255 also gets around the same FPS as vulkan did too, maybe something else is bottlenecking it to that FPS...)

Hmm, do you think there would be any issue when using #255 with vulkan?

(E3: seems like it's CPU limited due to something, at least reducing CPU clock did reduce both VK & https://github.com/nipkownix/re4_tweaks/pull/255 the same, wonder if we can get some kind of profiler to work with the fakepdb...) Gonna see if I can dig something up. I remember doing something similar on Wine once.

There's also a few things I've been meaning to fix up, just some ideas to make things tidier, will probably try fixing them after we merge this refactor (but if anyone wants to look into them feel free):

Nice! I've read about injector::hook_back before, but never really tried to use it. I think being explicit about __cdecl is a good idea because of other compilers, as you mentioned. Never a good idea to rely on defaults being the "default" forever, I guess :p

Btw, I think you should be able to push to this branch, right? If not, I can just merge this right now to net get in your way.

Been thinking about just going ahead and merging #255, even though not many people tested it so far. Do you think we should be safe-ish to do that?

Edit: Looks like VS's profiler picks up the fakepdb if we use ChkMatch as described here: https://stackoverflow.com/questions/38147487/forcing-to-load-unmatched-symbols-in-visual-studio-2015-debugger

emoose commented 1 year ago

Yeah we'd probably be fine merging #255, I never had any issues with it at least, what do you think of making MultithreadFix enabled by default too? Maybe a better chance of finding out if there are any problems with it with certain configurations. (could add a note in INI/UI about disabling it in case of issues too)

Still need to add some cfgMenu options for that too, I'll add that now. (E: updated it, also moved to framerate section & made it default enabled, let me know if you think it should be disabled instead)

About dxvk, it seemed to work fine with the 1.10.3 release at least, haven't tried it with our async enabled ver though. I think that should be fine too tho, might have read somewhere that the D3DCREATE_MULTITHREADED flag doesn't actually get used for anything in DXVK (but could be wrong about that), so maybe wouldn't make any difference besides adding mutex locks. E: ah looks like I was wrong about it, dxvk does have a few checks for that flag, looks like those mainly just do what we're implementing in #255 though, so should still be fine I think.

nipkownix commented 1 year ago

(E: updated it, also moved to framerate section & made it default enabled, let me know if you think it should be disabled instead)

Sounds good to me. I was about to suggest leaving it enabled by default.

nipkownix commented 1 year ago

Just merged the MT fix here and did some testing in the cave area after the village (same position, camera angle, etc):

VK + MT fix enabled: 470 fps d3d9 + MT fix enabled: 340 fps

Quite a nice improvement, even if it can change depending on the area you are in or your camera angle.

Edit: Also been playing a bit of mercenaries with UseDynamicFrametime = true and everything has been surprisingly stable at 350+ fps. I remember having way more problems when I tried the same using #50. Hmm.

nipkownix commented 1 year ago

Ah, by the way, just noticed ReplaceFramelimiter is in the Display section, not the Frame Rate one. Guess we should move it?

Edit: The d3d9 fallback was surprisingly simple to implement. Seems to be working fine too. Leaving UseVulkanRenderer enabled by default should be safe now 😄

emoose commented 1 year ago

Ah I was thinking of moving that to frame rate too, would probably make more sense there.

Is that ImproveWater fix new? Does it help with the water bug you mentioned before maybe? (https://github.com/nipkownix/re4_tweaks/issues/175#issuecomment-1243043667)

BTW just wondering, have you started on any changelog for the stuff after 1.7.7.6? I'm not even sure where to start with that lol, I guess we can just look through the commits since that release: https://github.com/nipkownix/re4_tweaks/compare/1.7.7.6...master 200+ commits though... hopefully we can condense it down a bit.

nipkownix commented 1 year ago

Is that ImproveWater fix new? Does it help with the water bug you mentioned before maybe? (https://github.com/nipkownix/re4_tweaks/issues/175#issuecomment-1243043667)

Ah, yes, it does help with that. It is a "new" fix, but I've had it in a local branch more or less ever since I posted that comment. It makes the water much more similar to the GC version, but I have an impression that the workaround isn't a 1-1 perfect match with GC. Still much better than the jelly water, though :p

BTW just wondering, have you started on any changelog for the stuff after 1.7.7.6?

Ouch, 200 commits since then? lol I haven't started it yet. I guess I can come up with something later today, unless you want to do it yourself.

nipkownix commented 1 year ago

@emoose, here's what I got so far for the changelog:

- Added EnableModExpansion: Enables patches/hooks for expanded modding capabilities, such as allowing enemy speed & scale to be defined when spawning. (See https://github.com/nipkownix/re4_tweaks/wiki/Modding-Enhancements#mod-expansion for more info)

- Added CameraImprovements: Provides a small set of camera tweaks (KB/M only)

- MouseTurning: Added new MouseTurning type, enabled by default. The original mode (Type B), is still available in the config menu.

- OverrideCostumes: Workaround crashes due to missing inventory models for some of Ada's costumes. (#309)

- Added new Trainer tab with features such as invincibility, heal player/Ashley, free camera, weapon hotkeys, no-clip, Em Manager, flag editor, Debug Tools (area jump, etc), Inventory Item Adder and more.

- Added command-line params support. (See https://github.com/nipkownix/re4_tweaks/wiki/Modding-Enhancements#command-line-parameters for more info)

- EnableGCBlur: Add new blur mode, slightly improved to look better on modern high-definition displays.

- AllowReloadWithoutAiming: Change hook to prevent aiming issues under certain conditions. (#314)

- Added ReplaceFramelimiter: implements new reduced-CPU-usage limiter

- Added DebugDisplay: recreates effect of DBG_SAT_DISP & DBG_EAT_DISP flags

- re4_tweaks UI: Added high-DPI support 

- Fixed face morphing (PlSetFace), fixing the bug with face animations not being shown during gameplay

- Added hook to allow the Merchant's item list to skip to top/bottom when going past bounds of the list

- Added re4_tweaks AutoUpdater

- Improved SkipIntroLogos

- Added SkipMenuFades

- Added RestoreGCSoundEffects: Restores certain sound effects that were changed from the original GC release. (currently only changes sound of the knife)

- Fixed melee range firing bug with type III controls

- Added MultithreadFix: Enables fixes for multithreaded DirectX9, giving a slight performance boost

- Added UseVulkanRenderer: Enables the use of the DXVK-based vulkan renderer, which provides better performance on newer hardware

- Added ImproveWater: Improves some of the water effects, fixing the "jelly-like" water that is present in some areas.

- FrameLimiter: Workaround flickering/shaky r117 chandelier

- FixFallingItemsSpeed: Properly fixed treasure falling speed and added a fix for the falling speed of door locks 

- Added FixCompartmentsOpeningSpeed: Fixes the speed of opening compartments such as drawers, cabinets, chests, boxes, shelves, etc.

- Added FixMovingGeometrySpeed: Fixes the speed of moving geometry such as: doors, passages, gates, ceilings, etc.

- Added RestoreDemoVideos: Restores the playback of demo videos that play when the game is idle at the main menu or the "press any key" screen for about 20 seconds. More info in the config menu/.ini file.

Should probably go more in-depth about all the trainer features, though. Besides that, I think I haven't missed anything?

There also PR #369. I'm not sure if that's ready to merge yet. @pas-de-2, I guess we still have to fix that non-English languages issue, right?

pas-de-2 commented 1 year ago

Yeah, it's just a relatively minor issue that I got stuck on. Everything else tested okay. If you wanted to put out a release soon it should be okay to merge, and add a TODO to the tracker.

emoose commented 1 year ago

@nipkownix looks good! Seems pretty complete too, nothing missing really came to mind going through it at least, guess trainer stuff might need it's own separate changelog though lol. Maybe could add a link to the modding wiki page as a separate item too, in case people miss it.

If you want to do a release soon there are a couple things I might try getting added, the item glow fix at https://github.com/nipkownix/re4_tweaks/issues/331#issuecomment-1340079614 seems to work fine, should only affect Saddler there (the fix there is more a workaround then a real fix tho, need to check GC ver and see if it acts any different there, if we do find a better fix we can always add it in a later update though), will try PRing it soon.

Haven't been able to test them much yet but think @pas-de-2's NTSC difficulty fixes should be fine to merge, mostly seems to be enabling existing code that changes the difficulty, so don't think there should be any huge issues with it, if there are problems users can always just disable the patch I guess :p Maybe worth adding a note about the 1000 -> 3000 shooting range point difference though, might take some work to get the non-English texts updated (need to implement side-loading for the containers they use, or some way of editing the messages in memory...)

Did find a way to improve the "disable enemy targetting" patch as well to let it work on more enemies, that needs a bunch more testing though, maybe something for a later release.

emoose commented 1 year ago

Made a quick WIP changelog for the trainer, feel free to edit if wanted (or disregard if you already started on one :p), not sure if I might have forgot something here though...

Introducing the new (optional) Trainer section in the re4_tweaks config menu! This section allows you to modify various aspects of the game during gameplay, such as:

- Trainer patches: Enable invincibility, weak enemies, infinite ammo, disable enemy targeting, disable collision, free camera, player speed override, difficulty level override, enemy HP multiplier, Ashley presence override, disable enemy spawns, disable bodies from disappearing, allow exit without Ashley, and enable the unused DebugTrg function.

- Weapon hotkeys: Switch weapons by pressing specific keys without needing to open the inventory first. By default keys 1-5 are assigned to cycle through weapons in different categories. Alternatively, you can assign a specific weapon to a hotkey by highlighting it in the inventory and pressing the corresponding key.

- Inventory manager: View a list of all items in your inventory and edit certain values, such as stack count and weapon upgrade values. You can also add new items to your inventory by selecting them from a simple list, without needing to set up slots for each item separately. The inventory can also be exported to or imported from an easily editable JSON file.

- Hotkeys: Allows binding specific key presses to trigger certain trainer patches or functions.

- Flag editor: Tweak internal game flags, such as flags for debug features, level progression, and unlockables earned through gameplay.

- Em Manager: View a list of all "cEm" (enemy) instances in the current level and edit certain information about them. The info can also be undocked from the main re4_tweaks UI and made visible during gameplay.

- ESP: Display information about enemies on top of them during gameplay, without requiring you to open separate UI windows. Can be configured to display various pieces of information about the enemy, such as health bars, current AI routine, and other debug info. Can also draw information about all active enemies in range to the side of the screen.

- Area jump: Teleport to different rooms in the game (including certain test/debug levels that were accidentally included)

- Filter tool: Adjust post-processing effects in the game, mainly to help with testing improvements to these filters.

- Globals: Show a separate UI window that lists important internal information about the game, such as the current room number, dynamic difficulty points and level, current game routine, and current HP for both the player and Ashley. (may be useful for mod developers when debugging their mods)

E: ran the list through ChatGPT, had to fix up a couple small issues but seemed to improve it a little (I like the new intro it made :p)

nipkownix commented 1 year ago

Just pushed some changes to settings, mainly moving some stuff to the new Overrides tab. Should probably make sure everything is still saving/loading fine and that I've missed nothing. We should probably be good for another release after that 😅

Thanks for the trainer changelog btw. I had no idea where to begin with that :p

pas-de-2 commented 1 year ago

What do you think of creating a Gameplay tab to help clean up the Misc tab? It's getting kind of overstuffed in there.

gameplay-tab

nipkownix commented 1 year ago

Huh, sounds like a good idea to me, actually. I remember being against adding new tabs in the past (before the UI redesign, I think it had something to do with that?) for some reason, but even the Trainer tab wasn't a thing back then.

@pas-de-2, would you like to open a PR for that? If not, I could just do it myself.

pas-de-2 commented 1 year ago

Sure, up

nipkownix commented 1 year ago

Updated changelog:

### **Note: This update changed some sections of the .ini structure, so you will notice that some of your existing settings will revert to default.**

- Added EnableModExpansion: Enables patches/hooks for expanded modding capabilities, such as allowing enemy speed & scale to be defined when spawning. (See https://github.com/nipkownix/re4_tweaks/wiki/Modding-Enhancements#mod-expansion for more info)
- Added CameraImprovements: Provides a small set of camera tweaks (KB/M only)
- MouseTurning: Added new MouseTurning type, enabled by default. The original mode (Type B), is still available in the config menu.
- OverrideCostumes: Workaround crashes due to missing inventory models for some of Ada's costumes. (#309)
- Moved OverrideCostumes to the Trainer tab. (Overrides)
- Added command-line params support. (See https://github.com/nipkownix/re4_tweaks/wiki/Modding-Enhancements#command-line-parameters for more info)
- EnableGCBlur: Add new blur mode, slightly improved to look better on modern high-definition displays.
- AllowReloadWithoutAiming: Change hook to prevent aiming issues under certain conditions. (#314)
- Added ReplaceFramelimiter: implements new reduced-CPU-usage limiter
- Added DebugDisplay: recreates effect of DBG_SAT_DISP & DBG_EAT_DISP flags
- re4_tweaks UI: Added high-DPI support 
- Fixed face morphing (PlSetFace), fixing the bug with face animations not being shown during gameplay
- Added hook to allow the Merchant's item list to skip to top/bottom when going past bounds of the list
- Added re4_tweaks AutoUpdater
- Improved SkipIntroLogos
- Added SkipMenuFades
- Added RestoreGCSoundEffects: Restores certain sound effects that were changed from the original GC release. (currently only changes sound of the knife)
- Fixed melee range firing bug with type III controls
- Added MultithreadFix: Enables fixes for multithreaded DirectX9, giving a slight performance boost
- Added UseVulkanRenderer: Enables the use of the DXVK-based vulkan renderer, which provides better performance on newer hardware
- Added ImproveWater: Improves some of the water effects, fixing the "jelly-like" water that is present in some areas.
- FrameLimiter: Workaround flickering/shaky r117 chandelier
- FixFallingItemsSpeed: Properly fixed treasure falling speed and added a fix for the falling speed of door locks 
- Added FixCompartmentsOpeningSpeed: Fixes the speed of opening compartments such as drawers, cabinets, chests, boxes, shelves, etc.
- Added FixMovingGeometrySpeed: Fixes the speed of moving geometry such as: doors, passages, gates, ceilings, etc.
- Added LimitMatildaBurst: Limit the Matilda to one three round burst per trigger pull.
- Added RifleScreenShake: Adds a screen shake effect when firing a rifle.
- Added RestoreDemoVideos: Restores the playback of demo videos that play when the game is idle at the main menu or the "press any key" screen for about 20 seconds. More info in the config menu/.ini file.
- Added SeparateWaysProfessional: Adds a menu to choose between Normal and Professional difficulty modes when starting a new game of Separate Ways.
- Added RestoreAnalogTitleScroll: Restores the ability to manipulate the background scroll of the post-new game title menu with the right analog stick or mouse movement.
- Added AlwaysShowOriginalTitleBackground: After beating the game, force the game to always show the original main menu background image of Leon and Ashley.
- Added ShowGameOutput: Displays the game's original logs/debug output into a console window.
- Added EnableNTSCMode: Unlocks minor difficulty boosts previously exclusive to the NTSC console versions of RE4.
- Added a workaround fix for an item glow issue in Saddler's last cutscene 

Introducing the new (optional) Trainer section in the re4_tweaks config menu! This section allows you to modify various aspects of the game during gameplay, such as:
- Trainer patches: Enable invincibility, weak enemies, infinite ammo, disable enemy targeting, disable collision, free camera, disable enemy spawns, disable bodies from disappearing, allow exiting area without Ashley, and enable the unused DebugTrg function.
- Trainer overrides:  Player speed override, game difficulty override, dynamic difficulty level override, enemy HP multiplier, and Ashley presence override.
- Weapon hotkeys: Switch weapons by pressing specific keys without needing to open the inventory first. By default, keys 1-5 are assigned to cycle through weapons in different categories. Alternatively, you can assign a specific weapon to a hotkey by highlighting it in the inventory and pressing the corresponding key.
- Inventory manager: View a list of all items in your inventory and edit certain values, such as stack count and weapon upgrade values. You can also add new items to your inventory by selecting them from a simple list, without needing to set up slots for each item separately. The inventory can also be exported to or imported from an easily editable JSON file.
- Hotkeys: Allows binding specific key presses to trigger certain trainer patches or functions.
- Flag editor: Tweak internal game flags, such as flags for debug features, level progression, and unlockables earned through gameplay.
- Em Manager: View a list of all "cEm" (enemy) instances in the current level and edit certain information about them. The info can also be undocked from the main re4_tweaks UI and made visible during gameplay.
- ESP: Display information about enemies on top of them during gameplay, without requiring you to open separate UI windows. It can be configured to display various pieces of information about the enemy, such as health bars, current AI routine, and other debug info. Can also draw information about all active enemies in range to the side of the screen.
- Area jump: Teleport to different rooms in the game (including certain test/debug levels that were accidentally included)
- Filter tool: Adjust post-processing effects in the game, mainly to help with testing improvements to these filters.
- Globals: Show a separate UI window that lists important internal information about the game, such as the current room number, dynamic difficulty points and level, current game routine, and current HP for both the player and Ashley. (May be useful for mod developers when debugging their mods.)

Just gotta make sure everything is saving/loading fine, and that I haven't missed anything from the changelog.

emoose commented 1 year ago

@nipkownix looks like a couple PRs might be missing there, I think it's just these in the pic + the NTSC difficulty setting, besides those it looks good, can't think of much else that's missing. image

nipkownix commented 1 year ago

Oh yeah, we're missing the Saddler fix and NTSC mode. Nice catch!

emoose commented 1 year ago

Good job with the release, great to see we made it in time for Christmas 🥳

I added a tiny section in the release text about updating from HD Project 1.1, think the instructions there should be correct, but feel free to change it if not. (mostly only added that section because some other sites like PCGW used to mention deleting all re4_tweaks files before updating, which I don't think we want :P)

Anyway have a good Christmas!

nipkownix commented 1 year ago

Thanks!

I added a tiny section in the release text about updating from HD Project 1.1, think the instructions there should be correct, but feel free to change it if not. (mostly only added that section because some other sites like PCGW used to mention deleting all re4_tweaks files before updating, which I don't think we want :P)

Ohh, nice thinking. I'll probably copy that over to the Nexus pages too. Thank you, and have a good Christmas as well!