search

CookiePLMonster / SilentPatchME

SilentPatch for Mass Effect, fixing black blob artifacts on AMD CPUs
https://cookieplmonster.github.io/2020/07/19/silentpatch-mass-effect/
MIT License
127 stars 2 forks source link

Not an issue, just a suggestion for the next step #1

Open adolfintel opened 4 years ago

adolfintel commented 4 years ago

First of all, congratulations on fixing this problem. I tried fixing this game a few years ago but I was only able to determine that the game was not using 3DNow at all and that the problem was some incorrect calculation. I loved your article, very well written, great fix.

The next logical step now would be... Mass Effect 2: determining what causes the crashes on Illium (when you walk under Liara's office and at the end of Kasumi's loyalty mission). The first one I think is just memory corruption caused by some incorrect handling of pointers, as it can be worked around by switching between fullscreen and borderless; the second one I haven't got a clue, but it might be somethng inside the bink playback code in the game, since it crashes when a bik file is supposed to play.

Again, congratulations on this massive achievement :heart:

mirh commented 4 years ago

Silent doesn't really care much for the games... And besides, I believe those crashes are fairly "normal" matter/oversights (like when you turn some texture group size limits too high or low), not the rocket science he usually bang his head against.

adolfintel commented 4 years ago

@mirh I see your point. Those crashes are almost 100% reproducible on all modern hardware though, especially the one under liara's office which is also mentioned on PCGW.

mirh commented 4 years ago

I'm sure of it, but I don't really think that's a bug to be fixed in the executable. It's probably something in the 3D models or game scripts, that should be handles in ME2Recalibrated.. or whatever other mod actually fixing the game.

riverar commented 4 years ago

Can you provide saves that repro this quickly?

adolfintel commented 4 years ago

Here's a save on Illium under Liara's office: Cleaner_13_Vanguard_271216.zip
Go through the door in front of you, if you're in fullscreen, the game crashes before you can make it to the end of the corridor. This bug was introduced when The Lair of the Shadow Broker was released in 2010.

Here's another save at the end of Kasumi's loyalty mission: Jane_32_Soldato_221110.zip
This is a very old save but it can be used to trigger the bug, at least on my machine. You'll have to defeat the boss and reach the shuttle, when you do, the game crashes on some machines. I wasn't able to figure out exactly why, and it doesn't happen on specific hardware, some machines are just "cursed" and do it. If you launch the game with -nomoviestartup it won't crash so it must have something to do with bink. The crash is caused by a null pointer inside MassEffect2.exe. It might be trying to play a nonexistent file.

To use the saves, extract the 2 folders into My Documents\BioWare\Mass Effect 2\Save
Copy the entire folder, not just the .pcsav file inside it.

mirh commented 4 years ago

It might be trying to play a nonexistent file.

If that's the case, then it should be pretty easy to spot files not found with procmon. Do you have a modded game (like for example with shorter videos?)?. I remember older versions of such mods were bugged with DLCs.

adolfintel commented 4 years ago

I'm not using mods at all, just the game, fully patched and with all the DLC installed. You can replicate the problem on the origin version. When I tried to investigate this issue 4 years ago, I used a tool similar to strace to intercept system calls (I don't remember the exact name of the tool) and there were a lot of file not found errors, even during normal gameplay: missing packages, missing cutscenes, missing loading screens. I imagine they're leftovers from early versions of the game because the game plays normally.

Mgamerz commented 4 years ago

From what I can tell this only happens to users in full screen mode.

The engine will look in multiple locations to find files, it not finding them at the first location is normal.

riverar commented 4 years ago

@adolfintel Do you have a workaround for the "Unable to authorize the listed DLC"? I used an option in Mass Effect 1 to bypass this, does one exist for Mass Effect 2?

adolfintel commented 4 years ago

@adolfintel Do you have a workaround for the "Unable to authorize the listed DLC"? I used an option in Mass Effect 1 to bypass this, does one exist for Mass Effect 2?

I have it but I don't know if it's a good idea to post it here since it's technically a crack. Do you need it?

riverar commented 4 years ago

@adolfintel No worries, got the DLC unlocker!

riverar commented 4 years ago

@adolfintel Can't reproduce the crashes sadly. :(

adolfintel commented 4 years ago

That's good to know. What resolution are you playing at? Did you try fullscreen and borderless modes? The issue has been known for years and I've always been able to replicate it: https://www.pcgamingwiki.com/wiki/Mass_Effect_2#Game_freeze_on_Illium

riverar commented 3 years ago

Still can't reproduce this, any other ideas? Tried windowed, fullscreen windowed, fullscreen, various resolutions. I'm using the latest Steam version of the executable, with ME2 DLC Unlocker.exe applied to make life easier. (This makes the file version 1.2.1604.0.)

SHA-256 of MassEffect2.exe: C78DB175B4296E403561AEC336BAA02E32E3677C3A374A795FD39C0FD4A85090

Example of one tested configuration image

riverar commented 3 years ago

Some questions:

adolfintel commented 3 years ago

Some questions:

* Do you have any gamepad or other input devices plugged in? Joysticks? Controllers?

Nope, just mouse and keyboard

* Any mods installed? Trainers in use? NOCDs?

No mods, I'm using a cracked exe with the binkw32.dll that unlock all the DLC. I was also able to replicate the issue with the origin version so the crack is not to blame.

* What's the full name of the default/primary audio device? (i.e. where do you hear game audio from?)

Nothing special, Realtek HD Audio. The device is set to its default 44100Hz 16bit stereo configuration. NVIDIA HDMI Audio is also installed but not in use.

* What version of `binkw32.dll` do you have in the Binaries folder?

https://github.com/Erik-JS/masseffect-binkw32

I know the kasumi crash is kinda rare but the crash on Illium can be replicated on all of my machines.

adolfintel commented 3 years ago

@riverar By the way, I have some experience with tools like IDA pro. I wasn't able to find out anything useful by myself but you clearly know more about it than I do, so if you give me a few pointers I might be able to investigate it further

riverar commented 3 years ago

I have no insight to share on this issue, ha! At this point, I'm just trying to reproduce the issue so we have somewhere to start.

Can you perform the following steps and reproduce the crash? We'll at least have a stack to look at.

  1. Open Regedit, navigate to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps (create the LocalDumps key if needed).
  2. Create a REG_DWORD value named DumpType and set its data to 2.
  3. Reproduce the crash.
  4. Find the MassEffect2.exe.dmp in %LOCALAPPDATA%\CrashDumps, zip it up and upload it to OneDrive/send me a copy (rafael@withinwindows.com). (It may contain PII, so do not recommend attaching it here.)
  5. Repeat step 1 above then delete DumpType to clean up.
adolfintel commented 3 years ago

Ok, I'll get back to you in a day or two with all the info I can find.

By the way, can you make a pastebin with the hashes of all the files in your ME2 installation? Especially those regarding the Kasumi DLC

riverar commented 3 years ago

@adolfintel Per request report.zip

adolfintel commented 3 years ago

@riverar Thank you. Did you get my email with the dumps?

riverar commented 3 years ago

@adolfintel oops just dug it out of my junk folder, let's take a peek!

riverar commented 3 years ago

@adolfintel Thanks, sent you a few emails back. Just a heads up, in case those end in the junk mail folder.

adolfintel commented 3 years ago

I got them, running the tests now. I can tell you right away that RTSS was not the problem and OBS was not running

adolfintel commented 3 years ago

@riverar Check your email :)

adolfintel commented 3 years ago

@riverar I sent you another email with a full dump of a kasumi crash, hope that helps

adolfintel commented 3 years ago

So, I'm 90% sure that the kasumi crash is caused by some Windows update that came out between 2014 and 2016.

I have some old hardware and software here so I decided to try various combinations:

Personally, I recall playing the game without issues in 2014, but having the crash in 2016, and this matches what I'm seeing here.

Thoughts?

adolfintel commented 3 years ago

@riverar Don't get your hopes up, but I think I fixed the kasumi crash!

I'm sending you an email with a modified exe and more info, but here's what I found out: The kasumi crash is caused by a null pointer being dereferenced. There is a piece of code that does something like this:

struct stuff=myFunction();
if(stuff.a==NULL) return; else{code...}
if(stuff.b==NULL) return; else{more code...}
...

The thing is: it never checks if stuff is NULL in the first place. So, I modified it so that after getting the value of stuff, it would jump to a small piece of code that I put in one of the empty areas in the exe where it checks if it's NULL, and if it is it returns from the function, otherwise it runs the original code

The crash seems to have been fixed, and while I haven't played through the entire game yet, it doesn't seem to cause problems so far.

Now obviously I can't post a cracked exe on github, even if it's just a fix, so with your help, I'd like to make something like SilentPatch that injects the modified code. Can you do that?

mirh commented 3 years ago

There are really countless of reports before 2014, so I'm afraid it's not Windows (also, I don't think 2011 drivers should have problems with a 2010 game?).

Also.. Maybe the issue isn't that the NULL pointer has no checks, but rather that sometimes it ends up being NULL? Anyway, I don't think silent's interested in any further ME'ing, especially if we are talking about something this "mundane".

But I'm sure @Mgamerz will have some ideas (hex editing seems a no-go on origin stupidly obfuscated exe, but maybe the binkw32 loader could come to the rescue)

adolfintel commented 3 years ago

@mirh True, reports of it exist as back as 2010, but as I remember, it was a rare issue, now all of my machines are affected, which is what made me think of a windows update.

Anyway, I'm starting a playthrough with my modified exe to see if it's stable or not. Would you like to try it?

mirh commented 3 years ago

I'm kinda out of the "playing dem games" business since a long time...

adolfintel commented 3 years ago

Well, I made an ASI :tada:

https://github.com/adolfintel/ME2KasumiCrashFix

I just finished Freedom's Progress and no issues so far :)

mirh commented 3 years ago

Well, if you are at it, there should be another couple of points where crashing happens. https://web.archive.org/web/20141201064022/http://forum.bioware.com/topic/72719-list-of-me2-bugs-and-known-solutions-if-possible http://web.archive.org/web/20160812001042/https://forum.bioware.com/topic/201096-mass-effect-2-arrival-dlc-crash/

adolfintel commented 3 years ago

As far as I know only the kasumi crash is game breaking; all other issues are occasional and "fixed" by restarting the game.

Either way, I will investigate any crash I might run into during this playthrough.

Mgamerz commented 3 years ago

ASI mods should be placed under the /asi subdirectory of the game executable folder. Modding tools in the scene will not look for ASI mods next to the game executable and this may cause duplicate ASIs which will crash the game.

On Sun, Nov 15, 2020 at 1:32 PM Federico Dossena notifications@github.com wrote:

As far as I know only the kasumi crash is game breaking; all other issues are occasional and "fixed" by restarting the game.

  • In the lair of the shadow broker DLC, when you first meet Vasir, it has maybe a 30% chance of occurring and restarting the game solves it, also happens on consoles I think
  • Occasionally, the UI starts glitching when assigning powers to quick slots, the game crashes shortly after
  • Rarely, the game crashes during the Arrival DLC

Either way, I will investigate any crash I might run into during this playthrough.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/CookiePLMonster/SilentPatchME/issues/1#issuecomment-727631901, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAU4VFHSDSO4JYWWVTYV6I3SQA3FRANCNFSM4PCC7SGA .

adolfintel commented 3 years ago

@Mgamerz Thanks for the tip. I'll update the readme accordingly. I saw in Erik's binkw32 code that it loads asi files from the exe folder so I thought it was the proper way to do it.

Mgamerz commented 3 years ago

It works in both. After he developed it I talked with him about a location that is ASI-only and had added the /asi subdirectory, which is what the modding tools in the scene use, as in older days some asi's had external dll dependencies such as zlib (they don't anymore). The original load-next-to-exe is just there for old time's sake. Good write up on how it works.Once you get more feedback I will add it to the ASI mod database so users can install it from within ME3Tweaks Mod Manager and devs from ME3Explorer.

On Mon, Nov 16, 2020 at 2:16 PM Federico Dossena notifications@github.com wrote:

@Mgamerz https://github.com/Mgamerz Thanks for the tip. I'll update the readme accordingly. I saw in Erik's binkw32 code that it loads asi files from the exe folder so I thought it was the proper way to do it.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/CookiePLMonster/SilentPatchME/issues/1#issuecomment-728334262, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAU4VFCMZG5D65PKJ5OMYJLSQGJD7ANCNFSM4PCC7SGA .

adolfintel commented 3 years ago

I'll let you know as soon as I finish this playthough, I wouldn't want people to install a mod I haven't fully tested yet.

By the way, I'm testing it on the yarrr version, if you have origin or steam installed, let me know if it works.

Mgamerz commented 3 years ago

Unfortunately I have never had this issue occur on my systems, so can't really help test the fix. Might want to ask /r/masseffect reddit for people to test, or maybe steam forums, as people who have issues typically go there and ask it seems.

On Mon, Nov 16, 2020 at 2:23 PM Federico Dossena notifications@github.com wrote:

I'll let you know as soon as I finish this playthough, I wouldn't want people to install a mod I haven't fully tested yet.

By the way, I'm testing it on the yarrr version, if you have origin or steam installed, let me know if it works.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/CookiePLMonster/SilentPatchME/issues/1#issuecomment-728337431, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAU4VFHIF26CY5R7UF5O4U3SQGJ6BANCNFSM4PCC7SGA .

adolfintel commented 3 years ago

I just need to know if it loads or not. It shows an error dialog if it can't find the pattern

Mgamerz commented 3 years ago

Yeah I can do that. I have the Origin version. Give me a few mins.

MIchael

On Mon, Nov 16, 2020 at 2:26 PM Federico Dossena notifications@github.com wrote:

I just need to know if it loads or not. It shows an error dialog if it can't find the pattern

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/CookiePLMonster/SilentPatchME/issues/1#issuecomment-728338731, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAU4VFA6YMU6EXVT7JF7QETSQGKITANCNFSM4PCC7SGA .

Mgamerz commented 3 years ago

It worked on my Origin MassEffect2.exe, but not ME2Game.exe. Origin for some reason ships both of these - MassEffect2.exe is a 144KB... shim, I assume. ME2Game is the actual game executable - but it also appears to have Origin DRM on it, so no idea what's going on with that. Some users must run ME2Game.exe as MassEffect2.exe will not work for them - no idea why, something broken in Origin that (like most origin things) never was fixed.

2020-11-16_14h38_12

Mgamerz commented 3 years ago

Also, make sure your .asi is static linked (/MT). It will be about 200KB, so it'll be bigger, but users will not have a dependency on VC++. When we used to build then without this option some users who don't have that specific VC++ installed, they'd get really fun not-obvious game crashes.

[image: image.png]

On Mon, Nov 16, 2020 at 2:41 PM Mike Perez mgamerz2@gmail.com wrote:

It worked on my Origin MassEffect2.exe, but not ME2Game.exe. Origin for some reason ships both of these - MassEffect2.exe is a 144KB... shim, I assume. ME2Game is the actual game executable - but it also appears to have Origin DRM on it, so no idea what's going on with that. Some users must run ME2Game.exe as MassEffect2.exe will not work for them - no idea why, something broken in Origin that (like most origin things) never was fixed.

[image: image.png]

On Mon, Nov 16, 2020 at 2:29 PM Mike Perez mgamerz2@gmail.com wrote:

Yeah I can do that. I have the Origin version. Give me a few mins.

MIchael

On Mon, Nov 16, 2020 at 2:26 PM Federico Dossena < notifications@github.com> wrote:

I just need to know if it loads or not. It shows an error dialog if it can't find the pattern

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/CookiePLMonster/SilentPatchME/issues/1#issuecomment-728338731, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAU4VFA6YMU6EXVT7JF7QETSQGKITANCNFSM4PCC7SGA .

Mgamerz commented 3 years ago

Guess you can't attach images in email replies, nor edit them in later. Here's the images for the last two posts:

2020-11-16_14h38_12 Comes up when running ME2Game.exe

2020-11-16_14h44_06 How to build static-linked to vc++ (ignore /Gy)

mirh commented 3 years ago

When we used to build then without this option some users who don't have that specific VC++ installed, they'd get really fun not-obvious game crashes.

Or you could build with VS 2005, whose redist comes with all three games. You wouldn't recommend that to your worst enemy usually, but it can't be that bad for what's more or less a hundred lines of simple code.

Anyway, it seems stupid to treat this fix like it was some other sort of "random fix". Just like with the physx saga, everybody's potentially affected by it, and there's no reason for one not to have it.

adolfintel commented 3 years ago

Thanks for the feedback. The asi file should already be statically linked with the MSVC 140 runtime: image

Did it ask for a missing DLL?

adolfintel commented 3 years ago

Any suggestions on how I could get it running with the weird origin exe?

Btw, it's probably best if we open an issue on my project instad of polluting this one.

Mgamerz commented 3 years ago

Might have to dump the process (I don't know how to do that, however). I'm not entirely sure what the difference between retail me2 and me2game from origin is. Don't even know why they shipped this version either, but I know a significant part of the userbase has to use it cause it doesn't even launch the game otherwise.

On Mon, Nov 16, 2020, 10:47 PM Federico Dossena notifications@github.com wrote:

Any suggestions on how I could get it running with the weird origin exe?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/CookiePLMonster/SilentPatchME/issues/1#issuecomment-728701131, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAU4VFEHPVJV5VWPCD37JH3SQIE5XANCNFSM4PCC7SGA .

adolfintel commented 3 years ago

@Mgamerz I've opened an issue here: https://github.com/adolfintel/ME2KasumiCrashFix/issues/1 Post your findings there.

This should help you dump the process memory: https://github.com/glmcdona/Process-Dump

mirh commented 3 years ago

Wtf Why in the heaven has the MassEffect2.exe launcher references (and is actually reading and writing) on Player1.prf? Also, for some reason it seems to be doing twice as much more operations with ActivationUI.exe in procmon than ME2Game (despite launching it with the same arguments) EDIT: even wireshark seems to generate twice as much events.. I wonder if Duh, MassEffect2.exe also launches ME2Game, which is the reason for ActivationUI firing again

EDIT2: the only other interesting string in the file is a mention to the EAGenericAuthToken environment variable