[GB-137] Games crash on startup with Epic Games Overlay

[BramTeurlings]

At least the following games crash on startup when the Epic Games overlay is turned on:

• Shadow of the Tomb Raider
• City Skylines

We need to support Epic Games overlay in our addon because the game does not crash when we disable our addon.

Additionally, Splitsecond velocity is also crashing on startup. This could be due to DimencoWeaving.dll and the game’s built-in anti cheat too.

---

Originally reported by , imported from: Games crash on startup with Epic Games Overlay

• assignee:
• status: In Progress
• priority: Medium
• resolution: Unresolved
• imported: 2024-04-04

[BramTeurlings]

712020:c26664a4-c3de-4473-a0b4-5a76804d5628:

Unloading the Epic Games DLLs on DLL_ATTACH doesn’t solve the issue.

None of the code we run in the addon causes the problem.

I think the problem is related to the other SR DLLs we load as dependencies.

[BramTeurlings]

712020:c26664a4-c3de-4473-a0b4-5a76804d5628:

Can confirm, when we comment out SRDirectX related code, the game will run fine. It seems SRDirectX.dll is the problem.

Current plan is to make a debug build of SimulatedReality and use it to determine where in the code the crash occurs. We can do this by linking with the debug build of SR and loading it into our addon.

[BramTeurlings]

712020:c26664a4-c3de-4473-a0b4-5a76804d5628:

We found out the overlay is crashing because it’s trying to hook into DimencoWeaving.dll which is protected by armadillo.

[BramTeurlings]

712020:c26664a4-c3de-4473-a0b4-5a76804d5628:

Attempting to shut down EOS will cause undefined behavior and crashes the game.

[BramTeurlings]

712020:c26664a4-c3de-4473-a0b4-5a76804d5628:

Two possible solutions:

• Make a dummy SRDirectX or DimencoWeaving.dll which has the same endpoints but no logic and is unprotected. Load the game, then wait for the EOS to load. Then unload the dummy dll and use LoadLibrary to load the real dll from the path we get from the registry.
• Hook LoadLibrary and unload the EOSOVH dll in our addon.

[BramTeurlings]

712020:c26664a4-c3de-4473-a0b4-5a76804d5628:

Hooking loadlibrary does not yield the desired results, the EOSOVH is already loaded by the time we catch the call.

The hooking itself is quite cool though and it works.

[BramTeurlings]

712020:c26664a4-c3de-4473-a0b4-5a76804d5628:

Looks like neither of the methods work. Here’s what else I tried:

• Unhooking the kernel32.dll
• Unhooking the ntdll.dll
• Catching the thrown exception (game hangs afterwards)

We’ll have to send Epic an email about this issue with a detailed report.

I’ll ask Sander Brouwers for an address.

[BramTeurlings]

712020:c26664a4-c3de-4473-a0b4-5a76804d5628:

DimencoWeaving.dll is also crashing Splitsecond with a protected dll. This is now confirmed. Let’s talk to Jurjen about alternative protection methods that don’t cause these errors.

[BramTeurlings]

712020:c26664a4-c3de-4473-a0b4-5a76804d5628:

Jurjen told me he would help make a DimencoWeaving.dll protected by XProtector instead of armadillo. We can use this to check if it still causes problems with our overlays.

Let’s talk him on Monday.

[BramTeurlings]

712020:c26664a4-c3de-4473-a0b4-5a76804d5628:

We’ve built a new version of DimencoWeaving.dll and it works in SoTTR!
I’m testing Control via the Epic Games store now.

[BramTeurlings]

712020:c26664a4-c3de-4473-a0b4-5a76804d5628:

Testing done:

• SoTTR crash is fixed by this new DLL
• Control is not crashing with the previous or new DLL
• RoTTR crash is fixed by this new DLL
• Cities skylines crash is fixed by this new DLL

[BramTeurlings]

712020:c26664a4-c3de-4473-a0b4-5a76804d5628:

Jurjen has the AxProtector project on the build server. We should inform Mike that this DLL fixes the issues we’ve observed so we can include it in a new release.

[BramTeurlings]

712020:c26664a4-c3de-4473-a0b4-5a76804d5628:

Todo:

• Investigate of the 32 bit version of the AxProtected DLL is also working (first version from local build does not while the unprotected one does)

[BramTeurlings]

712020:c26664a4-c3de-4473-a0b4-5a76804d5628:

Jurjen has given me two new sets of DLLs to test. We found that one of the protection options was not avaialble in 64 bit and it may be causing crashes.

Set1 : AxProtected baseline test results:

• Shadow of the Tomb Raider (caused crash before) (64-bit): Runs
• Tales of Symphonia (32-bit): Crash
• Split/Second (caused crash before) (32-bit): -

Set2 : AxProtected extended test results:

• Shadow of the Tomb Raider (64-bit): Runs
• Tales of Symphonia (32-bit): Crash
• Split/Second (caused crash before) (32-bit):

[BramTeurlings]

712020:c26664a4-c3de-4473-a0b4-5a76804d5628:

Todo:

• Build an example with the newest dll from Jurjen. He enables a bunch of extra flags that should increase compatibility but I want to know if the dll works even for a normal use case like an example. After knowing that, we can ask Wibu for support.

Result:

• The example also failed to build meaning there’s something else wrong with the DLL that doesn’t relate to our Game Bridge software.

Alternative solution:

If we can’t get AxProtector to work with 32-bit DLLs, we have one other option, although it’s not the neatest one. We could choose to protect the 64-bit DLL with AxProtector to be compatible with EOS overlay and other applications. Then we could choose to protect the 32-bit DLL with armadillo as usual.

This would leave us with two protected DLLs, one of which not fully compatible with everything but the other more protected and compatible than before.

[BramTeurlings]

712020:c26664a4-c3de-4473-a0b4-5a76804d5628:

Todo:

• Test the new DLL Jurjen made when creating a new project in AxProtector using only IP protection. > Doesn’t work
• Make a debug version of DimencoWeaving.dll and have that one protected to see if something goes wrong in code. > Debugger protection still enabled in AxProt
• Send a message with a clear example of what’s going wrong to Wibu. Ask them what we could be doing wrong.

[BramTeurlings]

712020:c26664a4-c3de-4473-a0b4-5a76804d5628:

I ran a minimal weaver example with the new DLL (protected) and it works fine unless I run it with the visual studio debugger. Let’s see if we can have the DLL not protected from debugging to see what’s going on.

Update:

Debugging protection is already turned off. It looks like the exception I’m getting don’t actually crash the software because if you click continue long enough, it will eventually load the program.

We should look with a debug build of DimencoWeaving.dll to see where these exceptions occur in 32-bit so we can fix them.

Update:

I’m getting a DEP error when I try to execute any function after loading the DLL. Looks like we can’t resolve the addresses of the functions properly or that the global statics we defined in the DLL are causing issues. We could try to make a build of DimencoWeaving without the statics…

[BramTeurlings]

712020:c26664a4-c3de-4473-a0b4-5a76804d5628:

I’ve made the following observation. The protected DLL in its current from (with AxProtector) does work but every call first gets a bunch of Access Violations and sometimes eventually gets through to the actual endpoint but other times it crashes.
The unprotected version will immediately get to the endpoint without any exceptions.

Either, AxProtector is causing these exceptions or DimencoWeaving isn’t entirely 32-bit compatible and is causing issues internally.

[BramTeurlings]

712020:c26664a4-c3de-4473-a0b4-5a76804d5628:

Todo:

• Prepare a simple example that calls DimencoWeaving32.dll protected and unprotected. Show that the protected one doesn’t work.
• Prepare a simple example that calls SimpleStaticDLL.dll and shows that it works unprotected only.
• Prepare an email for Wibu to ask your question about 32-bit protection against debuggers and why we keep getting a bunch of access violations and if there’s a way to stop that from happening since it’s crashing some programs.

Well it turns out all the examples (including the weaving one) work unless you attach a debugger.

When we use ReShade, we do get some unknown event which I assume is an exception as well but the program continues afterwards. In tales of Symphonia, the program crashes as soon as we call the weave() function so something is still going wrong there. Maybe it’s best to move the IP protected code outside of DimencoWeaving so we won’t have to protect it at all. That could be a better project at this point.

[BramTeurlings]

712020:c26664a4-c3de-4473-a0b4-5a76804d5628:

I want to discuss options with Jurjen/Mike/Pieter. If we can move the protected code into the already protected SRService then I won't have to worry at all about any kind of debuggers or crashes with the SDK. I think it would be a good QoL feature for developers and end-users.

[BramTeurlings]

712020:c26664a4-c3de-4473-a0b4-5a76804d5628:

Meeting notes:

Notes:

• We can try protection only certain functions with AxProtector but not the whole DLL.
• We can message Wibu with a minimal example of what goes wrong with a debugger in our 32-bit DLL
• We can use a special 32-bit build of Armadillo and use AxProtector for 64-bit as an intermediary solution.
• Use shared memory for weaving between the service and dimencoweaving.dll
  • Possible negative latency impact
    • Unless we make an independant loop in the service that outputs the current weaving parameters at a set rate and a timestamp. No waiting would be required.
  • What happens to the weaver when the service becomes unavailable

Todo:

• Check server version of Armadillo 32-bit
• Send Wibu a message with a question of why the visual studio debugger is getting breakpoints/access violations. > Newest version of AxProtecter was used
• Test AxProtector build of DimencoWeaving with 64-bit
• Test Armadillo 32-bit of DimencoWeaving
• Obfuscate the code ourselves without Wibu/Armadillo

[BramTeurlings]

712020:c26664a4-c3de-4473-a0b4-5a76804d5628:

New todos defined. Moving back to new column to be picked up later.

[BramTeurlings]

712020:c26664a4-c3de-4473-a0b4-5a76804d5628:

Found some arguments on why not to obfuscate while looking for ways to obfuscate DLLs:
https://softwareengineering.stackexchange.com/questions/155131/is-it-important-to-obfuscate-c-application-code

I think this is an uphill battle that we won’t win. We’d be putting a lot of time into protecting code all the while making dimencoweaving.dll slower. The DLL is performance critical so this is not desired. I also can’t guerantee that every app will be compatible with the AxProtector protected DLLs even if we would protect it properly in the end. I think we should remove the protection entirely.

I want to send one more email to Wibu to see if the protection can be minimized to remove anti-debugger checks and prevents these random breakpoints and access violations during debugging.

[BramTeurlings]

712020:c26664a4-c3de-4473-a0b4-5a76804d5628:

Todo:

Test if armadillo 32-bit works on dimencoweaving.dll > Works in tales of Symphonia but doesn’t work in Overcooked (Epic games). It has the same issue as the 64-bit armadillo ��

Create a SR platform branch where Armadillo protects 32 bit and x protector protects 64 bit.

At least the AxProtector version will work on 64-bit applications.

Made a MR but it will still need an updated AxProtector script to compile (ask Jurjen for the location of this): https://10.2.100.100/Dimenco/DimencoWeaving/merge_requests/35
32-bit will still have the same issues as before however.