Handle DLL files properly when WinLicense detects rundll32.exe

[hyunsikjeong]

Hello, I found out that WinLicense detects rundll32.exe in some cases. When I tried debugging my samples, it goes to a pitfall that allocates a string "OLLYDBG" in memory (maybe ollydbg uses rundll32.exe for DLL debugging? 🤔), and then tries to access an invalid address which cannot be mmaped/mprotected. So I manually compiled a simple program just calls LoadLibrary and changed rundll32_path in frida_exec.py:spawn_and_instrument() to it, then unpacking successfully worked.

I cannot share a sample program as it's a bit sensitive, but I hope this would be helpful for future DLL handlings.

[greenozon]

Interesting finding though.. Ollydbg has internal mechanics to load dll - it's using part of itself named loaddll.exe I"ve double checked - there are no any refs in Ollydbg to rundll32.exe string inside itself or parties

Most probably it is how frida works

[hyunsikjeong]

That's pretty weird then. Why is WinLicense showing a such string... I think WinLicense messed up some anti-debugging logics, probably.

I checked that when I load the DLL file with x32dbg, it loads successfully w/o falling into that pitfall logic. Also, Frida loads successfully with my hand-compiled DLL-loading program, not using rundll32.exe.

[ergrelet]

Hi! Thanks for the report. That's weird indeed, I don't know what could be the cause, hard to tell without reproducing but I'll keep that in mind.

However, I was already thinking of embedding a custom executable for DLL loading as rundll32 is pretty limited in term of features (and doesn't allow loading .NET assemblies for example). The lack of authenticode signature for that executable would probably trigger some more AVs but I guess we're not in a case where unlicense is doing anything safe anyway...