As it turns out, symbols exported by a DLL don't necessary have to have text symbols - but they do have a number, the symbol ordinal, which is a valid way to address and access the symbol. GetProcAddress supports this behaviour - in this case, the ordinal is passed as the pointer value. Which, in Doorstop's detour, still gets treated as a string, and which subsequently causes lstrcmp to segfault under some very specific conditions (i.e. when the calling module tries to resolve a procedure by its ordinal).
In my case Unity 6 was accessing the 0x65 ordinal of d3d12.dll (located in system directory). (Did some poking - the symbol is named, which is weird, and named "D3D12CreateDevice", which checks out.)
As per GetProcAddress docs, if the value of the lpProcName pointer is an ordinal, high-word value of the pointer must be zero. This pull request addresses that quirk by inserting HIWORD macro as an additional condition to the detour check routine.
On a side note, this trend of looking up function names by their ordinals may hinder the detour.. Although doubt using ordinals is that stable. Weird.
As it turns out, symbols exported by a DLL don't necessary have to have text symbols - but they do have a number, the symbol ordinal, which is a valid way to address and access the symbol.
GetProcAddresssupports this behaviour - in this case, the ordinal is passed as the pointer value. Which, in Doorstop's detour, still gets treated as a string, and which subsequently causeslstrcmpto segfault under some very specific conditions (i.e. when the calling module tries to resolve a procedure by its ordinal).In my case Unity 6 was accessing the
0x65ordinal ofd3d12.dll(located in system directory). (Did some poking - the symbol is named, which is weird, and named "D3D12CreateDevice", which checks out.)As per
GetProcAddressdocs, if the value of thelpProcNamepointer is an ordinal, high-word value of the pointer must be zero. This pull request addresses that quirk by insertingHIWORDmacro as an additional condition to the detour check routine.On a side note, this trend of looking up function names by their ordinals may hinder the detour.. Although doubt using ordinals is that stable. Weird.
GetProcAddressdocs: https://learn.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-getprocaddress