Open Zhentar opened 5 years ago
jazzdelightsme
commented
5 years ago Thank you for doing this analysis and sharing it. I have long wondered about the SymEnumSymbols vs SymSearch thing. (You'd think I could've just asked someone who works here, and I did try that, but got no reply.)
Zhentar
commented
5 years ago Apparently SymSearch is only super slow if you're using the Recurse or AllItems search options in combination with undecorating symbols - because it doesn't apply the search mask until after populating the symbol info, causing every single symbol's name to go through an extra unnecessary (and suprisingly slow) utf8 -> utf16 -> utf8 round trip before undecorating it.
I also can't actually tell how AllItems is different from GlobalsOnly when there' isn't also Recurse (aside from being much slower).
And an actual DbgShell issue: Recurse with a "*" symbol name search errors out because some of the symbols returned don't have names
The purpose of this issue is to share some learnings (and some salt about DbgHelp) that I have gained in the past few days furthering
my descent into madnessmy understanding of symbols; whether any of this should be considered an issue that prompts changes to DbgShell or an endorsement of any particular approach I leave to the readers' judgement. SymbolSearchPerf.cs.txt"But wait!" you say, "I have done searches that get results from SymEnumSymbols but not SymSearch!" And indeed, you are correct. So I shall share the sad tale of a simple, unassuming symbol, known to its friends as
mshtml!g_hIsolatedHeap.As expected. And what about SymSearch?
Nothing. Have I lied to you? No! I have not! It is DbgHelp that lies! This symbol's proper name is not
mshtml!g_hIsolatedHeapat all, but in truth ismshtml!_g_hIsolatedHeap!SymEnumSymbols succceeds where SymSearch does not because DbgHelp secretly inserts search globs to cover for the lies it tells! But at what cost, you ask? "No, I didn-" Yes, you did. But unfortunately, there are so many different ways that DbgHelp is heckin' slow, you can't unambiguously pin it to one single thing† - but one can at least quantify how slow the various ways of calling into it to ask about our friend
mshtml!_g_hIsolatedHeapare!(spoiler alert: Benchmarkdotnet picked microseconds for this table because some of the rows I cut out actually need it)
There are few things you can tell from that table (should the names happen to make sense to you, or should you reference against the attached code). First is that with the default settings DbgHelp does try to match against your original search terms before searching again with the injected star glob. Next you might notice that
SymSearchmanages to take twice as long (unless you set symopts to search against still-"decorated" streams, a trival concession for our symbol with no decoration to be had). I'm not really sure how it does that. One might also notice that there are undocumented wildcards (that one might want to, you know, escape). In this case, I used#, which seems to mean "match the preceding character 0 or 1 times"‡, but there's also some handling for a[a-z]style syntax.Now, if one were to examine DbgHelp more closely††, one might notice that the public apis seem to pretty much just be messy wrappers aound DIA functions. And there is an undocumented
SymGetDiaSession, what might one get from that? Crashes, unless one also notices theSymFreeDiaStringand surmises that DbgHelp's DIA is special and doesn't useSysAllocStringto allocate "BSTR" values.Wow, using .NET COM interop with an exact search string gets us an answer 100 times faster (as long as we don't look at any string properties...). C++/CLI our way around the string issue and we get a whopping 3,000 times faster. Even undecoration & search globs can't get us as slow as the DbgHelp APIs go. But wait! There's more... most of those 112 microsecond were just fetching the DiaSession!
But why stop there? If you profile it, it's painfully obvious that DbgHelp's DIA is hamstrung by some inefficient UTF8 -> UTF16 conversion logic... and the
disassemblytarot cards will tell you that the version of DIA in msdia140 somewhat improves on that conversion, in addition to a statically linkedtowlower... so how would that fare?Well, shoot. Those numbers are rather remarkably close to DbgHelp's... I profiled them to make sure I wasn't accidentally double testing. I'm starting to suspect there was a reason the DbgHelp DIA used
LocalAllocinstead ofSysAllocString... msdia140 will use LocalAlloc if you use a secret alternate GUID to get your IDiaDataSource but I think I should post this and go to sleep before I get lured into testing that( SymbolSearchPerf.cs.txt contains the benchmark code, which should compile & run against the head of my personal DbgShell repository)
† aside from the dynamically linked towlower call in DbgHelp!nextToken
‡ except in
mshtml!_#_hIsolatedHeapwhere it means "get stuck in an infinite loop"†† Not by diassembling it, of course, that would be against the terms of use!