dump_syms slow on Windows 64 bit binaries

[GoogleCodeExporter]

What steps will reproduce the problem?
1. a large dll w/ pdb
2. run dymp_syms.exe
3. wait hours for it to complete

What is the expected output? What do you see instead?
I expect the same performance I see with 32 bit binaries (minutes).

What version of the product are you using? On what operating system?
Visual Studio 2008 on Windows 7.

Please provide any additional information below.

Original issue reported on code.google.com by cjstimp...@gmail.com on 6 May 2011 at 2:06

[GoogleCodeExporter]

I've seen this as well on our 64-bit Firefox builds. I haven't investigated it 
yet.

Original comment by ted.mielczarek on 31 Aug 2011 at 4:23

[GoogleCodeExporter]

Issue 439 has been merged into this issue.

Original comment by ted.mielczarek on 31 Aug 2011 at 4:27

[GoogleCodeExporter]

We just ran into this while trying to set up Socorro at National Instruments. 
We thought dump_syms was hanging. Seems strange that it would take so long to 
complete.

Original comment by alejandr...@gmail.com on 27 Oct 2011 at 3:05

[GoogleCodeExporter]

Also, I filed same issue 
(https://connect.microsoft.com/VisualStudio/feedback/details/722366/idiaenumsymb
olsbyaddr-next-doesnt-return-huge-pdb) to Microsoft.

Original comment by makoto...@gmail.com on 3 Feb 2012 at 7:23

[GoogleCodeExporter]

workaround fix

Original comment by makoto...@gmail.com on 13 Feb 2012 at 3:39

Attachments:

• win64-workaround.patch

[GoogleCodeExporter]

workaround not work with msvcr90.dll

dump_syms.exe msvcr90.dll
MODULE windows x86_64 36E1A242D2644825ABB92EDBD4058CA81 msvcr90.amd64.pdb
INFO CODE_ID 4DACE4E7A3000 msvcr90.dll
failed to get symbol
WriteMap failed

Original comment by izmmish...@gmail.com on 13 Feb 2012 at 6:29

[GoogleCodeExporter]

I must consider exe that there is no FUNC table.

Original comment by makoto...@gmail.com on 13 Feb 2012 at 6:56

Attachments:

• win64-workaround.patch

[GoogleCodeExporter]

2012-02-13 11:42:43: source_line_resolver_base.cc:220: INFO: Loading symbols 
for module 
C:\Windows\winsxs\amd64_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.30729.6161_none_
08e61857a83bc251\msvcp90.dll from memory buffer
2012-02-13 11:42:43: address_map-inl.h:53: INFO: Store failed, address 0x4d40 
is already present
2012-02-13 11:42:43: basic_source_line_resolver.cc:114: ERROR: 
ParsePublicSymbol failed at :28

after that symbols for msvcp90.dll not loaded at all

at 28 line I see duplicate of 4d40 

Original comment by izmmish...@gmail.com on 13 Feb 2012 at 7:57

[GoogleCodeExporter]

I have modified address_map-inl.h to temporary ignore duplicates, and all looks 
fine except I can't dump symbols for smartheap (shw64.dll)

I see only two PUBLIC records, but it have much more and they dumped successful 
with previous method.

Original comment by izmmish...@gmail.com on 13 Feb 2012 at 8:26

[GoogleCodeExporter]

FUNC is _setjmp, but PUBLIC is setjmp.  '_' prefix is removed by 
GetSymbolFunctionName...

Original comment by makoto...@gmail.com on 13 Feb 2012 at 10:23

[GoogleCodeExporter]

I mean what there is multiple symbols by same address
http://pastebin.com/G4Knq2Dg

Original comment by izmmish...@gmail.com on 14 Feb 2012 at 6:05

[GoogleCodeExporter]

shw64.dll contains both symbols types SymTagFunction and SymTagPublicSymbol

so ignoreFunction changed to true and public symbols ignored.

Original comment by izmmish...@gmail.com on 14 Feb 2012 at 6:39

[GoogleCodeExporter]

From the Microsoft issue that Makoto filed in comment 4:
"We have fixed this problem in our code and the fix will be included in our 
next major product release. Unfortunately there is no work around to this 
problem, unless you are willing to query the symbols any other way than 
strictly by address."

Original comment by ted.mielczarek on 14 Feb 2012 at 3:58

[GoogleCodeExporter]

Here is my fix.
The result is very close to the original.(Only some extra sympols)

Original comment by lcaum...@e-onsoftware.com on 14 Feb 2012 at 4:41

Attachments:

• pdb_source_line_writer.cc

[GoogleCodeExporter]

Duplicated symbols for public and func can be removed by resolving again like 
the following.

if (symbol->get_function(&is_func) == S_OK && !is_func &&
    symbol->get_code(&is_code) == S_OK && is_code) {
  // resolving symbol again to remove duplicated address entry
  DWORD rva;
  LONG disp;
  CComPtr<IDiaSymbol> function;
  CComPtr<IDiaSymbol> public_symbol;
  if (symbol->get_relativeVirtualAddress(&rva) == S_OK &&
      session_->findSymbolByRVA(rva, SymTagPublicSymbol,
                                &public_symbol) == S_OK &&
      session_->symsAreEquiv(symbol, public_symbol) == S_OK &&
      session_->findSymbolByRVAEx(rva, SymTagFunction, &function,
                                  &disp) == S_OK &&
      disp != 0) {
    if (!PrintCodePublicSymbol(symbol)) {
      return false;
    }
  }
}

Also, @ILT (increment linking table) doesn't output by 
findChildren(SymTagPublicSymbol).  We have to use old way for @ILT.

Original comment by makoto...@gmail.com on 15 Feb 2012 at 1:52

[GoogleCodeExporter]

> Here is my fix.
> The result is very close to the original.(Only some extra sympols)

this code will skip symbols with duplicates

          if (((i >= 1) && (SymTab[i-1].rva == SymTab[i].rva)) ||
              ((i < SymTab.size() - 1) && (SymTab[i+1].rva == SymTab[i].rva)))
          {
              symbol->Release();
              continue;
          }

so it must be changed to something like
          if (((i >= 1) && (SymTab[i-1].rva == SymTab[i].rva)))
          {
              symbol->Release();
              continue;
          }

Original comment by izmmish...@gmail.com on 15 Feb 2012 at 7:03

[GoogleCodeExporter]

patch for pdb_source_line_writer.cc from comment 14

Original comment by izmmish...@gmail.com on 15 Feb 2012 at 7:19

Attachments:

• pdb_source_line_writer.cc.patch

[GoogleCodeExporter]

I applied the patch, but when I run stackwalker pointing at a symbol file 
produced by the patched executable, I get a bunch of "ERROR: Found source line 
data without a function" errors. I didn't get the same errors when pointing at 
symbols translated with an unpatched dump_syms. This is with a dump from a 
32-bit process.

Original comment by Ramsey.N...@gmail.com on 24 May 2012 at 8:25

[GoogleCodeExporter]

Fixed in r1316 and r1319

https://breakpad.appspot.com/1574002/ and https://breakpad.appspot.com/1634002/

Original comment by wfh@chromium.org on 23 Apr 2014 at 5:28

• Changed state: Fixed

[GoogleCodeExporter]

Thank you!

Original comment by alejandr...@gmail.com on 24 Apr 2014 at 12:48