No function names when profiling cpu usage

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Compile a dynamic library and an application using it with "-g" flag.
2. Execute the application with: LD_PRELOAD=/path/to/libprofiler.so 
CPUPROFILE=file.prof application
3. Analyze the result with: pprof --text application file.prof

What is the expected output? What do you see instead?
As a regular user, the result is as expected. As "jenkins" user (the user that 
executes tasks in Jenkins CI) the result contains addresses instead of function 
names.

What version of the product are you using? On what operating system?
gperftools-2.1 with libunwind-1.1, on Ubuntu 12.04

Please provide any additional information below.
The result is the same if running a build taks on Jenkins CI, or compiling and 
running the application manually.
As a regular user, I obtain function names either compiling with "-g" flag or 
without it.

Original issue reported on code.google.com by cher...@gmail.com on 11 Nov 2013 at 6:31

[GoogleCodeExporter]

It's very hard to debug issue with so little info. Double check your flags as 
used by jenkins. Make sure that nothing is stripping debug info (which might 
happen as part of creating package).

Original comment by alkondratenko on 11 Nov 2013 at 6:59

[GoogleCodeExporter]

I see the same behavior on 12.04 with gperftools-2.1 and libunwind-1.1

I believe this is symbol extraction/mapping issue. For example pprof reports:

     291   3.0%   3.0%      291   3.0% 0x00007fac80d7e604

Even though the profile contains the corresponding entry:

7fac80d3f000-7fac80fbb000 r-xp 00000000 00:00 2651706     /home/tmm1/bin/ruby

And I can look up the address by hand:

$ addr2line -e /home/tmm1/bin/ruby 0x3f604
/home/tmm1/ruby/src/gc.c:1245

I'm guessing it's related to the leading zeros, but haven't been able to track 
it down.

Original comment by themaste...@gmail.com on 14 Dec 2013 at 3:44

[GoogleCodeExporter]

I missed the jenkins part of the original issue. Maybe the jenkins user didn't 
have access to addr2line or nm?

In my case, I tracked the issue down to the pmap parser, which only loads in 
shared libraries. I needed it to load symbols for the binary as well. The 
attached patch resolved my issue.

Original comment by themaste...@gmail.com on 14 Dec 2013 at 6:29

Attachments:

• pprof-binary-symbols.patch

[GoogleCodeExporter]

Thanks for patch. I have a question however. So your commit excludes entries 
like [heap]. But it's not clear to me how exactly it may help. Can you please 
elaborate ?

Original comment by alkondratenko on 14 Dec 2013 at 6:50

[GoogleCodeExporter]

I skipped [ in the regex to prevent these warnings:

objdump: '[vdso]': No such file
objdump: '[vsyscall]': No such file
addr2line: '[vdso]': No such file

Maybe you have a better solution.

Original comment by themaste...@gmail.com on 14 Dec 2013 at 7:00

[GoogleCodeExporter]

The part that helps is making the .so/.dylib optional. This allows my binary 
(/opt/bin/ruby) to have its symbols decoded, even though it has no file 
extension.

Original comment by themaste...@gmail.com on 14 Dec 2013 at 7:01

[GoogleCodeExporter]

I'm not familiar with internals of pprof. So please bear with me a bit.

Your explanation makes sense. But what still sounds weird is that pprof must 
have some other path to handle binaries. Otherwise it would not work in all 
cases.

Original comment by alkondratenko on 14 Dec 2013 at 7:08

[GoogleCodeExporter]

> I'm not familiar with internals of pprof. So please bear with me a bit.

I've dipped in a few times but my understanding is limited at best.

> But what still sounds weird is that pprof must have some other path to handle 
binaries.

Indeed. I've never run into this problem before, so there's probably something 
else going on here.

Original comment by themaste...@gmail.com on 14 Dec 2013 at 7:17

[GoogleCodeExporter]

I think pprof assumes the binary is mapped into the beginning of the address 
space.

For some reason, my binary is mapped elsewhere. I don't see any compiler flags 
that would cause this. The binary is large (21mb), so that could be why.

Original comment by themaste...@gmail.com on 14 Dec 2013 at 7:34

[GoogleCodeExporter]

Based on your idea I've prepared different patch. What do you think ?

Original comment by alkondratenko on 14 Dec 2013 at 8:16

Attachments:

• 0001-issue-586-detect-main-executable-even-if-PIE-is-acti.patch

[GoogleCodeExporter]

Cool. I like your patch, and I confirmed it works for me.

Original comment by themaste...@gmail.com on 15 Dec 2013 at 12:28

[GoogleCodeExporter]

Merged slightly modified version of attached patch. Going to assume it was "it" 
and close.

Original comment by alkondratenko on 15 Dec 2013 at 2:01

• Changed state: Fixed