CPU-profiled cython generated programs lack symbols in CPU reports

[GoogleCodeExporter]

A complete yet minimal setup to reproduce the issue is available here:

  https://gist.github.com/923957

What is the expected output? What do you see instead?

  Most entries in the profile report are memory addresses instead of intelligible symbols.

What version of the product are you using? On what operating system?

  1.5-1 as packaged in Ubuntu 10.10

Original issue reported on code.google.com by olivier....@gmail.com on 17 Apr 2011 at 11:54

[GoogleCodeExporter]

I have updated the gist with a sample output of perftools when using the 
current trunk version instead of the packaged version available in Ubuntu. The 
problem remains mostly the same.

Original comment by olivier....@gmail.com on 17 Apr 2011 at 12:18

[GoogleCodeExporter]

BTW: I would like to link this issue as duplicate (with more details) as issue 
#291 but I do not have the rights to do so.

Original comment by olivier....@gmail.com on 17 Apr 2011 at 12:19

[GoogleCodeExporter]

I attach the generated .prof file.

Original comment by olivier....@gmail.com on 17 Apr 2011 at 12:33

Attachments:

• superapp.py.prof

[GoogleCodeExporter]

Sorry it's taken me a few days to get back to you.

Do you happen to know what library the addresses belong to, that aren't being 
properly symbolized?  Sometimes using gdb can help figure that out.

Otherwise, you may have to go through the logic that pprof is using to try to 
symbolize, and see where it's failing.  We see reports a lot of symbolization 
failing, but there can be lots of causes.  (Check some of the other bug reports 
here.  You mention issue #291, but I don't know yet if this is the same as 
that.  I think there may be some others as well.)  Dynamic libraries can be 
one, if your app has them.

Original comment by csilv...@gmail.com on 22 Apr 2011 at 7:45

• Added labels: Priority-Medium, Type-Defect

[GoogleCodeExporter]

Thanks for your feedback.

The compiled extension module is indeed a dynamic library but the 2 main 
functions show up with proper symbol names, e.g:

  static PyObject *__pyx_pf_9supermath_compute_something

Hence dynamic loading does not seem to be the issue here.

I tend to suspect the static functions generated by cython and marked with the 
CYTHON_INLINE macro, for instance at line 1280 in 
https://gist.github.com/923957#file_supermath.c :

  static CYTHON_INLINE unsigned short __Pyx_PyInt_AsUnsignedShort(PyObject* x)

Other than this function calls I don't see than many functions called in the 
nested loops that would show up as high in the profiler output.

Original comment by olivier....@gmail.com on 22 Apr 2011 at 8:29

[GoogleCodeExporter]

Hmm, inlined functions should show up (assuming that the libraries were 
compiled with debugging information, of course!), assuming you can use 
addr2line.  But maybe it's having trouble for some reason.  It's hard to tell 
what's going on without tracing through the perl script.

Original comment by csilv...@gmail.com on 22 Apr 2011 at 8:43

[GoogleCodeExporter]

The exact build flags are given by:

  $ gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall \
    -Wstrict-prototypes -fPIC -I/usr/include/python2.6 -c supermath.c \
    -o supermath.o
  $ gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions \
    supermath.o -o supermath.so

I'll try to dig further with manual addr2line testing this WE (I am not 
familiar with this tool and neither with the perftools codebase though...).

Original comment by olivier....@gmail.com on 22 Apr 2011 at 8:50

[GoogleCodeExporter]

Did you ever have a chance to look into this?

Original comment by csilv...@gmail.com on 2 Sep 2011 at 1:55

[GoogleCodeExporter]

Haven't heard more from this in a while, so I'm closing the bug report.  Feel 
free to reopen if/when you have more information!

Original comment by csilv...@gmail.com on 18 Oct 2011 at 5:43

• Changed state: CannotReproduce