very (Very) minor command line bug, and unexpected result question

[StephenJTurnbull]

In a Ubuntu system I did

pip install gprof2dot

/path/to/MY/executable.exe arg1 arg2 gprof path/to/MY/executable.exe | gprof2dot.py | dot -Tpng -o output.png

and encountered two problems.

firstly my system could not find any gprof2dot.py.

Which was easy I changed it to;

/path/to/your/executable arg1 arg2 gprof path/to/your/executable | gprof2dot | dot -Tpng -o output.png

This produced an output.png that was "MAIN__100.00% (100.00%)"

I compiled /path/to/MY/executable.exe with

/usr//bin/gfortran-4.9 -g -pg -O3 -cpp -Wall -Wtabs -fbacktrace

1. The instructions might want to include a line saying use

gprof path/to/your/executable | gprof2dot.py | dot -Tpng -o output.png or gprof path/to/your/executable | gprof2dot | dot -Tpng -o output.png

1. Do I have the -g-pg flags misplaced?

[zbeekman]

Hi @StephenJTurnbull, I have no official affiliation with gprof2dot, but thought I might weigh in here with some ideas of what might be happening and further steps you could try to understand the behavior you are observing.

First of all, to determine whether or not the issue is even related to gprof2dot, have you examined the output of gprof path/to/your/executable? You could do something like this to get the output and create the image, and then upload both here. (Drag and drop should work)

gprof path/to/your/executable | tee executable.prof.txt | gprof2dot.py | dot -Tpng -o output.png 

If the only profile data you observe in executable.prof.txt is "MAIN__100.00%" then the issue lies with gprof/gfrotran and not gprof2dot.

If this is the case, then I suspect the issue could be optimization. Is it possible that -O3 is causing all the code in your program to be inlined?

My second question, is whats going on with the preprocessing? (You're passing the -cpp flag to gfortran, indicating the Fortran sources should be preprocessed with the C preprocessor.)

[StephenJTurnbull]

Thank you @zbeekman,

You're right my problem is definitely in gprof/gfrotran and not gprof2dot.

The fact that I get when I run my code, but get when I run (c++ ) test code from http://www.thegeekstuff.com/2012/08/gprof-tutorial/ suggest that something is definitely a foot.

Unfortunately the real reason I am running this code is in an attempt to reverse engineer a 100k line 250+ file monster code that I am trying to understand and ultimately edit.

The original coder did not provide a diagram of classes/functions/ or logic flow. so I was hoping that gprof and gprof2dot combined would be able to build an initial tree. or higher-achy.

All of which is to say I don't know why the -cpp or -03 flags are being used. (but I do know that removing the -O3 flag causes it to fail and return: Program received signal SIGFPE: Floating-point exception - erroneous arithmetic operation.

So whatever it is doing is ... necessary.

[zbeekman]

Search the code base for # characters in stuff like #defines etc. #ifdef yada yada. Fortran has no standard for preprocessing, so the -cpp flag is requesting C preprocessor pre-processing services. Definitely ditch the -O3 flag if you are trying to reverse engineer the code and run it again and then through gprof , gprof2dot etc.. You may get a proper call graph.

If you are fortunate enough for the code to be in Fortran >= 90 then you can use the wonderful FORD tool on it. Adding a very simple markup based project configuration file and then executing FORD on your Fortran codebase can yield amazing insight into callgraphs, module dependencies, code structure, etc etc. However if < f90 FORD can't parse the sources correctly. YMMV with mixed language etc. Check the FORD wiki for documentation. All you need to know to get up and running is how to write a minimal project file and invoke ford on the command line pointing it to your source dir.

[jrfonseca]

gfortran is probably inlining more functions.

Please read https://github.com/jrfonseca/gprof2dot#which-options-should-i-pass-to-gcc-when-compiling-for-profiling. Particularly -fno-inline-functions might help.

[StephenJTurnbull]

I tried running without the -O3 flag, the program compiles but then crashed on running.

The code does have

ifdef DEBUGGER

ifdef BENCHMARK

ifdef DEBUGGER_LONG

ifdef MONITORING

. . . flags scattered in many of it's supporting libraries.

I am looking into FORD now.

[zbeekman]

Try @jrfonseca 's suggestion too/first.

Also you could try-O0 instead of -O3.

Crashing when omitting -O3 is very alarming. This most likely means that there's a bug in the code. (If I had to guess I'd put my money on an uninitialized variable) It could be a Gfortran bug but I find this unlikely unless the code uses Fortran 2003 or 2008 features and Gfortran < 5.0

With FORD, assuming the code is > f77, you may need to do some extra work due to the preprocessing directives. Support for handling preprocessing is a WIP for FORD, IIRC.

[StephenJTurnbull]

Three further observations:

Firstly: Re: try-O0 instead of -O3, It crashed, in the same way at the same place.

Second adding FFLAGS += -fno-inline-functions did not change the output in any way.

Thirdly: When I looked at the gprof text output. gprof said that the program had run a total of 0.08 seconds. (actual execution time is on the order of minutes).

The MAIN__ function is really quite small, and could conceivably be running in 0.08 seconds (or even less) but calls a number of modules, which call other modules which call even more modules.

The main function, and all the modules it directly calls(and most of the modules called by them...) are compiled with -pg flags, however it would appear potentially possible that the time spent in the modules is being ignored by gprof.

Does this sound likely?

[jrfonseca]

Yep. It sounds like the modules weren't built with -pg, or that -pg doesn't work well for fortran.

If this is on Linux you might want to try something like Linux perf profiler. It doesn't need source to be compiled with -pg, so it might work better for you.

[StephenJTurnbull]

Thank you for your time and attention. I am a starting to think that -pg is failing for the modules. It is in the list of flags... but not doing much.

[StephenJTurnbull]

I re-wrote the (c++ ) test code from http://www.thegeekstuff.com/2012/08/gprof-tutorial/ into Fortran.

test_gprof.txt test_gprof_new.txt and ran the gfortran-4.9 compiler with the -pg flags.

and got a result: which is similar but not identical to the result above, further supporting the hypothesis that gfortran does not -pg well.

[zbeekman]

Did you compile with -fno-omit-frame-pointers?

DO make sure to try everything in the FAQ on options these should be applied to ALL sources being compiled --fno-omit-frame-pointers -pg -g -O3 (Try lower levels here, 0,1,2... I still find it alarming that -O0 causes runtime errors--definitely a bug in your code or gfortran) -fno-inline -fno-inline-functions-called-once -cpp -fno-inline-functions -fno-optimize-sibling-calls and check the textual output of gprof as well.