Some source files show "1 executable, 1 executed" lines for file with thousands of lines of code

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Set up and build Google Unit Test for a source file (e.g., foo.cpp and 
fooTest.cpp)
2. Run the test in XCode 4.6 using C++ LLVM 4.2 compiler with gcov output set 
up.

What is the expected output? What do you see instead?

CoverStory should show the lines of code in foo.cpp which were executed by 
fooTest.cpp.  In the foo.gcda window, each line should display the number of 
times it was executed in the left gutter of the source text pane

Actual output shows a single line (sometimes inside a comment) and the coverage 
is given as "Executed 100% of 1 lines (1 executed, 1 executable, 4313 total 
lines".  The code in the window is grey (unexecuted) except for a single line 
which shows as executed 1 time in the left gutter of the pane.

Number of lines in file is given correctly, executable and executed line counts 
are incorrect.  In the 4313 line example file, there should be 1609 executed 
out of 1610 lines total.

What version of the product are you using? On what operating system?

CoverStory 4.3.0 (no updates available) on Mac OS 10.8.2.

Please provide any additional information below.

Note that the fooTest.gcda file was accurately interpreted, showing 2557 of 
2570 lines being executed, gutter contains expected numbers of executions of 
the different lines and only comment lines are displayed in grey.

The foo.gcda file built by XCode 4.5.2 and C++ LLVM 4.1 compiler was correctly 
interpreted and displayed.  A brief examination of a foo.gcda file shows that 
the 4.2 output differs from the 4.1 output by having additional header data and 
possibly other changes.  This appears to confuse CoverStory but does not cause 
an error message.  The message drawer has five information items but no errors 
or warnings.

Original issue reported on code.google.com by whemb...@qiggroup.com on 31 Jan 2013 at 8:15

[GoogleCodeExporter]

Ugh, so this one sucks.  It seems the clang support for (probably in the 
generated gcda/gcno files) no longer has full paths.  So when invoked with just 
a data file, it fails to find the source.  To fix this we'll have to find all 
the match sources ourselves and change the command line for how gcov is invoked.

Original comment by thoma...@gmail.com on 5 Feb 2013 at 9:21

[GoogleCodeExporter]

Actually, I take that back, it works for the normal cases of xcode doing a 
build.  If you manually invoke clang it's possible to get combinations where 
gcov can't find the sources.  What's your build setup where this isn't working?

Original comment by thoma...@gmail.com on 5 Feb 2013 at 10:18

[GoogleCodeExporter]

The project is built using cmake version 2.8.9, because the actual code targets 
iOS and Win CE.

CMake does use a couple of one-line scripts as part of the build.  They are 
named ALL_BUILD and ZERO_CHECK.  Both run in /bin/sh.  I have substituted (Path 
from root to) for the full path from /, as the lines are otherwise very long

Here's the CMake "ALL_BUILD" command line
make -C (Path from root to)/Build/Debug -f (Path from root 
to)/Build/Debug/CMakeScripts/ALL_BUILD_cmakeRulesBuildPhase.make$CONFIGURATION 
all

Here's the CMake "ZERO_CHECK" command line:
make -C (Path from root to)/Build/Debug -f (Path from root 
to)/Build/Debug/CMakeScripts/ZERO_CHECK_cmakeRulesBuildPhase.make$CONFIGURATION 
all

The above scripts basically invoke makefiles to update the project to reflect 
changes in the files in the project, etc.

Below are the settings for the actual target, named UnitTests.

Architecture is Standard (32/64 bit) and we're building as a 64-bit executable 
with DWARF debugging format.  The build is a Debug build.
Target is "Current Mac OS", 10.8.2 for our separate unit test XCode projects 
where we need the code coverage information from CoverStory.

"Apple LLVM compiler 4.2 - Code Generation" options which are non default, 
i.e., bold-faced, are given below (with the current setting):
* Generate Debug Symbols option (No)
* Inline Methods Hidden (No)
* Optimization Level (-O0 for Debug)
* Symbols Hidden by Default (No)

"Apple LLVM compiler 4.2-Language" not-default option is (with current 
settings):
* Other C++ Flags (-g for Debug)

"Apple LLVM compiler 4.2 - Preprocessing sets the following macros:
* 'CMAKE_INTDIR="$(CONFIGURATION)$(EFFECTIVE_PLATFORM_NAME)"'
* DARWIN_NO_CARBON
* FT2_BUILD_LIBRARY

One puzzling thing is that our GoogleTest unit test files (with names like 
FooTest.cpp) are being compiled by XCode 4.6 and interpreted correctly, giving 
the expected line counts.  I've cleared the project and rebuilt it several 
times with no changes in the outcome, so there is some difference between the 
compiler output for the unit test files and the files being tested, yet both 
are .cpp files and are compiled in the same target.

Original comment by whemb...@qiggroup.com on 5 Feb 2013 at 11:49

[GoogleCodeExporter]

Ok, this isn't specific to Xcode 4.6 then.  What it probably comes down to is 
how CMake is invoking the compiler.  CoverStory invokes gcov on the .gcda/.gcno 
files, gcov then needs to be able to figure out there the source files are to 
generate the proper reports.  Xcode always invokes the compiler with full 
paths, so the full paths get written into the data files and gcov can find the 
source.  Odds are your CMake process is invoking with a relative path, so gcov 
isn't able to find your sources.  If you can tweak your rules to use full paths 
things should work out.  Without the full path, there really isn't a way to 
tell gcov where to find the sources to line up with the data, the alternative 
would be for CoverStory to require you to pick both the data dir and the source 
dir, and then it would have to try and line everything up itself to invoke gcov.

Original comment by thoma...@gmail.com on 6 Feb 2013 at 12:10

[GoogleCodeExporter]

I may have been unclear in my initial description.  The source files are 
displayed in full by CoverStory, so they are being located.  The problem is 
that CoverStory says that foo.cpp has only 1 executable line, when under Xcode 
4.5 it report 1610 executable lines, and CoverStory shows that the line was 
executed once (in 4.5, some lines were shown to have been executed hundreds of 
times).  Since the GoogleTest output shows over 200 tests, in 7 test cases, run 
on that file alone, the single line executed once claim is unambiguously 
incorrect.

One possibly salient observation: I checked one of the associated header files 
displayed in the foo.gcda file's window.  It showed 6 of 6 lines executed (all 
automatically-generated constructors of structs) with some being executed 
hundreds of times.  This in the same window that showed foo.cpp with a single 
executable line executed once.  The fooTest.cpp file's coverage is displayed 
correctly.  Only the foo.cpp file's coverage shows the problem.  This is true 
for all five of the source files tested by this unit test project, and for no 
other files in the project (meaning headers and fooText.cpp-type files).

So the problem exists for .cpp files in one directory but not for .cpp files in 
a different directory, nor for header files in several different directories.  
None of the header files is in the same directory as the "problem" .cpp files.

Cmake does not appear to be driving clang from the command line.  From all that 
I can observe, Xcode is driving the build process and reporting build results 
in its usual manner, in the same way it behaves with normal, non-cmake-built 
projects.

CoverStory works correctly with the output of the project as generated under 
Xcode 4.5 (tested again today on a coworker's machine) but has the problems 
described earlier when the same project is built and run in Xcode 4.6.  The 
project has been under development since May and CoverStory has worked 
flawlessly until I upgraded my build environment to Xcode 4.6.

Paths used:

Checking in the project settings of the UnitTest target, most of the paths are 
full paths and none that I examined differed from the Xcode 4.5 paths.  The 
only relative paths are:
* In Linking, Path to Link Map File 
(Build/Debug/UnitTest.build/UnitTest-LinkMap--i386.txt)
* In Packaging, Private Headers Folder Path and Public Headers Folder Path are 
to subdirectories of UnitTest.app/Contents/
* In Build Locations, the Builds Products Path and Precompiled Headers Cache 
Path are full paths while the other 3 entries are all relative paths.  Entries 
are the same as in the working Xcode 4.5 project, except for an inconsequential 
variation in the precompiled headers path inside /var.

Environment:

The cmake version (2.8.9) has not changed, nor has the OS version, at least in 
the last several weeks.  Checking in git, the last change to any of the files 
used by cmake to build the project was on Jan 4 of this year and that was just 
adding a new header file.  The only change to the build environment that I can 
identify has been the upgrade to Xcode 4.6.

Inference:

This problem is caused by a change in Xcode 4.6 relative to Xcode 4.5.

Original comment by whemb...@qiggroup.com on 6 Feb 2013 at 5:02

[GoogleCodeExporter]

Oops, in the second paragraph of my comment #5, that should have been 
fooTest.cpp, not fooText.cpp

Original comment by whemb...@qiggroup.com on 6 Feb 2013 at 5:16

[GoogleCodeExporter]

Sounds like this is could be a bug in gcov or the instrumentation clang is 
generating.  Any chance you can post a sample project that shows the problem?

Do you have more then one test binary?

Original comment by thoma...@gmail.com on 6 Feb 2013 at 8:48

[GoogleCodeExporter]

I suspect that the problem is in clang.  I examined the foo.gcda files 
generated using both 4.5 and 4.6.  This was just looking at them in a text 
editor (BBEdit) and I didn't try to examine the binary parts.  However, the two 
files had very different text fragments embedded in them, which is certainly 
suggestive.

I can't use any of our code in a sample project, its proprietary for an 
unreleased product and has an NDA that is rather intimidating. I'll try to 
create a sample project starting from an open source project and building a 
UnitTest project for it using the same structure as the problem project.  
Hopefully, I can recreate the problem there.  This may take a couple of days 
since I can't devote a large block of time to this right now.

There is only one test binary in the project, which is the unit tests for a 
library used in a larger project.

Original comment by whemb...@qiggroup.com on 6 Feb 2013 at 9:54

[GoogleCodeExporter]

To clarify the last sentence in my previous comment, the XCode project for the 
unit tests is separate from the main project that builds the library.  We did 
this because the actual library build is for iOS but GoogleTest runs in MacOS, 
not iOS, and cmake can't build a combined Xcode project.

Original comment by whemb...@qiggroup.com on 6 Feb 2013 at 9:58

[GoogleCodeExporter]

I have been able to partially reproduce the problem using an Open Source 
project as a surrogate.  The first attachment, CoverStoryIssue41.txt, is 
effectively a README file with additional analysis of my results and a 
suggestion based on that analysis.  The second file is a .zip file that 
contains both the Google protobuf source and a UnitTest subdirectory with the 
test code.

Original comment by whemb...@qiggroup.com on 13 Feb 2013 at 10:28

Attachments:

• CoverStoryIssue41.txt
• protobuf-2.4.1.zip