Code that is not executed is reported as being executed

[sebastianbergmann]

When I run the tests for the diff test suite with code coverage reporting based on PCOV then I get two tests marked as risky:

There were 2 risky tests:

1) SebastianBergmann\Diff\DifferTest::testArrayRepresentationOfDiffCanBeRenderedUsingTimeEfficientLcsImplementation with data set #0 (array(array('a', 2), array('b', 1)), 'a', 'b')
This test executed code that is not listed as code to be covered or used:
- SebastianBergmann\Diff\Chunk::__construct
- SebastianBergmann\Diff\Diff::__construct
- SebastianBergmann\Diff\Diff::getChunks
- SebastianBergmann\Diff\Diff::setChunks

2) SebastianBergmann\Diff\MemoryEfficientImplementationTest::testBothEmpty
This test executed code that is not listed as code to be covered or used:
- SebastianBergmann\Diff\Line::__construct
- SebastianBergmann\Diff\Line::getContent

These tests are not marked as risky when code coverage is reported using Xdebug or PHPDBG.

When I add an exit statement to the constructor of Chunk like so

diff --git a/src/Chunk.php b/src/Chunk.php
index d030954..21f3367 100644
--- a/src/Chunk.php
+++ b/src/Chunk.php
@@ -39,6 +39,7 @@ final class Chunk

     public function __construct(int $start = 0, int $startRange = 1, int $end = 0, int $endRange = 1, array $lines = [])
     {
+        exit;
         $this->start      = $start;
         $this->startRange = $startRange;
         $this->end        = $end;

and run the testArrayRepresentationOfDiffCanBeRenderedUsingTimeEfficientLcsImplementation test (./vendor/bin/phpunit --filter testArrayRepresentationOfDiffCanBeRenderedUsingTimeEfficientLcsImplementation) then it works, meaning the exit statement is not reached, meaning that Chunk is not used.

When I add an exit statement to the constructor of Diff like so

diff --git a/src/Diff.php b/src/Diff.php
index 3029e59..ea5c15d 100644
--- a/src/Diff.php
+++ b/src/Diff.php
@@ -34,6 +34,7 @@ final class Diff
      */
     public function __construct(string $from, string $to, array $chunks = [])
     {
+        exit;
         $this->from   = $from;
         $this->to     = $to;
         $this->chunks = $chunks;

and run the testArrayRepresentationOfDiffCanBeRenderedUsingTimeEfficientLcsImplementation test (./vendor/bin/phpunit --filter testArrayRepresentationOfDiffCanBeRenderedUsingTimeEfficientLcsImplementation) then it works, meaning the exit statement is not reached, meaning that Diff is not used.

When I add an exit statement to the constructor of Line like so

diff --git a/src/Line.php b/src/Line.php
index 125bafd..0ec4266 100644
--- a/src/Line.php
+++ b/src/Line.php
@@ -28,6 +28,7 @@ final class Line

     public function __construct(int $type = self::UNCHANGED, string $content = '')
     {
+        exit;
         $this->type    = $type;
         $this->content = $content;
     }

and run the testBothEmpty test (./vendor/bin/phpunit --filter testBothEmpty) then it works, meaning the exit statement is not reached, meaning that Line is not used.

How to run the tests where I see the problem

git clone https://github.com/sebastianbergmann/diff.git
cd diff
composer install
./vendor/bin/phpunit

[krakjoe]

Are you using the latest release of pcov ?

[sebastianbergmann]

Sorry: this is with PHP 7.3.6 and PCOV 1.0.3.

[krakjoe]

krakjoe@fiji:/opt/src/diff$ vendor/bin/phpunit
PHPUnit 8.1.6 by Sebastian Bergmann and contributors.

Runtime:       PHP 7.3.7-dev with PCOV 1.0.3-dev
Configuration: /opt/src/diff/phpunit.xml

.............................................................   61 / 3655 (  1%)
.............................................................  122 / 3655 (  3%)
.............................................................  183 / 3655 (  5%)
.............................................................  244 / 3655 (  6%)
.............................................................  305 / 3655 (  8%)
.............................................................  366 / 3655 ( 10%)
.............................................................  427 / 3655 ( 11%)
.............................................................  488 / 3655 ( 13%)
.............................................................  549 / 3655 ( 15%)
.............................................................  610 / 3655 ( 16%)
.............................................................  671 / 3655 ( 18%)
.............................................................  732 / 3655 ( 20%)
.............................................................  793 / 3655 ( 21%)
.............................................................  854 / 3655 ( 23%)
.............................................................  915 / 3655 ( 25%)
.............................................................  976 / 3655 ( 26%)
............................................................. 1037 / 3655 ( 28%)
............................................................. 1098 / 3655 ( 30%)
............................................................. 1159 / 3655 ( 31%)
............................................................. 1220 / 3655 ( 33%)
............................................................. 1281 / 3655 ( 35%)
............................................................. 1342 / 3655 ( 36%)
............................................................. 1403 / 3655 ( 38%)
............................................................. 1464 / 3655 ( 40%)
............................................................. 1525 / 3655 ( 41%)
............................................................. 1586 / 3655 ( 43%)
............................................................. 1647 / 3655 ( 45%)
............................................................. 1708 / 3655 ( 46%)
............................................................. 1769 / 3655 ( 48%)
............................................................. 1830 / 3655 ( 50%)
............................................................. 1891 / 3655 ( 51%)
............................................................. 1952 / 3655 ( 53%)
............................................................. 2013 / 3655 ( 55%)
............................................................. 2074 / 3655 ( 56%)
............................................................. 2135 / 3655 ( 58%)
............................................................. 2196 / 3655 ( 60%)
............................................................. 2257 / 3655 ( 61%)
............................................................. 2318 / 3655 ( 63%)
............................................................. 2379 / 3655 ( 65%)
............................................................. 2440 / 3655 ( 66%)
............................................................. 2501 / 3655 ( 68%)
............................................................. 2562 / 3655 ( 70%)
............................................................. 2623 / 3655 ( 71%)
............................................................. 2684 / 3655 ( 73%)
............................................................. 2745 / 3655 ( 75%)
............................................................. 2806 / 3655 ( 76%)
............................................................. 2867 / 3655 ( 78%)
............................................................. 2928 / 3655 ( 80%)
............................................................. 2989 / 3655 ( 81%)
............................................................. 3050 / 3655 ( 83%)
............................................................. 3111 / 3655 ( 85%)
............................................................. 3172 / 3655 ( 86%)
............................................................. 3233 / 3655 ( 88%)
............................................................. 3294 / 3655 ( 90%)
............................................................. 3355 / 3655 ( 91%)
............................................................. 3416 / 3655 ( 93%)
............................................................. 3477 / 3655 ( 95%)
............................................................. 3538 / 3655 ( 96%)
............................................................. 3599 / 3655 ( 98%)
........................................................      3655 / 3655 (100%)

Time: 1.33 minutes, Memory: 79.10 MB

OK (3655 tests, 16475 assertions)

Can you tell me any more about your setup, opcache settings maybe ?

I don't think opcache should make any difference, and this looks exactly like #12 which was fixed in 1.0.3 ...

It's definitely of the same nature as #12, it must be the case that some array is not dup'd, there's no way for pcov to include a line that wasn't executed, it can't happen, but #12 showed that the way it reports may be faulty ... although I'm failing to see where, and as you can see, can't even reproduce it :s

[sebastianbergmann]

vendor/bin/phpunit does not generate code coverage. Have you tried vendor/bin/phpunit --coverage-html /tmp/coverage?

[krakjoe]

I can reproduce it ...

[sebastianbergmann]

"Good" :)

[krakjoe]

Fixed in sebastianbergmann/php-code-coverage#682