[WIP] Lines marked as red or white but never green

[Slamdunk]

Hi, I'm having issues on how PCOV reports some lines.

This is a WIP since I know all the following details still aren't enough for you to help me, I need to dig deeper and isolate better where the problem lies, but still I prefer to open an Issue now, maybe you can help me find it.

Env details

1. Codeception 3.0
2. PHPUnit 8.3.3
3. php-code-coverage 7.0.7
4. PHP 7.3.8
5. PCOV 1.0.6
6. OS: Docker, see https://github.com/Slamdunk/docker-php/blob/master/7.3.Dockerfile#L53

1) Executed lines may differ depending on the run

Two different runs, the only change was removing every @covers annotations from unit and functional tests.

Without @covers:

With @covers:

All 3 lines 60, 61 and 62 are executed for sure, as lines 62 is green, but the output differs.

1. How can lines 60 and 61 marked as red?
2. How is that executed lines differ?

2) Function parameters may be signed as executed or not

1. How can lines 990 and 991 not marked as executed?

[Slamdunk]

1) Executed lines may differ depending on the run

I've been able to pack a self contained repository with all the stuff needed to reproduce the bug:

https://github.com/Slamdunk/pcov-bug

I would expect that for a trivial class like Repository's MyApp the lines reported would be always the same.

I've run the tests with xDebug too, and for one suite PCOV report differs.

I'll continue investigations to understand if and where PCOV is involved, or maybe the issue is within libraries.

[krakjoe]

Sorry, I don't see the error, having reviewed that repository ...

Please try to distil your thoughts further, I'm really failing to understand the problem at this point ...

Simply the fact that pcov and xdebug differ doesn't make pcov wrong ... the reflection class constructor doesn't invoke any code, it just looks up the class and provides a reflector, xdebug takes special action to mark some lines as executable and then detect their use ... the source of truth for pcov is zend (opcache) itself ...

[Slamdunk]

I'm sorry if I captured your attention before a formed thought; I have zero experience in C and Zend Engine, and this issue involves such a big amount of code that inspecting it is overwhelming. I would like to ask you few question that will help me inspecting this.

Take the following code:

1. final class MyApp
2. {
3.     public function foo()
4.     {
5.         return true;
6.     }
7. }

1. Is it true to expect that, given a fixed PHP version, a fixed CC driver and a fixed environment, the reported lines should always be the same, regardless of the PHP testing library and the covered lines?
2. If yes, which lines should be reported?

[Slamdunk]

Oh, just found out the solution, processUncoveredFilesFromWhitelist="true" isn't handled correctly by PHPunit with @coversNothing annotation. I'll open an issue/pr on that repository.

https://phpunit.readthedocs.io/en/8.3/code-coverage-analysis.html#whitelisting-files

Thank you for your patience :pray:

[Slamdunk]

Ref: https://github.com/sebastianbergmann/phpunit/issues/3796