Set up build automation

[sebastianbergmann]

I cannot get the tests to work on GitHub Actions: https://github.com/sebastianbergmann/phpcov/actions/runs/4136062962/jobs/7149383819

@Slamdunk @localheinz Any idea what I might be doing wrong? Thanks!

[localheinz]

@sebastianbergmann

The tests are failing for me locally as well.

Running

vendor/bin/phpunit --configuration=phpunit.xml

yields

PHPUnit 10.0.7 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.1.14
Configuration: phpunit.xml

..SSS....FF.........                                              20 / 20 (100%)

Time: 00:00.738, Memory: 6.00 MB

There were 2 failures:

1) /Users/am/Sites/sebastianbergmann/phpcov/tests/end-to-end/merge/valid-directory-with-text-report-stdout.phpt
Failed asserting that string matches format description.
--- Expected
+++ Actual
@@ @@

 Generating code coverage report in text format ...

-Code Coverage Report:
-  %s
-
- Summary:
-  Classes: 100.00% (1/1)
-  Methods: 100.00% (2/2)
-  Lines:   66.67% (2/3)
-
-SebastianBergmann\PHPCOV\TestFixture\Greeter
-  Methods: 100.00% ( 2/ 2)   Lines: 100.00% (  2/  2)
+Code Coverage Report:
+  2023-02-09 16:15:39
+
+ Summary:
+  Classes:        (0/0)
+  Methods:        (0/0)
+  Lines:          (0/0)

/Users/am/Sites/sebastianbergmann/phpcov/tests/end-to-end/merge/valid-directory-with-text-report-stdout.phpt:20

2) /Users/am/Sites/sebastianbergmann/phpcov/tests/end-to-end/merge/valid-directory-with-text-report.phpt
Failed asserting that string matches format description.
--- Expected
+++ Actual
@@ @@
 int(0)

-Code Coverage Report:
-  %s
-
- Summary:
-  Classes: 100.00% (1/1)
-  Methods: 100.00% (2/2)
-  Lines:   66.67% (2/3)
-
-SebastianBergmann\PHPCOV\TestFixture\Greeter
-  Methods: 100.00% ( 2/ 2)   Lines: 100.00% (  2/  2)
+Code Coverage Report:
+  2023-02-09 16:15:39
+
+ Summary:
+  Classes:        (0/0)
+  Methods:        (0/0)
+  Lines:          (0/0)

/Users/am/Sites/sebastianbergmann/phpcov/tests/end-to-end/merge/valid-directory-with-text-report.phpt:28

FAILURES!
Tests: 20, Assertions: 17, Failures: 2, Skipped: 3.

[sebastianbergmann]

They work for me locally:

$ XDEBUG_MODE=coverage php -d pcov.enabled=0 vendor/bin/phpunit  
PHPUnit 10.0-dev by Sebastian Bergmann and contributors.

Runtime:       PHP 8.2.2
Configuration: /usr/local/src/phpcov/phpunit.xml

S.................SS                                              20 / 20 (100%)

Time: 00:01.268, Memory: 6.00 MB

OK, but some tests were skipped!
Tests: 20, Assertions: 17, Skipped: 3.

[Slamdunk]

The reason why the suite fails with PCOV is in its readme:

When pcov.directory is left unset, PCOV will attempt to find src, lib or app in the current working directory, in that order; If none are found the current directory will be used, which may waste resources storing coverage information for the test suite.

There is a src dir, so tests gets excluded. If you manually set tests as the collection directory, it works again with PCOV too:

$ php -d pcov.enabled=1 -d pcov.directory=tests vendor/bin/phpunit 
PHPUnit 10.0.7 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.2.2
Configuration: /usr/local/src/phpcov/phpunit.xml

....................                                              20 / 20 (100%)

Time: 00:00.783, Memory: 6.00 MB

OK (20 tests, 20 assertions)

I also bet this is the reason the update-fixture target didn't work when you run it @sebastianbergmann

[sebastianbergmann]

We configure the setup-php action to set up Xdebug for code coverage, not PCOV (https://github.com/sebastianbergmann/phpcov/blob/main/.github/workflows/ci.yml#L73).

PHPUnit prints that it is using Xdebug, not PCOV (https://github.com/sebastianbergmann/phpcov/actions/runs/4136354441/jobs/7150093791#step:5:11).

I am confused :-/

[Slamdunk]

That's another issue: tests/fixture/example/coverage/*.cov files have /usr/local/src/phpcov/[...] as the hardcoded path for the produced coverage data. Indeed to get the test pass I had to move the project under /usr/local/src on my local PC.

I guess the only way is to recreate fixtures before each test run, so it can be easily managed both locally and on CI

[sebastianbergmann]

You are absolutely right!

[sebastianbergmann]

After 337d169d28d12df47189544e5757450f8e830a5b I now run into https://github.com/sebastianbergmann/phpcov/actions/runs/4141720052/jobs/7161596636#step:7:35 because of https://github.com/sebastianbergmann/phpcov/blob/main/tests/end-to-end/patch-coverage/valid-arguments-with-valid-path-prefix.phpt#L11.

Now we "just" need to make $_SERVER['argv'][3] = '/usr/local/src/phpcov'; dynamic.

[sebastianbergmann]

Now we "just" need to make $_SERVER['argv'][3] = '/usr/local/src/phpcov'; dynamic.

Et voilà: 166b95bdc23ebc50ac3e5c7cbe307f4a6f780056