Call to undefined method PHPUnit\Util\TestDox\CliTestDoxPrinter::formatWithColor() using testdox mode

[williamdes]

Q	A
PHPUnit version	8.0.0
PHP version	7.4.2
Installation Method	Composer

Summary

Current behavior

PHPUnit 8.0.0 by Sebastian Bergmann and contributors.

PHP Fatal error:  Uncaught Error: Call to undefined method PHPUnit\Util\TestDox\CliTestDoxPrinter::formatWithColor() in /usr/share/php/PHPUnit/Util/TestDox/CliTestDoxPrinter.php:183
Stack trace:
#0 vendor/phpunit/phpunit/src/Framework/TestResult.php(313): PHPUnit\Util\TestDox\CliTestDoxPrinter->addWarning()
#1 vendor/phpunit/phpunit/src/Framework/TestResult.php(848): PHPUnit\Framework\TestResult->addWarning()
#2 vendor/phpunit/phpunit/src/Framework/TestCase.php(797): PHPUnit\Framework\TestResult->run()
#3 vendor/phpunit/phpunit/src/Framework/TestSuite.php(761): PHPUnit\Framework\TestCase->run()
#4 vendor/phpunit/phpunit/src/Framework/TestSuite.php(761): PHPUnit\Framework\TestSuite->run()
#5 vendor/phpunit/phpunit/src/TextUI/TestRunner.php(632): PHPUnit\Framework\TestSuite->run()
#6 vendor/phpunit/phpunit/src/TextUI/Comma in /usr/share/php/PHPUnit/Util/TestDox/CliTestDoxPrinter.php on line 183

How to reproduce

composer.json

{
    "name": "williamdes/testrepo",
    "require": {
        "phpunit/phpunit": "8.0"
    }
}

phpunit.xml

<phpunit
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="http://schema.phpunit.de/5.4/phpunit.xsd"
        colors="true"
        backupGlobals="false"
        bootstrap="vendor/autoload.php"
        stderr="true"
>

    <testsuites>
        <testsuite name="Unit tests">
            <directory suffix=".php">test/</directory>
        </testsuite>
    </testsuites>

</phpunit>

test/DataTest.php

<?php
declare(strict_types = 1);
namespace Williamdes\MariaDBMySQLKBS\Test;

use \PHPUnit\Framework\TestCase;

class DataTest extends TestCase
{

}

Run ./vendor/bin/phpunit --testdox

Expected behavior

Run tests and show testdox output

Duplicate of https://github.com/sebastianbergmann/phpunit/issues/3906

[epdenouden]

This even misses CliTestDoxPrinter::formatWithColor() not the colorizeTextBox(). Definitely something with code versions. I need to reproduce this locally, from the provided config, code and output I do not understand what's going wrong here.

[williamdes]

phpunit --version
PHPUnit 7.5.6 by Sebastian Bergmann and contributors.

For my apt-get installed phpunit

[sebastianbergmann]

You are not showing how you actually invoke PHPUnit. Just the output is not enough. I need to see which command you invoke.

[williamdes]

You are not showing how you actually invoke PHPUnit. Just the output is not enough. I need to see which command you invoke.

Run ./vendor/bin/phpunit --testdox https://github.com/sebastianbergmann/phpunit/issues/4097#issue-565261804

[epdenouden]

That is not the one being called, right? Your globally installed PHPUnit should not interfere when calling ./vendor/bin/phpunit from inside your project.

A PHPUnit 8. and 7. colliding will most certainly give you strange errors like this!

[sebastianbergmann]

I cannot reproduce this:

$ composer require phpunit/phpunit 8.0.0   
./composer.json has been created
Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 28 installs, 0 updates, 0 removals
  - Installing sebastian/version (2.0.1): Loading from cache
  - Installing sebastian/resource-operations (2.0.1): Loading from cache
  - Installing sebastian/recursion-context (3.0.0): Loading from cache
  - Installing sebastian/object-reflector (1.1.1): Loading from cache
  - Installing sebastian/object-enumerator (3.0.3): Loading from cache
  - Installing sebastian/global-state (3.0.0): Loading from cache
  - Installing sebastian/exporter (3.1.2): Loading from cache
  - Installing sebastian/environment (4.2.3): Loading from cache
  - Installing sebastian/diff (3.0.2): Loading from cache
  - Installing sebastian/comparator (3.0.2): Loading from cache
  - Installing phpunit/php-timer (2.1.2): Loading from cache
  - Installing phpunit/php-text-template (1.2.1): Loading from cache
  - Installing phpunit/php-file-iterator (2.0.2): Loading from cache
  - Installing theseer/tokenizer (1.1.3): Loading from cache
  - Installing sebastian/code-unit-reverse-lookup (1.0.1): Loading from cache
  - Installing phpunit/php-token-stream (3.1.1): Loading from cache
  - Installing phpunit/php-code-coverage (7.0.10): Loading from cache
  - Installing doctrine/instantiator (1.3.0): Loading from cache
  - Installing symfony/polyfill-ctype (v1.14.0): Loading from cache
  - Installing webmozart/assert (1.6.0): Loading from cache
  - Installing phpdocumentor/reflection-common (2.0.0): Loading from cache
  - Installing phpdocumentor/type-resolver (1.0.1): Loading from cache
  - Installing phpdocumentor/reflection-docblock (5.0.0): Loading from cache
  - Installing phpspec/prophecy (v1.10.2): Loading from cache
  - Installing phar-io/version (2.0.1): Loading from cache
  - Installing phar-io/manifest (1.0.3): Loading from cache
  - Installing myclabs/deep-copy (1.9.5): Loading from cache
  - Installing phpunit/phpunit (8.0.0): Downloading (100%)         
sebastian/global-state suggests installing ext-uopz (*)
sebastian/environment suggests installing ext-posix (*)
phpunit/php-code-coverage suggests installing ext-xdebug (^2.7.2)
phpunit/phpunit suggests installing phpunit/php-invoker (^2.0)
phpunit/phpunit suggests installing ext-xdebug (*)
Writing lock file
Generating autoload files

$ cat composer.json                     
{
    "require": {
        "phpunit/phpunit": "8.0.0"
    }
}

$ ./vendor/bin/phpunit --testdox                         
PHPUnit 8.0.0 by Sebastian Bergmann and contributors.

Usage:
  phpunit [options] UnitTest [UnitTest.php]
  phpunit [options] <directory>

Code Coverage Options:
  --coverage-clover <file>    Generate code coverage report in Clover XML format
  --coverage-crap4j <file>    Generate code coverage report in Crap4J XML format
  --coverage-html <dir>       Generate code coverage report in HTML format
  --coverage-php <file>       Export PHP_CodeCoverage object to file
  --coverage-text=<file>      Generate code coverage report in text format [default: standard output]
  --coverage-xml <dir>        Generate code coverage report in PHPUnit XML format
  --whitelist <dir>           Whitelist <dir> for code coverage analysis
  --disable-coverage-ignore   Disable annotations for ignoring code coverage
  --no-coverage               Ignore code coverage configuration
  --dump-xdebug-filter <file> Generate script to set Xdebug code coverage filter

Logging Options:
  --log-junit <file>          Log test execution in JUnit XML format to file
  --log-teamcity <file>       Log test execution in TeamCity format to file
  --testdox-html <file>       Write agile documentation in HTML format to file
  --testdox-text <file>       Write agile documentation in Text format to file
  --testdox-xml <file>        Write agile documentation in XML format to file
  --reverse-list              Print defects in reverse order

Test Selection Options:
  --filter <pattern>          Filter which tests to run
  --testsuite <name>          Filter which testsuite to run
  --group <name>              Only runs tests from the specified group(s)
  --exclude-group <name>      Exclude tests from the specified group(s)
  --list-groups               List available test groups
  --list-suites               List available test suites
  --list-tests                List available tests
  --list-tests-xml <file>     List available tests in XML format
  --test-suffix <suffixes>    Only search for test in files with specified suffix(es). Default: Test.php,.phpt

Test Execution Options:
  --dont-report-useless-tests Do not report tests that do not test anything
  --strict-coverage           Be strict about @covers annotation usage
  --strict-global-state       Be strict about changes to global state
  --disallow-test-output      Be strict about output during tests
  --disallow-resource-usage   Be strict about resource usage during small tests
  --enforce-time-limit        Enforce time limit based on test size
  --default-time-limit=<sec>  Timeout in seconds for tests without @small, @medium or @large
  --disallow-todo-tests       Disallow @todo-annotated tests

  --process-isolation         Run each test in a separate PHP process
  --globals-backup            Backup and restore $GLOBALS for each test
  --static-backup             Backup and restore static attributes for each test

  --colors=<flag>             Use colors in output ("never", "auto" or "always")
  --columns <n>               Number of columns to use for progress output
  --columns max               Use maximum number of columns for progress output
  --stderr                    Write to STDERR instead of STDOUT
  --stop-on-defect            Stop execution upon first not-passed test
  --stop-on-error             Stop execution upon first error
  --stop-on-failure           Stop execution upon first error or failure
  --stop-on-warning           Stop execution upon first warning
  --stop-on-risky             Stop execution upon first risky test
  --stop-on-skipped           Stop execution upon first skipped test
  --stop-on-incomplete        Stop execution upon first incomplete test
  --fail-on-warning           Treat tests with warnings as failures
  --fail-on-risky             Treat risky tests as failures
  -v|--verbose                Output more verbose information
  --debug                     Display debugging information

  --loader <loader>           TestSuiteLoader implementation to use
  --repeat <times>            Runs the test(s) repeatedly
  --teamcity                  Report test execution progress in TeamCity format
  --testdox                   Report test execution progress in TestDox format
  --testdox-group             Only include tests from the specified group(s)
  --testdox-exclude-group     Exclude tests from the specified group(s)
  --printer <printer>         TestListener implementation to use

  --order-by=<order>          Run tests in order: default|reverse|random|defects|no-depends
  --random-order-seed=<N>     Use a specific random seed <N> for random order
  --cache-result              Write test results to cache file
  --do-not-cache-result       Do not write test results to cache file

Configuration Options:
  --prepend <file>            A PHP script that is included as early as possible
  --bootstrap <file>          A PHP script that is included before the tests run
  -c|--configuration <file>   Read configuration from XML file
  --no-configuration          Ignore default configuration file (phpunit.xml)
  --no-logging                Ignore logging configuration
  --no-extensions             Do not load PHPUnit extensions
  --include-path <path(s)>    Prepend PHP's include_path with given path(s)
  -d <key[=value]>            Sets a php.ini value
  --generate-configuration    Generate configuration file with suggested settings
  --cache-result-file=<file>  Specify result cache path and filename

Miscellaneous Options:
  -h|--help                   Prints this usage information
  --version                   Prints the version and exits
  --atleast-version <min>     Checks that version is greater than min and exits
  --check-version             Check whether PHPUnit is the latest version

[williamdes]

@sebastianbergmann do you have a phpunit installed globaly ?

[sebastianbergmann]

Of course not.

[epdenouden]

@williamdes I'll back out of this until later when I have my work laptop in front of me. Yes, there obviously a versioning issue but I have never seen this before; there is something strange in your OS/PHP/PHPUnit/something?-environment.

@sebastianbergmann yeah... it's a strange one

[williamdes]

You know what, I will do a docker version of this issue :)

[sebastianbergmann]

It appears that you have different installations of PHPUnit mixed up.

For instance, you may have used Composer to install PHPUnit and have configured the autoloader generated by Composer as PHPUnit's bootstrap script but then you invoke PHPUnit using an executable other than vendor/bin/phpunit.

[epdenouden]

@williamdes Well this has been a waste of time :-(

cd /tmp/4097
./vendor/bin/phpunit --testdox                                                                                                                                   PHPUnit 8.0.0 by Sebastian Bergmann and contributors.

Time: 38 ms, Memory: 4.00 MB

No tests executed!

Adding a dummy test works just fine. Please have a good look at your environment.

[williamdes]

I understand, for now the docker script is not working fine so I am digging to understand and finally have a solution/reproduce to a bug that exists. Whatever the reason.

[williamdes]

you did not take me seriously

FROM ubuntu:eoan

RUN export DEBIAN_FRONTEND=noninteractive
RUN ln -fs /usr/share/zoneinfo/Europe/Paris /etc/localtime
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install phpunit composer -y

RUN mkdir -p /mnt/Dev/@phpmyadmin/testrepo/test
RUN cd /mnt/Dev/@phpmyadmin/testrepo

COPY composer.json /mnt/Dev/@phpmyadmin/testrepo/composer.json
COPY phpunit.xml /mnt/Dev/@phpmyadmin/testrepo/phpunit.xml
COPY test/DataTest.php /mnt/Dev/@phpmyadmin/testrepo/test/DataTest.php

RUN cd /mnt/Dev/@phpmyadmin/testrepo && composer install
# BUG !
RUN cd /mnt/Dev/@phpmyadmin/testrepo/ && ./vendor/bin/phpunit --testdox
# WORKS !!! WTF
RUN cd /mnt/Dev/@phpmyadmin/testrepo/vendor/bin/phpunit --testdox

Please @sebastianbergmann re-open this issue

docker build ./ -t phpunit-bug

The relative call breaks and the full path call works fine

[epdenouden]

you did not take me seriously

I did. Deleted my jokey comment and tried it in a fresh /tmp folder, which worked fine. I don't doubt you are seeing the errors you are pasting here, yet it is quite obvious there is something going on with different versions of PHPUnit on your system.

Whatever the details: this is an uncommon edge case. I am putting my money on your global install being v7 and the example being v8.

[williamdes]

@sebastianbergmann you want this issue to be fixed or not ?

[epdenouden]

@williamdes Have you tried removing the global install of PHPUnit, also in the docker container?

As said before: trying to debug from afar has been very time consuming, as neither @sebastianbergmann nor me could reproduce the error.

[williamdes]

@williamdes Have you tried removing the global install of PHPUnit, also in the docker container?

All the point is having a phpunit global install. I will not remove it because I package Debian packages and it requires for the package I build to have phpunit installed on the machine.

NB: if I remove global phpunit it the bug disappears

As said before: trying to debug from afar has been very time consuming, as neither @sebastianbergmann nor me could reproduce the error.

But did you run my docker file ? I spent time building it, and it has a 100% reproduce rate for this issue.

I can spend time fixing the bug if necessary ?

[epdenouden]

@williamdes Yes, I just tried it again and you are right: it does fail in this environment. But that is the crux of this issue, it is a problem that is related to environment, packages, (mounted) directories.

For sure it is a problem you encounter while working with PHPUnit but it isn't an @internal problem of PHPUnit itself. If you install any recent version of PHPUnit it will happily fingerpaint your TestDox output.

To get a handle on this I would have to start comparing and debugging the difference between both scenarios. This is too much work for an edge case that is not caused by the internals of PHPUnit itself. However, if you manage to pinpoint the exact issue and make a proper pull request: sure, go for it!

I'll be in my corner refactoring the execution reordering and @dataProvider stuff

[sebastianbergmann]

@williamdes There is no issue here to fix (in PHPUnit).

[williamdes]

            echo 'get_include_path: ' . get_include_path() . PHP_EOL;
            echo 'printerFile: ' . $printerFile . PHP_EOL;

            $printerFile = \stream_resolve_include_path($printerFile);

            echo 'printerFile-stream: ' . $printerFile . PHP_EOL;

$ ./vendor/bin/phpunit --no-coverage --testdox -v

/mnt/Dev/@phpmyadmin/theREALphpMyAdminREPO/vendor/phpunit/phpunit/../../autoload.php

[".\/vendor\/bin\/phpunit","--no-coverage","--testdox","-v"]

ClassPHPUnit\Util\TestDox\CliTestDoxPrinter

get_include_path: .:/usr/share/php:

printerFile: PHPUnit/Util/TestDox/CliTestDoxPrinter.php

printerFile-stream: /usr/share/php/PHPUnit/Util/TestDox/CliTestDoxPrinter.php

PHP Fatal error:  Class PHPUnit\Util\TestDox\CliTestDoxPrinter cannot extend from interface PHPUnit\TextUI\ResultPrinter in /usr/share/php/PHPUnit/Util/TestDox/CliTestDoxPrinter.php on line 429
PHP Stack trace:
PHP   1. {main}() /mnt/Dev/@phpmyadmin/theREALphpMyAdminREPO/vendor/phpunit/phpunit/phpunit:0
PHP   2. PHPUnit\TextUI\Command::main() /mnt/Dev/@phpmyadmin/theREALphpMyAdminREPO/vendor/phpunit/phpunit/phpunit:61
PHP   3. PHPUnit\TextUI\Command->run() /mnt/Dev/@phpmyadmin/theREALphpMyAdminREPO/vendor/phpunit/phpunit/src/TextUI/Command.php:69
PHP   4. PHPUnit\TextUI\Command->handleArguments() /mnt/Dev/@phpmyadmin/theREALphpMyAdminREPO/vendor/phpunit/phpunit/src/TextUI/Command.php:77
PHP   5. PHPUnit\TextUI\Command->handlePrinter() /mnt/Dev/@phpmyadmin/theREALphpMyAdminREPO/vendor/phpunit/phpunit/src/TextUI/Command.php:412

[williamdes]

@epdenouden @sebastianbergmann From the debug data I think you should re-open this issue. Command::handlePrinter has an issue at stream_resolve_include_path because of get_include_path

[williamdes]

:ping_pong: