Scoped PHAR xdebug issue

sebastianfeldmann commented 5 years ago

Q	A
PHPUnit version	8.0.6
PHP version	7.3.2
Installation Method	PHAR scoped

When running the 8.0.6 scoped phar on a test suite with code coverage I get this error.

phpunit
PHPUnit 8.0.6 by Sebastian Bergmann and contributors.

PHP Fatal error:  Uncaught Error: Call to undefined function PHPUnit\xdebug_start_code_coverage() in phar:///usr/local/bin/phpunit/php-code-coverage/Driver/Xdebug.php:53
Stack trace:
#0 phar:///usr/local/bin/phpunit/php-code-coverage/CodeCoverage.php(210): SebastianBergmann\CodeCoverage\Driver\Xdebug->start(true)
#1 phar:///usr/local/bin/phpunit/phpunit/Framework/TestResult.php(532): SebastianBergmann\CodeCoverage\CodeCoverage->start(Object(CaptainHook\App\Composer\CmdTest))
#2 phar:///usr/local/bin/phpunit/phpunit/Framework/TestCase.php(635): PHPUnit\Framework\TestResult->run(Object(CaptainHook\App\Composer\CmdTest))
#3 phar:///usr/local/bin/phpunit/phpunit/Framework/TestSuite.php(556): PHPUnit\Framework\TestCase->run(Object(PHPUnit\Framework\TestResult))
#4 phar:///usr/local/bin/phpunit/phpunit/Framework/TestSuite.php(556): PHPUnit\Framework\TestSuite->run(Object(PHPUnit\Framework\TestResult))
#5 phar:///usr/local/bin/phpunit/phpunit/TextUI/TestRunner.php(414): PHPUnit\Framework\TestSuite->run(Object(PHPUnit\Framework\TestRe in phar:///usr/local/bin/phpunit/php-code-coverage/Driver/Xdebug.php on line 53

Fatal error: Uncaught Error: Call to undefined function PHPUnit\xdebug_start_code_coverage() in phar:///usr/local/bin/phpunit/php-code-coverage/Driver/Xdebug.php on line 53

Error: Call to undefined function PHPUnit\xdebug_start_code_coverage() in phar:///usr/local/bin/phpunit/php-code-coverage/Driver/Xdebug.php on line 53

Call Stack:
    0.0013     550656   1. {main}() /usr/local/bin/phpunit:0
    0.0838    9760864   2. PHPUnit\TextUI\Command::main() /usr/local/bin/phpunit:619
    0.0838    9760976   3. PHPUnit\TextUI\Command->run() phar:///usr/local/bin/phpunit/phpunit/TextUI/Command.php:71
    0.2137   17031528   4. PHPUnit\TextUI\TestRunner->doRun() phar:///usr/local/bin/phpunit/phpunit/TextUI/Command.php:102
    0.2340   17125680   5. PHPUnit\Framework\TestSuite->run() phar:///usr/local/bin/phpunit/phpunit/TextUI/TestRunner.php:414
    0.2380   17128592   6. PHPUnit\Framework\TestSuite->run() phar:///usr/local/bin/phpunit/phpunit/Framework/TestSuite.php:556
    0.2398   17129248   7. CaptainHook\App\Composer\CmdTest->run() phar:///usr/local/bin/phpunit/phpunit/Framework/TestSuite.php:556
    0.2399   17129248   8. PHPUnit\Framework\TestResult->run() phar:///usr/local/bin/phpunit/phpunit/Framework/TestCase.php:635
    0.2400   17129336   9. SebastianBergmann\CodeCoverage\CodeCoverage->start() phar:///usr/local/bin/phpunit/phpunit/Framework/TestResult.php:532
    0.2400   17129336  10. SebastianBergmann\CodeCoverage\Driver\Xdebug->start() phar:///usr/local/bin/phpunit/php-code-coverage/CodeCoverage.php:210

When I checkout the latest 8.0 branch and create the PHAR myself it works just fine. Is it just me or does someone get the same error with code coverage

Ocramius commented 5 years ago

Maybe this is because the PHAR was built in an environment with/without xDebug installed? May that be the difference?

sebastianfeldmann commented 5 years ago

Could be the case, not sure in what environment @sebastianbergmann is creating the phars

When I look at the scoped code that gets bundled into the scoped phar, I have no clue how this could ever fail, because all xdebug functions are called with a \upfront . And because the CodeCoverageDriver stuff is on a scoping whitelist so it doesn't get scoped anyway.

        if ($determineUnusedAndDead) {
            \xdebug_start_code_coverage(\XDEBUG_CC_UNUSED | \XDEBUG_CC_DEAD_CODE);
        } else {
            \xdebug_start_code_coverage();
        }

sebastianbergmann commented 5 years ago

@theofidry ping

sebastianbergmann commented 5 years ago

Yes, I created those PHARs in an environment where Xdebug was not loaded.

theofidry commented 5 years ago

Yes, I created those PHARs in an environment where Xdebug was not loaded.

Urg this is annoying. I think that's pretty much the issue since the Reflector will think it's an internal function as it should. So I think there is two immediate solutions:

load xdebug in the environment
deal with it with patchers

The long-term solution is to improve BetterReflection (which PHP-Scoper Reflector is using) in order to spot which function is effectively internal or not without having to load any code.

Ocramius commented 5 years ago

@theofidry I think that would need https://github.com/Roave/BetterReflection/pull/373

sebastianbergmann commented 5 years ago

I have uploaded an updated scoped PHAR to https://phar.phpunit.de/phpunit-scoped-8.0.6-1-g3faa14214.phar that was built with 3faa14214cb017d0901380c99bf3fb8eb9f75c21 and Xdebug enabled in the PHP runtime that was used to run PHP-Scoper.

Can you please check whether this solves your problem? Thanks!

sebastianfeldmann commented 5 years ago

Yes now it works with code coverage as well.

theseer commented 5 years ago

While it might work for xdebug now, the problem persists for pcov:

theseer@nyda ~/storage/php/tokenizer master $ ./tools/phpunit-scoped-8.0.6-1-g3faa14214.phar 
PHPUnit 8.0.6-1-g3faa14214 by Sebastian Bergmann and contributors.

Runtime:       PHP 7.3.3 with PCOV 1.0.1
Configuration: /home/theseer/storage/php/tokenizer/phpunit.xml

PHP Fatal error:  Uncaught Error: Call to undefined function PHPUnit\pcov\start() in phar:///home/theseer/storage/php/tokenizer/tools/phpunit-scoped-8.0.6-1-g3faa14214.phar/php-code-coverage/Driver/PCOV.php:30

Or should that be a new issue?

sebastianbergmann commented 5 years ago

It's a different manifestation of the same underlying issue. And to fix that we need https://github.com/Roave/BetterReflection/pull/373.

sebastianbergmann commented 5 years ago

@theofidry I wonder whether php-scoper really needs to know that a function or class is built-in (provided by an extension). What happens right now is that php-scoper sees a call to a function without seeing the declaration of that function. Would it not be better then to simply not change calls to units of code for which the declaration is not part of the code that is to prefixed?

Ocramius commented 5 years ago

Is the issue specifically about internal/non-internal or just defined/not-defined?

To check this: what happens if those functions are explicitly imported via use function?

theofidry commented 5 years ago

Would it not be better then to simply not change calls to units of code for which the declaration is not part of the code that is to prefixed?

The problem is it's hard to guess that. PHP-Scoper doesn't load all the code before scoping (and even if it were you may have some code not loaded under certain conditions) so it can't do any guess work there.

IMO there is only two good ways to improve the approach here:

Patch BetterReflection which will improve the detection of internal code (e.g. right now if you run PHP-Scoper with PHP 7.2, a new PHP 7.3 function will appear as non-internal); Note that PHP-Scoper includes some "hotfix" meanwhile.
Have yet another option for whitelisting, cf. https://github.com/humbug/php-scoper/issues/305, which would be "just don't change this code at all". I didn't want to introduce it until now. It was actually one of the first approach but caused a lot of troubles at that time, so it couldn't be used for the "default whitelist mechanism". It's however necessary in some scenarios so now that PHP-Scoper is more stable & tested, I think now is an appropriate time to do so

Meanwhile, isn't it possible to build the scoped PHAR with the latest supported PHP version and all the extensions PHPUnit interacts with? It is not ideal but at least will would reduce the number of issues caused by an extension functions being scoped incorrectly

sebastianbergmann commented 5 years ago

Meanwhile, isn't it possible to build the scoped PHAR with the latest supported PHP version and all the extensions PHPUnit interacts with?

That may work for PCOV and Xdebug but I am not so sure about PHPDBG because that is not an extension but a different SAPI.

~~What could work, though, is to put https://github.com/JetBrains/phpstorm-stubs into the directory that is processed by php-scoper and then do not package that directory into the PHAR.~~

That makes no sense, sorry for the noise.

theofidry commented 5 years ago

I think relying on patchers is the only way to fix it in the meantime then :/

sebastianbergmann commented 5 years ago

@theofidry Re: https://github.com/sebastianbergmann/phpunit/issues/3577#issuecomment-479846964

Correct me if I am wrong, but that "hotfix" does not help us as the KNOWN_INTERNAL_FUNCTIONS constant does not contain all built-in functions, most notably the ones in question here (ext/pcov and ext/xdebug).

Depending on how long we have to wait for https://github.com/Roave/BetterReflection/pull/373, you may want to consider depending on https://github.com/JetBrains/phpstorm-stubs in php-scoper to fill the KNOWN_INTERNAL_FUNCTIONS array.

I briefly looked at patchers but am not sure how they could help us.

Ocramius commented 5 years ago

Speaking of which, if you need to speed up merging Roave/BetterReflection#373, please do help in reviewing. I just noticed that it has been ready for weeks now :|

Ocramius commented 5 years ago

Heads up: Roave/BetterReflection#373 is merged, and 3.3.0 of said package was released.

sebastianbergmann commented 5 years ago

Thank you, @kukulich and @Ocramius!

theofidry commented 5 years ago

Thanks! Will check it out in details once back from holidays :)

On Sat 20 Apr 2019 at 17:24, Sebastian Bergmann notifications@github.com wrote:

Thank you, @kukulich https://github.com/kukulich and @Ocramius https://github.com/Ocramius!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/sebastianbergmann/phpunit/issues/3577#issuecomment-485069565, or mute the thread https://github.com/notifications/unsubscribe-auth/ABHPVANKF4FZUDTCCFHJNULPRLHLTANCNFSM4HBOXUCA .

theofidry commented 5 years ago

Just a small heads up: there is a lot of changes coming from BetterReflection and I'm doing some more work on both PHP-Scoper and Box. I can't give an ETA (I hope a couple of weeks) but I really hope the next version is going to help out a lot to move forward for this PR :)

sebastianbergmann commented 5 years ago

Thanks a lot, @theofidry, for all the work you do on PHP-Scoper!

sebastianbergmann commented 5 years ago

Please test with https://phar.phpunit.de/phpunit-scoped-8.2.0.phar which was built using the latest PHP-Scoper.

theseer commented 5 years ago

theseer@nyda ~/storage/php/templado/engine master $ ./tools/phpunit-scoped-8.2.0.phar 
PHPUnit 8.2.0 by Sebastian Bergmann and contributors.

Runtime:       PHP 7.3.6 with PCOV 1.0.4
Configuration: /home/theseer/storage/php/templado/engine/phpunit.xml

PHP Fatal error:  Uncaught Error: Call to undefined function PHPUnit\pcov\start() in phar:///home/theseer/storage/php/templado/engine/tools/phpunit-scoped-8.2.0.phar/php-code-coverage/Driver/PCOV.php:26

kukulich commented 5 years ago

It looks there is no pcov in https://github.com/JetBrains/phpstorm-stubs yet :(

theofidry commented 5 years ago

It should be fixable with patchers although it's a PITA and would be nice to fix it upstream in phpstorm stubs

sebastianbergmann commented 5 years ago

Please test with https://phar.phpunit.de/phpunit-scoped-8.2.2.phar. It was built with the latest version of PHP-Scoper and should solve this problem.

theseer commented 5 years ago

theseer@nyda ~/storage/php/tokenizer master $ ./tools/phpunit-scoped-8.2.2.phar 
PHPUnit 8.2.2 by Sebastian Bergmann and contributors.

Runtime:       PHP 7.3.6 with PCOV 1.0.6
Configuration: /home/theseer/storage/php/tokenizer/phpunit.xml

PHP Fatal error:  Uncaught Error: Call to undefined function PHPUnit\pcov\start() in phar:///home/theseer/storage/php/tokenizer/tools/phpunit-scoped-8.2.2.phar/php-code-coverage/Driver/PCOV.php:26

theofidry commented 5 years ago

@sebastianbergmann can you confirm it was PHP-Scoper 0.12.1+?

sebastianbergmann commented 5 years ago

https://github.com/sebastianbergmann/phpunit/blob/8.2.2/tools/php-scoper was used to create the scoped PHAR for PHPUnit 8.2.2.

theofidry commented 5 years ago

Arg, ofc I'm such an idiot:

private const MISSING_CLASSES = ['UV', '_HumbugBox2333ac9fbdb0\\Crypto\\Cipher', '_HumbugBox2333ac9fbdb0\\Crypto\\CipherException', '_HumbugBox2333ac9fbdb0\\Crypto\\Hash', '_HumbugBox2333ac9fbdb0\\Crypto\\HashException', '_HumbugBox2333ac9fbdb0\\Crypto\\MAC', '_HumbugBox2333ac9fbdb0\\Crypto\\MACException', '_HumbugBox2333ac9fbdb0\\Crypto\\HMAC', '_HumbugBox2333ac9fbdb0\\Crypto\\CMAC', '_HumbugBox2333ac9fbdb0\\Crypto\\KDF', '_HumbugBox2333ac9fbdb0\\Crypto\\KDFException', '_HumbugBox2333ac9fbdb0\\Crypto\\PBKDF2', '_HumbugBox2333ac9fbdb0\\Crypto\\PBKDF2Exception', '_HumbugBox2333ac9fbdb0\\Crypto\\Base64', '_HumbugBox2333ac9fbdb0\\Crypto\\Base64Exception', '_HumbugBox2333ac9fbdb0\\Crypto\\Rand', '_HumbugBox2333ac9fbdb0\\Crypto\\RandException', '_HumbugBox2333ac9fbdb0\\parallel\\Channel', '_HumbugBox2333ac9fbdb0\\parallel\\Channel\\Error', '_HumbugBox2333ac9fbdb0\\parallel\\Channel\\Error\\Closed', '_HumbugBox2333ac9fbdb0\\parallel\\Channel\\Error\\Existence', '_HumbugBox2333ac9fbdb0\\parallel\\Channel\\Error\\IllegalValue', '_HumbugBox2333ac9fbdb0\\parallel\\Error', '_HumbugBox2333ac9fbdb0\\parallel\\Events', '_HumbugBox2333ac9fbdb0\\parallel\\Events\\Error', '_HumbugBox2333ac9fbdb0\\parallel\\Events\\Error\\Existence', '_HumbugBox2333ac9fbdb0\\parallel\\Events\\Error\\Timeout', '_HumbugBox2333ac9fbdb0\\parallel\\Events\\Event', '_HumbugBox2333ac9fbdb0\\parallel\\Events\\Event\\Type', '_HumbugBox2333ac9fbdb0\\parallel\\Events\\Input', '_HumbugBox2333ac9fbdb0\\parallel\\Events\\Input\\Error', '_HumbugBox2333ac9fbdb0\\parallel\\Events\\Input\\Error\\Existence', '_HumbugBox2333ac9fbdb0\\parallel\\Events\\Input\\Error\\IllegalValue', '_HumbugBox2333ac9fbdb0\\parallel\\Future', '_HumbugBox2333ac9fbdb0\\parallel\\Future\\Error', '_HumbugBox2333ac9fbdb0\\parallel\\Future\\Error\\Cancelled', '_HumbugBox2333ac9fbdb0\\parallel\\Future\\Error\\Foreign', '_HumbugBox2333ac9fbdb0\\parallel\\Future\\Error\\Killed', '_HumbugBox2333ac9fbdb0\\parallel\\Runtime', '_HumbugBox2333ac9fbdb0\\parallel\\Runtime\\Bootstrap', '_HumbugBox2333ac9fbdb0\\parallel\\Runtime\\Error', '_HumbugBox2333ac9fbdb0\\parallel\\Runtime\\Error\\Bootstrap', '_HumbugBox2333ac9fbdb0\\parallel\\Runtime\\Error\\Closed', '_HumbugBox2333ac9fbdb0\\parallel\\Runtime\\Error\\IllegalFunction', '_HumbugBox2333ac9fbdb0\\parallel\\Runtime\\Error\\IllegalInstruction', '_HumbugBox2333ac9fbdb0\\parallel\\Runtime\\Error\\IllegalParameter', '_HumbugBox2333ac9fbdb0\\parallel\\Runtime\\Error\\IllegalReturn'];
    private const MISSING_FUNCTIONS = ['sapi_windows_vt100_support', 'uv_unref', 'uv_last_error', 'uv_err_name', 'uv_strerror', 'uv_update_time', 'uv_ref', 'uv_run', 'uv_run_once', 'uv_loop_delete', 'uv_now', 'uv_tcp_bind', 'uv_tcp_bind6', 'uv_write', 'uv_write2', 'uv_tcp_nodelay', 'uv_accept', 'uv_shutdown', 'uv_close', 'uv_read_start', 'uv_read2_start', 'uv_read_stop', 'uv_ip4_addr', 'uv_ip6_addr', 'uv_listen', 'uv_tcp_connect', 'uv_tcp_connect6', 'uv_timer_init', 'uv_timer_start', 'uv_timer_stop', 'uv_timer_again', 'uv_timer_set_repeat', 'uv_timer_get_repeat', 'uv_idle_init', 'uv_idle_start', 'uv_idle_stop', 'uv_getaddrinfo', 'uv_tcp_init', 'uv_default_loop', 'uv_loop_new', 'uv_udp_init', 'uv_udp_bind', 'uv_udp_bind6', 'uv_udp_recv_start', 'uv_udp_recv_stop', 'uv_udp_set_membership', 'uv_udp_set_multicast_loop', 'uv_udp_set_multicast_ttl', 'uv_udp_set_broadcast', 'uv_udp_send', 'uv_udp_send6', 'uv_is_active', 'uv_is_readable', 'uv_is_writable', 'uv_walk', 'uv_guess_handle', 'uv_handle_type', 'uv_pipe_init', 'uv_pipe_open', 'uv_pipe_bind', 'uv_pipe_connect', 'uv_pipe_pending_instances', 'uv_ares_init_options', 'ares_gethostbyname', 'uv_loadavg', 'uv_uptime', 'uv_get_free_memory', 'uv_get_total_memory', 'uv_hrtime', 'uv_exepath', 'uv_cpu_info', 'uv_interface_addresses', 'uv_stdio_new', 'uv_spawn', 'uv_process_kill', 'uv_kill', 'uv_chdir', 'uv_rwlock_init', 'uv_rwlock_rdlock', 'uv_rwlock_tryrdlock', 'uv_rwlock_rdunlock', 'uv_rwlock_wrlock', 'uv_rwlock_trywrlock', 'uv_rwlock_wrunlock', 'uv_mutex_init', 'uv_mutex_lock', 'uv_mutex_trylock', 'uv_sem_init', 'uv_sem_post', 'uv_sem_wait', 'uv_sem_trywait', 'uv_prepare_init', 'uv_prepare_start', 'uv_prepare_stop', 'uv_check_init', 'uv_check_start', 'uv_check_stop', 'uv_async_init', 'uv_async_send', 'uv_queue_work', 'uv_fs_open', 'uv_fs_read', 'uv_fs_close', 'uv_fs_write', 'uv_fs_fsync', 'uv_fs_fdatasync', 'uv_fs_ftruncate', 'uv_fs_mkdir', 'uv_fs_rmdir', 'uv_fs_unlink', 'uv_fs_rename', 'uv_fs_utime', 'uv_fs_futime', 'uv_fs_chmod', 'uv_fs_fchmod', 'uv_fs_chown', 'uv_fs_fchown', 'uv_fs_link', 'uv_fs_symlink', 'uv_fs_readlink', 'uv_fs_stat', 'uv_fs_lstat', 'uv_fs_fstat', 'uv_fs_readdir', 'uv_fs_sendfile', 'uv_fs_event_init', 'uv_tty_init', 'uv_tty_get_winsize', 'uv_tty_set_mode', 'uv_tty_reset_mode', 'uv_tcp_getsockname', 'uv_tcp_getpeername', 'uv_udp_getsockname', 'uv_resident_set_memory', 'uv_ip4_name', 'uv_ip6_name', 'uv_poll_init', 'uv_poll_start', 'uv_poll_stop', 'uv_fs_poll_init', 'uv_fs_poll_start', 'uv_fs_poll_stop', 'uv_stop', 'uv_signal_stop', '_HumbugBox2333ac9fbdb0\\parallel\\bootstrap', '_HumbugBox2333ac9fbdb0\\parallel\\run', '_HumbugBox2333ac9fbdb0\\pcov\\collect', '_HumbugBox2333ac9fbdb0\\pcov\\start', '_HumbugBox2333ac9fbdb0\\pcov\\stop', '_HumbugBox2333ac9fbdb0\\pcov\\clear', '_HumbugBox2333ac9fbdb0\\pcov\\waiting', '_HumbugBox2333ac9fbdb0\\pcov\\memory'];
    private const MISSING_CONSTANTS = ['STDIN', 'STDOUT', 'STDERR', '_HumbugBox2333ac9fbdb0\\pcov\\all', '_HumbugBox2333ac9fbdb0\\pcov\\inclusive', '_HumbugBox2333ac9fbdb0\\pcov\\exclusive'];

Will fix that ASAP

sebastianbergmann commented 5 years ago

@theofidry I think somebody should tell you this: you are not an idiot and your work is very much appreciated :-)

theofidry commented 5 years ago

Thanks :) Will try to fix that issue today, if I cannot it will have to way Monday/Tuesday

theofidry commented 5 years ago

Here you go: https://github.com/humbug/php-scoper/releases/tag/0.12.4

sebastianbergmann commented 5 years ago

Please test with https://phar.phpunit.de/phpunit-scoped-8.2.4.phar which was built using php-scoper 0.12.4.

theseer commented 5 years ago

theseer@nyda ~/storage/php/templado/engine master $ ./tools/phpunit-scoped-8.2.4.phar 
PHPUnit 8.2.4 by Sebastian Bergmann and contributors.

Runtime:       PHP 7.3.6 with PCOV 1.0.6
Configuration: /home/theseer/storage/php/templado/engine/phpunit.xml

...............................................................  63 / 182 ( 34%)
............................................................... 126 / 182 ( 69%)
........................................................        182 / 182 (100%)

Time: 291 ms, Memory: 18.00 MB

OK (182 tests, 624 assertions)

Generating code coverage report in HTML format ... done [97 ms]

Generating code coverage report in PHPUnit XML format ... done [123 ms]

\o/

sebastianbergmann commented 5 years ago

I think this has progressed enough to stop offering non-scoped PHARs of PHPUnit starting with PHPUnit 8.3.

Thank you to everyone involved in making this happen!

sebastianbergmann / phpunit

Scoped PHAR xdebug issue #3577