search

paratestphp / paratest

:computer: Parallel testing for PHPUnit
MIT License
2.3k stars 226 forks source link

Bug with --testdox option at v7.4.0 #839

Closed sertxudev closed 7 months ago

sertxudev commented 7 months ago
Q A
ParaTest version 7.4.0
PHPUnit version 10.5.10
PHP version 8.2.11

Summary

Hi, good evening,

There's a bug with the new v7.4.0, if I downgrade to v7.3.1 it works as expected. It seems the bug is related to the option --testdox at v7.4.0, if I remove that parameter works as expected.

Feel free to ask if you want me to test anything. šŸ˜„

Thanks!

Current behavior

ParaTest v7.4.0 with --testdox (Laravel Artisan)

/var/www/html # php artisan test --without-tty --parallel --testdox
ParaTest v7.4.0 upon PHPUnit 10.5.10 by Sebastian Bergmann and contributors.

Processes:     4
Runtime:       PHP 8.2.11
Configuration: /var/www/html/phpunit.xml

In WorkerCrashedException.php line 41:

  The test "PARATEST='1' TEST_TOKEN='1' UNIQUE_TEST_TOKEN='1_65c10b9d06124' t
  ests/Unit/ExampleTest.php" failed.

  Exit Code: 255(Unknown error)

  Working directory: /var/www/html

  Output:
  ================

  Error Output:
  ================
  PHP Warning:  Undefined array key "phpunit-argv" in /var/www/html/vendor/br
  ianium/paratest/bin/phpunit-wrapper.php on line 47
  PHP Fatal error:  Uncaught TypeError: unserialize(): Argument #1 ($data) mu
  st be of type string, null given in /var/www/html/vendor/brianium/paratest/
  bin/phpunit-wrapper.php:47
  Stack trace:
  #0 /var/www/html/vendor/brianium/paratest/bin/phpunit-wrapper.php(47): unse
  rialize(NULL, Array)
  #1 /var/www/html/vendor/brianium/paratest/bin/phpunit-wrapper.php(79): {clo
  sure}()
  #2 {main}
    thrown in /var/www/html/vendor/brianium/paratest/bin/phpunit-wrapper.php
  on line 47

paratest [--functional] [-m|--max-batch-size MAX-BATCH-SIZE] [--no-test-tokens] [--passthru-php PASSTHRU-PHP] [-p|--processes PROCESSES] [--runner RUNNER] [--tmp-dir TMP-DIR] [-v|--verbose] [--bootstrap BOOTSTRAP] [-c|--configuration CONFIGURATION] [--no-configuration] [--cache-directory CACHE-DIRECTORY] [--testsuite TESTSUITE] [--exclude-testsuite EXCLUDE-TESTSUITE] [--group GROUP] [--exclude-group EXCLUDE-GROUP] [--filter FILTER] [--process-isolation] [--strict-coverage] [--strict-global-state] [--disallow-test-output] [--dont-report-useless-tests] [--stop-on-defect] [--stop-on-error] [--stop-on-failure] [--stop-on-warning] [--stop-on-risky] [--stop-on-skipped] [--stop-on-incomplete] [--fail-on-incomplete] [--fail-on-risky] [--fail-on-skipped] [--fail-on-warning] [--order-by ORDER-BY] [--random-order-seed RANDOM-ORDER-SEED] [--colors [COLORS]] [--no-progress] [--display-incomplete] [--display-skipped] [--display-deprecations] [--display-errors] [--display-notices] [--display-warnings] [--teamcity] [--testdox] [--log-junit LOG-JUNIT] [--log-teamcity LOG-TEAMCITY] [--coverage-clover COVERAGE-CLOVER] [--coverage-cobertura COVERAGE-COBERTURA] [--coverage-crap4j COVERAGE-CRAP4J] [--coverage-html COVERAGE-HTML] [--coverage-php COVERAGE-PHP] [--coverage-text [COVERAGE-TEXT]] [--coverage-xml COVERAGE-XML] [--coverage-filter COVERAGE-FILTER] [--no-coverage] [--] [<path>]

ParaTest v7.4.0 with --testdox (Paratest binary)

/var/www/html # vendor/bin/paratest --testdox
ParaTest v7.4.0 upon PHPUnit 10.5.10 by Sebastian Bergmann and contributors.

Processes:     4
Runtime:       PHP 8.2.11
Configuration: /var/www/html/phpunit.xml

In WorkerCrashedException.php line 41:

  The test "PARATEST='1' TEST_TOKEN='1' UNIQUE_TEST_TOKEN='1_65c10cea49f7e' tests/Unit/ExampleTest.php" failed.

  Exit Code: 255(Unknown error)

  Working directory: /var/www/html

  Output:
  ================

  Error Output:
  ================
  PHP Warning:  Undefined array key "phpunit-argv" in /var/www/html/vendor/brianium/paratest/bin/phpunit-wrapper.php on line 47
  PHP Fatal error:  Uncaught TypeError: unserialize(): Argument #1 ($data) must be of type string, null given in /var/www/html/vendor/brianium/paratest/bin/phpunit-wrapper.php:47
  Stack trace:
  #0 /var/www/html/vendor/brianium/paratest/bin/phpunit-wrapper.php(47): unserialize(NULL, Array)
  #1 /var/www/html/vendor/brianium/paratest/bin/phpunit-wrapper.php(79): {closure}()
  #2 {main}
    thrown in /var/www/html/vendor/brianium/paratest/bin/phpunit-wrapper.php on line 47

How to reproduce: command, code and error stack trace

I'm using the following command:

php artisan test --without-tty --parallel --testdox

Expected behavior

ParaTest v7.3.1 with --testdox (Laravel Artisan)

/var/www/html # php artisan test --without-tty --parallel --testdox
ParaTest v7.3.1 upon PHPUnit 10.5.10 by Sebastian Bergmann and contributors.

Processes:     4
Runtime:       PHP 8.2.11
Configuration: /var/www/html/phpunit.xml

........................                                          24 / 24 (100%)

Time: 00:02.250, Memory: 26.00 MB

Example (Tests\Unit\Example)
 āœ” That true is true

Locale (Tests\Feature\Locale)
 āœ” The application returns a successful response
 āœ” The application redirects to default path
 āœ” The application returns a successful response in french
 āœ” The application returns a successful response in english

Authentication (Tests\Feature\Auth\Authentication)
 āœ” Login screen can be rendered
 āœ” Users can authenticate using the login screen
 āœ” Users can logout
 āœ” Users can not authenticate with invalid password
 āœ” Users can not authenticate with inactive account
 āœ” Users gets redirected to business
 āœ” User gets redirected to preferred locale after login

Password Reset (Tests\Feature\Auth\PasswordReset)
 āœ” Reset password link screen can be rendered
 āœ” Reset password link can be requested
 āœ” Reset password screen can be rendered
 āœ” Password can be reset with valid token

Create User (Tests\Feature\Commands\CreateUser)
 āœ” Can create user
 āœ” Cannot create user with existing email
 āœ” Cannot create user with existing username
 āœ” Can create user with password
 āœ” Can not create user with password mismatch
 āœ” Can login with created user
 āœ” Can create user with notification
 āœ” Can create user without notification

OK (24 tests, 106 assertions)

ParaTest v7.4.0 without --testdox (Laravel Artisan)

/var/www/html # php artisan test --without-tty --parallel
ParaTest v7.4.0 upon PHPUnit 10.5.10 by Sebastian Bergmann and contributors.

Processes:     4
Runtime:       PHP 8.2.11
Configuration: /var/www/html/phpunit.xml

........................                                          24 / 24 (100%)

Time: 00:02.152, Memory: 26.00 MB

OK (24 tests, 106 assertions)
rleguizamon01 commented 7 months ago

it happened to me too, had to downgrade to 7.3.1

Slamdunk commented 7 months ago

I am unable to reproduce this in ParaTest, maybe there's some layer in Laravel that's unrelated to ParaTest.

Can you create a dedicated repo that reproduces the issue?

sertxudev commented 7 months ago

Sure! Here you are https://github.com/sertxudev/paratest-bug

It's a Laravel fresh installation, I've only changed the database from MySQL to SQLite. Running the default tests fails.

vendor\bin\paratest --testdox

Z:\Website_Developments\home\laravel\data>vendor\bin\paratest --testdox
ParaTest v7.4.0 upon PHPUnit 10.5.10 by Sebastian Bergmann and contributors.

Processes:     6
Runtime:       PHP 8.2.3
Configuration: Z:\Website_Developments\home\laravel\data\phpunit.xml

In WorkerCrashedException.php line 41:

  The test "PARATEST="1" TEST_TOKEN="1" UNIQUE_TEST_TOKEN="1_65c212950b915" tests\Unit\ExampleTest.php" failed.

  Exit Code: 255(Unknown error)

  Working directory: Z:\Website_Developments\home\laravel\data

  Output:
  ================

  Fatal error: Uncaught AssertionError: assert(isset($getopt['phpunit-argv']) && is_string($getopt['phpunit-argv']))
  in Z:\Website_Developments\home\laravel\data\vendor\brianium\paratest\bin\phpunit-wrapper.php:46
  Stack trace:
  #0 Z:\Website_Developments\home\laravel\data\vendor\brianium\paratest\bin\phpunit-wrapper.php(46): assert(false, 'a
  ssert(isset($g...')
  #1 Z:\Website_Developments\home\laravel\data\vendor\brianium\paratest\bin\phpunit-wrapper.php(79): {closure}()
  #2 {main}
    thrown in Z:\Website_Developments\home\laravel\data\vendor\brianium\paratest\bin\phpunit-wrapper.php on line 46

  Error Output:
  ================
  PHP Fatal error:  Uncaught AssertionError: assert(isset($getopt['phpunit-argv']) && is_string($getopt['phpunit-argv
  '])) in Z:\Website_Developments\home\laravel\data\vendor\brianium\paratest\bin\phpunit-wrapper.php:46
  Stack trace:
  #0 Z:\Website_Developments\home\laravel\data\vendor\brianium\paratest\bin\phpunit-wrapper.php(46): assert(false, 'a
  ssert(isset($g...')
  #1 Z:\Website_Developments\home\laravel\data\vendor\brianium\paratest\bin\phpunit-wrapper.php(79): {closure}()
  #2 {main}
    thrown in Z:\Website_Developments\home\laravel\data\vendor\brianium\paratest\bin\phpunit-wrapper.php on line 46

php artisan test --parallel --testdox

Z:\Website_Developments\home\laravel\data>php artisan test --parallel --testdox
ParaTest v7.4.0 upon PHPUnit 10.5.10 by Sebastian Bergmann and contributors.

Processes:     6
Runtime:       PHP 8.2.3
Configuration: Z:\Website_Developments\home\laravel\data\phpunit.xml

In WorkerCrashedException.php line 41:

  The test "PARATEST="1" TEST_TOKEN="1" UNIQUE_TEST_TOKEN="1_65c2130ac243c" tests\Unit\ExampleTest.php" failed.

  Exit Code: 255(Unknown error)

  Working directory: Z:\Website_Developments\home\laravel\data

  Output:
  ================

  Fatal error: Uncaught AssertionError: assert(isset($getopt['phpunit-argv']) && is_string($getopt['phpunit-argv'])) in Z:\Website_Developments\home\laravel\d
  ata\vendor\brianium\paratest\bin\phpunit-wrapper.php:46
  Stack trace:
  #0 Z:\Website_Developments\home\laravel\data\vendor\brianium\paratest\bin\phpunit-wrapper.php(46): assert(false, 'assert(isset($g...')
  #1 Z:\Website_Developments\home\laravel\data\vendor\brianium\paratest\bin\phpunit-wrapper.php(79): {closure}()
  #2 {main}
    thrown in Z:\Website_Developments\home\laravel\data\vendor\brianium\paratest\bin\phpunit-wrapper.php on line 46

  Error Output:
  ================
  PHP Fatal error:  Uncaught AssertionError: assert(isset($getopt['phpunit-argv']) && is_string($getopt['phpunit-argv'])) in Z:\Website_Developments\home\lara
  vel\data\vendor\brianium\paratest\bin\phpunit-wrapper.php:46
  Stack trace:
  #0 Z:\Website_Developments\home\laravel\data\vendor\brianium\paratest\bin\phpunit-wrapper.php(46): assert(false, 'assert(isset($g...')
  #1 Z:\Website_Developments\home\laravel\data\vendor\brianium\paratest\bin\phpunit-wrapper.php(79): {closure}()
  #2 {main}
    thrown in Z:\Website_Developments\home\laravel\data\vendor\brianium\paratest\bin\phpunit-wrapper.php on line 46

[!NOTE] Downgrading to v7.3.1 works with both commands

vendor\bin\paratest --testdox (v7.3.1)

Z:\Website_Developments\home\laravel\data>vendor\bin\paratest --testdox
ParaTest v7.3.1 upon PHPUnit 10.5.10 by Sebastian Bergmann and contributors.

Processes:     6
Runtime:       PHP 8.2.3
Configuration: Z:\Website_Developments\home\laravel\data\phpunit.xml

..                                                                 2 / 2 (100%)

Time: 00:00.598, Memory: 10.00 MB

Example (Tests\Unit\Example)
 āœ” That true is true

Example (Tests\Feature\Example)
 āœ” The application returns a successful response

OK (2 tests, 2 assertions)

php artisan test --parallel --testdox (v7.3.1)

Z:\Website_Developments\home\laravel\data>php artisan test --parallel --testdox
ParaTest v7.3.1 upon PHPUnit 10.5.10 by Sebastian Bergmann and contributors.

Processes:     6
Runtime:       PHP 8.2.3
Configuration: Z:\Website_Developments\home\laravel\data\phpunit.xml

..                                                                 2 / 2 (100%)

Time: 00:00.766, Memory: 22.00 MB

Example (Tests\Unit\Example)
 āœ” That true is true

Example (Tests\Feature\Example)
 āœ” The application returns a successful response

OK (2 tests, 2 assertions)