paratestphp / paratest

:computer: Parallel testing for PHPUnit
MIT License
2.3k stars 226 forks source link

Issue with teamcity reporting in PhpStorm #850

Closed MatteoBiagini closed 4 months ago

MatteoBiagini commented 4 months ago
Q A
ParaTest version 7.4.3
PHPUnit version 10.5.19
PHP version 8.2.18

Summary

When running paratest from PhpStorm, the reporting shows a wrong count of tests, and the run appears as "stopped". Plus, the run output show pieces of the teamcity log.

Current behavior

When launching paratest with:

Some lines with ##teamcity are appearing in the output of the PhpStorm run, and the total count of executed tests is wrong:

image

I added the option --log-teamcity to inspect the teamcity log, and this is what I see in the sections corresponding to the output:

image

This issues are appearing randomly in terms of number of issues and line of logs involved. The issue is not happening when I run with only 1 process.

How to reproduce: command, code and error stack trace

The PhpStorm run:

[docker-compose://[/home/coder/projects/docker-compose.yml]:web-service/]:php" -dmemory_limit=-1 /var/www/project/vendor/bin/paratest_for_phpstorm /var/www/project/vendor/phpunit/phpunit/phpunit --testsuite=unit --processes=8 --configuration /var/www/project/phpunit.xml.dist --teamcity

Expected behavior

I expect PhpStorm to show the correct number of tests as executed, and the operation as successful. I expect no lines with ##teamcity in the output.

Analysis

My understanding is that the logic that is consolidating the teamcity logs from each process is sometimes reading incomplete lines, as the phpunit process is concurrently writing them. Before paratest is reading the last part of the line, lines from other processes are meanwhile read and added to the consolidated log. Broken lines can't be interpreted by PhpStorm, so they are left in the output and the corresponding tests are not marked as finished. As a result, for PhpStorm the run appears as stopped.

As a POC, I could fix the error working on the logic of ResultPrinter.php:

https://github.com/paratestphp/paratest/blob/b2c11a045db098881bbde638a040029b5c2700ad/src/WrapperRunner/ResultPrinter.php#L312C5-L334C6

making sure that tail() only reads complete log lines. I could submit a PR, if appreciated.

Slamdunk commented 4 months ago

I could submit a PR, if appreciated.

Yes please: I am unable to work on the PHPStorm plugin, so any help is very welcome :pray: