search

llaville / php-compatinfo

Library that find out the minimum version and the extensions required for a piece of code to run
https://llaville.github.io/php-compatinfo/7.1/
Other
372 stars 20 forks source link

Tests failure with PHPUnit on version 5.4.0 #280

Closed llaville closed 4 years ago

llaville commented 4 years ago

https://github.com/llaville/php-compat-info/issues/276#issuecomment-702653067

PHPUnit 5.7.27 by Sebastian Bergmann and contributors.

Runtime:       PHP 7.3.22 with Xdebug 2.9.7
Configuration: /shared/backups/bartlett/php-compat-info/phpunit.xml.dist

...................................F............................. 65 / 97 ( 67%)
................F.F.......F.....                                  97 / 97 (100%)

Time: 10.84 seconds, Memory: 42.00MB

There were 4 failures:

1) Bartlett\Tests\CompatInfo\Sniffs\ConstSyntaxSniffTest::testTotalRequirementsWithConstant
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'4.3.10'
+'4.3.0'

/shared/backups/bartlett/php-compat-info/tests/Sniffs/ConstSyntaxSniffTest.php:89

2) Bartlett\Tests\CompatInfo\ConditionIssueTest::testRegressionGH128
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'5.4.0RC3'
+'4.0.0'

/shared/backups/bartlett/php-compat-info/tests/ConditionIssueTest.php:53

3) Bartlett\Tests\CompatInfo\ConditionIssueTest::testRegressionGH160
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'5.6.0alpha1'
+'4.0.0'

/shared/backups/bartlett/php-compat-info/tests/ConditionIssueTest.php:125

4) Bartlett\Tests\CompatInfo\NamespaceIssueTest::testRegressionGH155
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'5.2.0'
+'4.0.0'

/shared/backups/bartlett/php-compat-info/tests/NamespaceIssueTest.php:75

FAILURES!
Tests: 97, Assertions: 230, Failures: 4.

Confirmed. Probably a minor regression since I run last time

remicollet commented 4 years ago

This is not related to PHPUnit 5, same failure with phpunit 8.5 and 9.4

remicollet commented 4 years ago 

Runtime:       PHP 8.0.0rc1
Configuration: /dev/shm/BUILD/php-compat-info-752536f679d15f581aeead8221e90cc026aec9fd/phpunit.xml.dist
Warning:       Your XML configuration validates against a deprecated schema.
Suggestion:    Migrate your XML configuration using "--migrate-configuration"!

...................................F............................. 65 / 97 ( 67%)
................F.F.......F.....                                  97 / 97 (100%)

Time: 00:00.418, Memory: 32.00 MB

There were 4 failures:

1) Bartlett\Tests\CompatInfo\Sniffs\ConstSyntaxSniffTest::testTotalRequirementsWithConstant
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'4.3.10'
+'4.3.0'

/dev/shm/BUILD/php-compat-info-752536f679d15f581aeead8221e90cc026aec9fd/tests/Sniffs/ConstSyntaxSniffTest.php:89

2) Bartlett\Tests\CompatInfo\ConditionIssueTest::testRegressionGH128
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'5.4.0RC3'
+'4.0.0'

/dev/shm/BUILD/php-compat-info-752536f679d15f581aeead8221e90cc026aec9fd/tests/ConditionIssueTest.php:53

3) Bartlett\Tests\CompatInfo\ConditionIssueTest::testRegressionGH160
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'5.6.0alpha1'
+'4.0.0'

/dev/shm/BUILD/php-compat-info-752536f679d15f581aeead8221e90cc026aec9fd/tests/ConditionIssueTest.php:125

4) Bartlett\Tests\CompatInfo\NamespaceIssueTest::testRegressionGH155
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'5.2.0'
+'4.0.0'

/dev/shm/BUILD/php-compat-info-752536f679d15f581aeead8221e90cc026aec9fd/tests/NamespaceIssueTest.php:75

FAILURES!
Tests: 97, Assertions: 230, Failures: 4.
llaville commented 4 years ago

I agree (title updated)

llaville commented 4 years ago

Finally there is only one regression (and I found origin). I'll try to explains. The main difficult goal in this version 5.4 was to refactor conditional code handler. Sniff Architecture is simple when I compare it !

The first one is a real regression because the Expr_ConstFetch node (of AST : nikic/php-parser) is not computed ==> PHP_EOL constant came with PHP 4.3.10 Not in report of command bin/phpcompatinfo analyser:run tests/fixtures/sniffs/constants/gh158.php


Directories                                          1
Files                                                1
Errors                                               0

Extensions Analysis

    Extension REF      EXT min/Max PHP min/Max
    Core      Core                 4.0.0
    mbstring  mbstring 4.3.0       4.3.0
    Total [2]                      4.3.0

Namespaces Analysis

    Namespace REF  EXT min/Max PHP min/Max
              Core             4.0.0
    Total [1]                  4.0.0

No interface found

No trait found

No class found

No generator found

Functions Analysis

    Function      REF      EXT min/Max PHP min/Max
    mb_strtolower mbstring 4.3.0       4.3.0
    Total [1]                          4.3.0

No constant found

No condition found

Requires PHP 4.3.0 (min)

The 3 others depends of what you read in each reports

2) Bartlett\Tests\CompatInfo\ConditionIssueTest::testRegressionGH128
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'5.4.0RC3'
+'4.0.0'

It's equivalent to run command: bin/phpcompatinfo analyser:run tests/fixtures/conditions/gh128.php that print this ouptut

Data Source Analysed

Directories                                          1
Files                                                1
Errors                                               0

Extensions Analysis

    Extension REF  EXT min/Max PHP min/Max
    Core      Core             4.0.0
 C  intl      intl 1.0.2       5.2.4
    Total [2]                  4.0.0

Namespaces Analysis

    Namespace REF  EXT min/Max PHP min/Max
              Core             4.0.0
    Total [1]                  4.0.0

No interface found

No trait found

No class found

No generator found

Functions Analysis

    Function        REF  EXT min/Max PHP min/Max
    defined         Core 4.0.0       4.0.0
    function_exists Core 4.0.0       4.0.0
 C  idn_to_ascii    intl 1.0.2       5.2.4
    Total [3]                        4.0.0

Constants Analysis

    Constant                REF  EXT min/Max PHP min/Max
 C  INTL_IDNA_VARIANT_UTS46 intl 2.0.0b1     5.2.4
    Total [1]                                4.0.0

Conditions Analysis

    Condition                        REF  EXT min/Max PHP min/Max
    defined(INTL_IDNA_VARIANT_UTS46) intl 2.0.0b1     5.2.4
    function_exists(idn_to_ascii)    intl 1.0.2       5.2.4
    Total [2]                                         5.2.4

Requires PHP 4.0.0 (min)

All elements that are marked with a C indicate a conditional code and versions are not computed to global (total) values.

3) Bartlett\Tests\CompatInfo\ConditionIssueTest::testRegressionGH160
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'5.6.0alpha1'
+'4.0.0'

/shared/backups/bartlett/php-compat-info/tests/ConditionIssueTest.php:125

It's equivalent to run command: bin/phpcompatinfo analyser:run tests/fixtures/conditions/gh160/ that print this ouptut

Data Source Analysed

Directories                                          1
Files                                                2
Errors                                               0

Extensions Analysis

    Extension REF       EXT min/Max PHP min/Max
    Core      Core                  4.0.0
 C  tokenizer tokenizer 5.6.0alpha1 5.6.0alpha1
    Total [2]                       4.0.0

Namespaces Analysis

    Namespace REF  EXT min/Max PHP min/Max
              Core             4.0.0
    Total [1]                  4.0.0

No interface found

No trait found

No class found

No generator found

Functions Analysis

    Function  REF  EXT min/Max PHP min/Max
    defined   Core 4.0.0       4.0.0
    Total [1]                  4.0.0

Constants Analysis

    Constant   REF       EXT min/Max PHP min/Max
 C  T_ELLIPSIS tokenizer 5.6.0alpha1 5.6.0alpha1
    Total [1]                        4.0.0

Conditions Analysis

    Condition           REF       EXT min/Max PHP min/Max
    defined(T_ELLIPSIS) tokenizer 5.6.0alpha1 5.6.0alpha1
    Total [1]                                 5.6.0alpha1

Requires PHP 4.0.0 (min)

And the last one 

4) Bartlett\Tests\CompatInfo\NamespaceIssueTest::testRegressionGH155
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'5.2.0'
+'4.0.0'

/shared/backups/bartlett/php-compat-info/tests/NamespaceIssueTest.php:75

It's equivalent to run command: bin/phpcompatinfo analyser:run tests/fixtures/conditions/gh155.php that print this ouptut

Data Source Analysed

Directories                                          1
Files                                                1
Errors                                               0

Extensions Analysis

    Extension REF  EXT min/Max PHP min/Max
    Core      Core             4.0.0
 C  json      json 5.2.0       5.2.0
    Total [2]                  4.0.0

Namespaces Analysis

    Namespace REF  EXT min/Max PHP min/Max
              Core             4.0.0
    Total [1]                  4.0.0

No interface found

No trait found

No class found

No generator found

Functions Analysis

    Function        REF  EXT min/Max PHP min/Max
    do_something    user             4.0.0
    function_exists Core 4.0.0       4.0.0
 C  json_encode     json 5.2.0       5.2.0
    Total [3]                        4.0.0

No constant found

Conditions Analysis

    Condition                    REF  EXT min/Max PHP min/Max
    function_exists(json_encode) json 5.2.0       5.2.0
    Total [1]                                     5.2.0

Requires PHP 4.0.0 (min)

Here json_encode is a user function not the one that came with version PHP 5.2

@remicollet Tell me if my explains are enough ! I 'll then fix the 3 last unit test cases asserts, and fix only the first one

remicollet commented 4 years ago

yes, explanation are enough, as soon as test suite passes in 5.4.1 ;)

llaville commented 4 years ago

I'll fix it tomorrow morning. And give a new chance to find other bugs. Bugfix 5.4.1 release is scheduled to monday 5 or tuesday 6 (next week)

llaville commented 4 years ago

Commit ca54cc227566b8d0841543bb7d1e1d77c992f7d8 solve use test case failure 1)

1) Bartlett\Tests\CompatInfo\Sniffs\ConstSyntaxSniffTest::testTotalRequirementsWithConstant
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'4.3.10'
+'4.3.0'

by implementing leaveExprConstFetch method, but it also introduces a new regression with conditional code 3)

3) Bartlett\Tests\CompatInfo\ConditionIssueTest::testRegressionGH160
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'5.6.0alpha1'
+'4.0.0'

We are confronted in a challlenge ! This use case gh160 is the main reason why we have previously a priority file queue, to proceed first conditional code. But without priority queue, we don't know when code will be marked as conditional and if we have already computed the good versions (php.min / php.max)

Actually we can't do it because we compute versions on the fly.

llaville commented 4 years ago

While working on code to solve this situation, I realized that conditional code may be misunderstood and I'll add tomorrow some pages in the document to explains (how I've already dit it in this bug report). Code patche + documentation will be pushed on reporistory tomorrow !

llaville commented 4 years ago

Still working on code patch, release delayed :(

llaville commented 4 years ago

@remicollet Just finish to test my code refactored to avoid regressions with conditional code. All unit tests PASSED on my side. I'll push the code fix in master branch in few minutes. I'll wait until tomorrow to see if we can find others issues before to release officially the new 5.4.1 That will let me the time to write a doc page about conditional code !

llaville commented 4 years ago

@remicollet All unit tests PASSED. If you're agree, feel free to close this issue.

PHPUnit 8.5.8 by Sebastian Bergmann and contributors.

Runtime:       PHP 7.2.34 with Xdebug 2.9.8
Configuration: /shared/backups/bartlett/php-compat-info/phpunit.xml.dist

...............................................................  63 / 123 ( 51%)
............................................................    123 / 123 (100%)

Time: 36.43 seconds, Memory: 60.00 MB

OK (123 tests, 375 assertions)
devilbox@php-7.2.34 in /shared/backups/bartlett/php-compat-info $ /shared/backups/php/phpunit-8.5.phar
PHPUnit 8.5.8 by Sebastian Bergmann and contributors.

Runtime:       PHP 7.2.34 with Xdebug 2.9.8
Configuration: /shared/backups/bartlett/php-compat-info/phpunit.xml.dist

................................................................. 65 / 97 ( 67%)
................................                                  97 / 97 (100%)

Time: 7.58 seconds, Memory: 40.00 MB

OK (97 tests, 237 assertions)
remicollet commented 4 years ago 
$ phpunit9
PHPUnit 9.4.0 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.0.0rc1
Configuration: /work/GIT/php-compat-info/phpunit.xml.dist
Warning:       Your XML configuration validates against a deprecated schema.
Suggestion:    Migrate your XML configuration using "--migrate-configuration"!

................................................................. 65 / 97 ( 67%)
................................                                  97 / 97 (100%)

Time: 00:01.708, Memory: 32.00 MB

OK (97 tests, 237 assertions)
remicollet commented 4 years ago

@remicollet All unit tests PASSED. If you're agree, feel free to close this issue.

I cannot, this one was open by you ;)