zendframework / zend-test

Test component from Zend Framework
BSD 3-Clause "New" or "Revised" License
18 stars 38 forks source link

test suite fail with error code 255 after update to release 3.2.1 #72

Closed fezfez closed 5 years ago

fezfez commented 5 years ago

Hello,

My test suite suddenly break and return code 255 without summary when i update to release 3.2.1.

With 3.2.1

stephane@stephane-Inspiron-17-7779:~/work/framework$ ./vendor/bin/phpunit
PHPUnit 7.5.1 by Sebastian Bergmann and contributors.

...............................................................  63 / 682 (  9%)
..........................SS................................... 126 / 682 ( 18%)
......................................FFFF...........F......... 189 / 682 ( 27%)
............................................................F.. 252 / 682 ( 36%)
............................................................... 315 / 682 ( 46%)
........................II..................................... 378 / 682 ( 55%)
...................IIII.......................III.............. 441 / 682 ( 64%)
............................................................... 504 / 682 ( 73%)
............................................................... 567 / 682 ( 83%)
............................................................... 630 / 682 ( 92%)
....................................................            682 / 682 (100%)stephane@stephane-Inspiron-17-7779:~/work/framework$ 

With 3.2.0

stephane@stephane-Inspiron-17-7779:~/work/framework$ ./vendor/bin/phpunit
PHPUnit 7.5.1 by Sebastian Bergmann and contributors.

...............................................................  63 / 682 (  9%)
..........................SS................................... 126 / 682 ( 18%)
......................................FFFF...........F......... 189 / 682 ( 27%)
............................................................F.. 252 / 682 ( 36%)
............................................................... 315 / 682 ( 46%)
........................II..................................... 378 / 682 ( 55%)
...................IIII.......................III.............. 441 / 682 ( 64%)
............................................................... 504 / 682 ( 73%)
............................................................... 567 / 682 ( 83%)
............................................................... 630 / 682 ( 92%)
....................................................            682 / 682 (100%)

Time: 15.16 seconds, Memory: 458.25MB

There were 6 failures:

.... error details ....

FAILURES!
Tests: 682, Assertions: 2681, Failures: 6, Skipped: 2, Incomplete: 9.

I was able to find the issue with strace and phpdbg, the php-timer library break on line https://github.com/sebastianbergmann/php-timer/blob/9ef9968ba27999219d76ae3cef97aa0fa87bd90f/src/Timer.php#L64

composer show

adoy/fastcgi-client                            1.0.0    Client for communication with a FastCGI (FCGI) application using the FastCGI protocol.
azuyalabs/yasumi                               1.8.0    Yasumi is an easy PHP Library for calculating national holidays.
beberlei/assert                                v2.9.6   Thin assertion library for input validation in business models.
beberlei/DoctrineExtensions                    v1.1.5   A set of extensions to Doctrine 2 that add support for additional query functions available in MySQL and Oracle.
bruli/event-bus-bundle                         0.5      Bundle to use event bus in symfony projects
bruli/ignore-files                             1.0.1    Ignore files
bruli/php-git-hooks                            v5.8     Git hooks for PHP projects.
bruli/php-value-objects                        v0.6     PHP Value objects to use for DDD domains.
composer/ca-bundle                             1.1.3    Lets you find a path to the system CA bundle, and includes a fallback to the Mozilla CA bundle.
composer/composer                              1.8.0    Composer helps you declare, manage and install dependencies of PHP projects, ensuring you have the right stack everywhere.
composer/semver                                1.4.2    Semver library that offers utilities, version constraint parsing and validation.
composer/spdx-licenses                         1.5.0    SPDX licenses list and validation library.
composer/xdebug-handler                        1.3.1    Restarts a process without xdebug.
container-interop/container-interop            1.2.0    Promoting the interoperability of container objects (DIC, SL, etc.)
dealerdirect/phpcodesniffer-composer-installer v0.4.4   PHP_CodeSniffer Standards Composer Installer Plugin
doctrine/annotations                           v1.6.0   Docblock Annotations Parser
doctrine/cache                                 v1.8.0   Caching library offering an object-oriented API for many cache backends
doctrine/coding-standard                       5.0.0    The Doctrine Coding Standard is a set of PHPCS rules applied to all Doctrine projects.
doctrine/collections                           v1.5.0   Collections Abstraction library
doctrine/common                                v2.10.0  PHP Doctrine Common project is a library that provides additional functionality that other Doctrine projects depend on such as better reflection suppo...
doctrine/data-fixtures                         v1.3.1   Data Fixtures for all Doctrine Object Managers
doctrine/dbal                                  v2.9.0   Powerful PHP database abstraction layer (DBAL) with many features for database schema introspection and management.
doctrine/doctrine-module                       2.1.7    Zend Framework Module that provides Doctrine basic functionality required for ORM and ODM modules
doctrine/doctrine-orm-module                   2.1.2    Zend Framework Module that provides Doctrine ORM functionality
doctrine/event-manager                         v1.0.0   Doctrine Event Manager component
doctrine/inflector                             v1.3.0   Common String Manipulations with regard to casing and singular/plural rules.
doctrine/instantiator                          1.1.0    A small, lightweight utility to instantiate objects in PHP without invoking their constructors
doctrine/lexer                                 v1.0.1   Base library for a lexer that can be used in Top-Down, Recursive Descent Parsers.
doctrine/orm                                   v2.6.3   Object-Relational-Mapper for PHP
doctrine/persistence                           v1.1.0   The Doctrine Persistence project is a set of shared interfaces and functionality that the different Doctrine object mappers share.
doctrine/reflection                            v1.0.0   Doctrine Reflection component
egulias/email-validator                        2.1.7    A library for validating emails against several RFCs
firebase/php-jwt                               v5.0.0   A simple library to encode and decode JSON Web Tokens (JWT) in PHP. Should conform to the current spec.
fiunchinho/phpunit-randomizer                  v4.0.0   Execute your test cases in random order, so you can check if they have hidden dependencies
friendsofphp/php-cs-fixer                      v2.13.1  A tool to automatically fix PHP code style
google/apiclient                               v2.2.2   Client library for Google APIs
google/apiclient-services                      v0.77    Client library for Google APIs
google/auth                                    v1.4.0   Google Auth Library for PHP
gordalina/cachetool                            4.0.0    Manage your opcode & apc through the cli
guzzlehttp/guzzle                              6.3.3    Guzzle is a PHP HTTP client library
guzzlehttp/promises                            v1.3.1   Guzzle promises library
guzzlehttp/psr7                                1.5.2    PSR-7 message implementation that also provides common utility methods
hellogerard/jobby                              v3.4.5   Manage all your cron jobs without modifying crontab. Handles locking, logging, error emails, and more.
jean85/pretty-package-versions                 1.2      A wrapper for ocramius/package-versions to get pretty versions strings
jeremeamia/SuperClosure                        2.4.0    Serialize Closure objects, including their context and binding
justinrainbow/json-schema                      5.2.7    A library to validate a json schema.
markbaker/complex                              1.4.7    PHP Class for working with complex numbers
monolog/monolog                                1.24.0   Sends your logs to files, sockets, inboxes, databases and various web services
mtdowling/cron-expression                      v1.2.1   CRON for PHP: Calculate the next or previous run date and determine if a CRON expression is due
mustache/mustache                              v2.12.0  A Mustache implementation in PHP.
myclabs/deep-copy                              1.8.1    Create deep copies (clones) of your objects
nette/bootstrap                                v2.4.6   ๐Ÿ…ฑ Nette Bootstrap: the simple way to configure and bootstrap your Nette application.
nette/di                                       v2.4.14  ๐Ÿ’Ž Nette Dependency Injection Container: Flexible, compiled and full-featured DIC with perfectly usable autowiring and support for all new PHP 7.1 f...
nette/finder                                   v2.4.2   ๐Ÿ” Nette Finder: find files and directories with an intuitive API.
nette/neon                                     v2.4.3   ๐Ÿธ Nette NEON: encodes and decodes NEON file format.
nette/php-generator                            v3.0.5   ๐Ÿ˜ Nette PHP Generator: generates neat PHP code for you. Supports new PHP 7.2 features.
nette/robot-loader                             v3.1.0   ๐Ÿ€ Nette RobotLoader: high performance and comfortable autoloader that will search and autoload classes within your application.
nette/utils                                    v2.5.3   ๐Ÿ›  Nette Utils: lightweight utilities for string & array manipulation, image handling, safe JSON encoding/decoding, validation, slug or strong passw...
nikic/php-parser                               v4.1.0   A PHP parser written in PHP
ocramius/package-versions                      1.3.0    Composer plugin that provides efficient querying for installed package versions (no runtime IO)
ocramius/proxy-manager                         2.2.2    A library providing utilities to generate, instantiate and generally operate with Object Proxies
paragonie/random_compat                        v9.99.99 PHP 5.x polyfill for random_bytes() and random_int() from PHP 7
pdepend/pdepend                                2.5.2    Official version of pdepend to be handled with Composer
phar-io/manifest                               1.0.3    Component for reading phar.io manifest information from a PHP Archive (PHAR)
phar-io/version                                2.0.1    Library for handling version information and constraints
php-cs-fixer/diff                              v1.3.0   sebastian/diff v2 backport support for PHP5.6
phpdocumentor/reflection-common                1.0.1    Common reflection classes used by phpdocumentor to reflect the code structure
phpdocumentor/reflection-docblock              4.3.0    With this component, a library can provide support for annotations via DocBlocks or otherwise retrieve information that is embedded in a DocBlock.
phpdocumentor/type-resolver                    0.4.0   
phpmd/phpmd                                    2.6.0    PHPMD is a spin-off project of PHP Depend and aims to be a PHP equivalent of the well known Java tool PMD.
phpoffice/phpspreadsheet                       1.5.2    PHPSpreadsheet - Read, Create and Write Spreadsheet documents in PHP - Spreadsheet engine
phpseclib/phpseclib                            2.0.12   PHP Secure Communications Library - Pure-PHP implementations of RSA, AES, SSH2, SFTP, X.509 etc.
phpspec/prophecy                               1.8.0    Highly opinionated mocking framework for PHP 5.3+
phpstan/phpdoc-parser                          0.3      PHPDoc parser with support for nullable, intersection and generic types
phpstan/phpstan                                0.10.6   PHPStan - PHP Static Analysis Tool
phpstan/phpstan-doctrine                       0.10     Doctrine extensions for PHPStan
phpstan/phpstan-phpunit                        0.10     PHPUnit extensions and rules for PHPStan
phpunit/php-code-coverage                      6.1.4    Library that provides collection, processing, and rendering functionality for PHP code coverage information.
phpunit/php-file-iterator                      2.0.2    FilterIterator implementation that filters files based on a list of suffixes.
phpunit/php-text-template                      1.2.1    Simple template engine.
phpunit/php-timer                              2.0.0    Utility class for timing
phpunit/php-token-stream                       3.0.1    Wrapper around PHP's tokenizer extension.
phpunit/phpunit                                7.5.1    The PHP Unit Testing framework.
psr/cache                                      1.0.1    Common interface for caching libraries
psr/container                                  1.0.0    Common Container Interface (PHP FIG PSR-11)
psr/http-message                               1.0.1    Common interface for HTTP messages
psr/log                                        1.1.0    Common interface for logging libraries
psr/simple-cache                               1.0.1    Common interfaces for simple caching
ralouphie/getallheaders                        2.0.5    A polyfill for getallheaders.
ramsey/uuid                                    3.8.0    Formerly rhumsaa/uuid. A PHP 5.4+ library for generating RFC 4122 version 1, 3, 4, and 5 universally unique identifiers (UUID).
ramsey/uuid-doctrine                           1.5.0    Allow the use of a ramsey/uuid UUID as Doctrine field type.
roave/better-reflection                        3.2.0    Better Reflection - an improved code reflection API
roave/signature                                1.0.0    Sign and verify stuff
sebastian/code-unit-reverse-lookup             1.0.1    Looks up which function or method a line of code belongs to
sebastian/comparator                           3.0.2    Provides the functionality to compare PHP values for equality
sebastian/diff                                 3.0.1    Diff implementation
sebastian/environment                          4.0.1    Provides functionality to handle HHVM/PHP environments
sebastian/exporter                             3.1.0    Provides the functionality to export PHP variables for visualization
sebastian/global-state                         2.0.0    Snapshotting of global state
sebastian/object-enumerator                    3.0.3    Traverses array structures and object graphs to enumerate all referenced objects
sebastian/object-reflector                     1.1.1    Allows reflection of object attributes, including inherited and non-public ones
sebastian/recursion-context                    3.0.0    Provides functionality to recursively process PHP variables
sebastian/resource-operations                  2.0.1    Provides a list of PHP built-in functions that operate on resources
sebastian/version                              2.0.1    Library that helps with managing the version number of Git-hosted PHP projects
seld/jsonlint                                  1.7.1    JSON Linter
seld/phar-utils                                1.0.1    PHAR file format utilities, for when PHP phars you up
slevomat/coding-standard                       4.8.6    Slevomat Coding Standard for PHP_CodeSniffer complements Consistence Coding Standard by providing sniffs with additional checks.
squizlabs/php_codesniffer                      3.3.2    PHP_CodeSniffer tokenizes PHP, JavaScript and CSS files and detects violations of a defined set of coding standards.
swiftmailer/swiftmailer                        v6.1.3   Swiftmailer, free feature-rich PHP mailer
symfony/config                                 v4.2.1   Symfony Config Component
symfony/console                                v4.2.1   Symfony Console Component
symfony/contracts                              v1.0.2   A set of abstractions extracted out of the Symfony components
symfony/debug                                  v4.2.1   Symfony Debug Component
symfony/dependency-injection                   v4.2.1   Symfony DependencyInjection Component
symfony/dotenv                                 v4.2.1   Registers environment variables from a .env file
symfony/event-dispatcher                       v4.2.1   Symfony EventDispatcher Component
symfony/filesystem                             v4.2.1   Symfony Filesystem Component
symfony/finder                                 v4.2.1   Symfony Finder Component
symfony/flex                                   v1.1.8   Composer plugin for Symfony
symfony/http-foundation                        v4.2.1   Symfony HttpFoundation Component
symfony/http-kernel                            v4.2.1   Symfony HttpKernel Component
symfony/intl                                   v4.2.1   A PHP replacement layer for the C intl extension that includes additional data from the ICU library.
symfony/options-resolver                       v4.2.1   Symfony OptionsResolver Component
symfony/polyfill-ctype                         v1.10.0  Symfony polyfill for ctype functions
symfony/polyfill-intl-icu                      v1.10.0  Symfony polyfill for intl's ICU-related data and classes
symfony/polyfill-mbstring                      v1.10.0  Symfony polyfill for the Mbstring extension
symfony/polyfill-php56                         v1.10.0  Symfony polyfill backporting some PHP 5.6+ features to lower PHP versions
symfony/polyfill-php70                         v1.10.0  Symfony polyfill backporting some PHP 7.0+ features to lower PHP versions
symfony/polyfill-php72                         v1.10.0  Symfony polyfill backporting some PHP 7.2+ features to lower PHP versions
symfony/polyfill-util                          v1.10.0  Symfony utilities for portability of PHP codes
symfony/process                                v4.2.1   Symfony Process Component
symfony/stopwatch                              v4.2.1   Symfony Stopwatch Component
symfony/yaml                                   v4.2.1   Symfony Yaml Component
tecnickcom/tcpdf                               6.2.26   TCPDF is a PHP class for generating PDF documents and barcodes.
thecodingmachine/phpstan-safe-rule             v0.1.0   A PHPStan rule to detect safety issues. Must be used in conjunction with thecodingmachine/safe
thecodingmachine/safe                          v0.1.8   PHP core functions that throw exceptions instead of returning FALSE on error
theseer/tokenizer                              1.1.0    A small library for converting tokenized PHP source code into XML and potentially other formats
true/punycode                                  v2.1.1   A Bootstring encoding of Unicode for Internationalized Domain Names in Applications (IDNA)
webmozart/assert                               1.3.0    Assertions to validate method input/output with nice error messages.
zendframework/zend-authentication              2.6.0    provides an API for authentication and includes concrete authentication adapters for common use case scenarios
zendframework/zend-cache                       2.8.2    Caching implementation with a variety of storage options, as well as codified caching strategies for callbacks, classes, and output
zendframework/zend-code                        3.3.1    provides facilities to generate arbitrary code using an object oriented interface
zendframework/zend-config                      3.2.0    provides a nested object property based user interface for accessing this configuration data within application code
zendframework/zend-console                     2.7.0    Build console applications using getopt syntax or routing, complete with prompts
zendframework/zend-debug                       2.6.0    Safely dump debug information to HTML
zendframework/zend-developer-tools             1.2.1    Module for developer and debug tools for use with zend-mvc applications.
zendframework/zend-dom                         2.7.1    provides tools for working with DOM documents and structures
zendframework/zend-escaper                     2.6.0    Securely and safely escape HTML, HTML attributes, JavaScript, CSS, and URLs
zendframework/zend-eventmanager                3.2.1    Trigger and listen to events within a PHP application
zendframework/zend-filter                      2.9.0    provides a set of commonly needed data filters
zendframework/zend-form                        2.13.0   Validate and display simple and complex forms, casting forms to business objects and vice versa
zendframework/zend-http                        2.8.2    Provides an easy interface for performing Hyper-Text Transfer Protocol (HTTP) requests
zendframework/zend-hydrator                    2.4.1    Serialize objects to arrays, and vice versa
zendframework/zend-i18n                        2.9.0    Provide translations for your application, and filter and validate internationalized values
zendframework/zend-i18n-resources              2.6.0    Provides validator translations for zend-i18n's Translator
zendframework/zend-inputfilter                 2.8.2    Normalize and validate input sets from the web, APIs, the CLI, and more, including files
zendframework/zend-json                        3.1.0    provides convenience methods for serializing native PHP to JSON and decoding JSON to native PHP
zendframework/zend-loader                      2.6.0    Autoloading and plugin loading strategies
zendframework/zend-log                         2.10.0   component for general purpose logging
zendframework/zend-mail                        2.10.0   Provides generalized functionality to compose and send both text and MIME-compliant multipart e-mail messages
zendframework/zend-mime                        2.7.1    Create and parse MIME messages and parts
zendframework/zend-modulemanager               2.8.2    Modular application system for zend-mvc applications
zendframework/zend-mvc                         3.1.1    Zend Framework's event-driven MVC layer, including MVC Applications, Controllers, and Plugins
zendframework/zend-mvc-i18n                    1.1.0    Integration between zend-mvc and zend-i18n
zendframework/zend-mvc-plugin-flashmessenger   1.1.0    Plugin for creating and exposing flash messages via zend-mvc controllers
zendframework/zend-mvc-plugin-identity         1.1.0    Plugin for retrieving the current authenticated identity within zend-mvc controllers
zendframework/zend-navigation                  2.9.0    Manage trees of pointers to web pages in order to build navigation systems
zendframework/zend-paginator                   2.8.1    zend-paginator is a flexible component for paginating collections of data and presenting that data to users.
zendframework/zend-permissions-acl             2.7.0    Provides a lightweight and flexible access control list (ACL) implementation for privileges management
zendframework/zend-router                      3.2.0    Flexible routing system for HTTP and console applications
zendframework/zend-servicemanager              3.3.2    Factory-Driven Dependency Injection Container
zendframework/zend-session                     2.8.5    manage and preserve session data, a logical complement of cookie data, across multiple page requests by the same client
zendframework/zend-stdlib                      3.2.1    SPL extensions, array utilities, error handlers, and more
zendframework/zend-test                        3.2.1    Tools to facilitate unit testing of zend-mvc applications
zendframework/zend-uri                         2.6.1    A component that aids in manipulating and validating ยป Uniform Resource Identifiers (URIs)
zendframework/zend-validator                   2.10.2   provides a set of commonly needed validators
zendframework/zend-view                        2.11.1   provides a system of helpers, output filters, and variable escaping

Do you know how to resolve this issue ?

Thanks in advance for your awesome work !

Ocramius commented 5 years ago

Possibly related to https://github.com/zendframework/zend-test/issues/71

ezimuel commented 5 years ago

@fezfez this is something related with this PHPUnit issue https://github.com/sebastianbergmann/phpunit/issues/3026. Can you try to apply the workaround reported here? Let me know, thanks!

fezfez commented 5 years ago

@ezimuel I indeed found the bug, thanks !

<?php

declare(strict_types=1);

namespace MyNamespace\Controller\ErrorController;

use MyNamespace\Test\ServiceManagerFactory;
use Zend\Test\PHPUnit\Controller\AbstractHttpControllerTestCase;

class MyIndexActionTest extends AbstractHttpControllerTestCase
{
    public function setUp() : void
    {
        $this->setApplicationConfig(ServiceManagerFactory::getConfiguration());
    }

    public function testCanBeAccess() : void
    {
        $this->dispatch('/aroute', 'GET');
        $this->assertResponseStatusCode(200);
    }
}

Result :

./vendor/bin/phpunit --filter MyIndexActionTest
PHPUnit 7.5.1 by Sebastian Bergmann and contributors.

.                                                                   1 / 1 (100%)
<?php

declare(strict_types=1);

namespace MyNamespace\Controller\ErrorController;

use MyNamespace\Test\ServiceManagerFactory;
use Zend\Test\PHPUnit\Controller\AbstractHttpControllerTestCase;

class MyIndexActionTest extends AbstractHttpControllerTestCase
{
    public function setUp() : void
    {
        $this->setApplicationConfig(ServiceManagerFactory::getConfiguration());
        parent::setUp();
    }

    public function testCanBeAccess() : void
    {
        $this->dispatch('/aroute', 'GET');
        $this->assertResponseStatusCode(200);
    }
}

Result :

./vendor/bin/phpunit --filter MyIndexActionTest
PHPUnit 7.5.1 by Sebastian Bergmann and contributors.

.                                                                   1 / 1 (100%)

Time: 277 ms, Memory: 36.00MB

OK (1 test, 1 assertions)

I wasn't calling parent::setUp !

Maybe we can fix this by providing a mechanism to check if setUp was call ?

Thanks for your help :+1:

ezimuel commented 5 years ago

@fezfez I don't see a way for checking this automatically. Anyway, I'm glad that you solved and thanks for reporting it.