Clover-coverage report causes PHPUnit to fail

[REBELinBLUE]

I have the following

    <logging>
        <log type="coverage-html" target="storage/coverage" charset="UTF-8" yui="true" highlight="true" lowUpperBound="35" highLowerBound="70" />
        <log type="coverage-clover" target="storage/coverage/index.xml" />
        <log type="tap" target="storage/logs/tap.log" logIncompleteSkipped="false" />
    </logging>

in my phpunit.xml

The coverage-clover log causes PHPCI to regard the build as failed

RUNNING PLUGIN: php_unit
TAP version 13
ok 1 - ExampleTest::testBasicExample
1..1
Generating code coverage report in Clover XML format ... done
Generating code coverage report in HTML format ... done
Exception: Invalid TAP string, number of tests does not match specified test count.
PLUGIN: FAILED

Removing it fixes the problem, but then you are unable to use tools like Code Climate for coverage reports

[REBELinBLUE]

For the time being I was able to work around the problem by running the unit tests for a second time

complete:
   shell:
       - "php /var/www/PHPCI/vendor/bin/phpunit --coverage-clover %BUILD_PATH%/storage/coverage/clover.xml -c %BUILD_PATH%/phpunit.xml %BUILD_PATH%/tests"
       - "php /var/www/PHPCI/vendor/bin/test-reporter --coverage-report=%BUILD_PATH%/storage/coverage/clover.xml"

[mavimo]

@REBELinBLUE yes, maybe the tripe dots in the format ... done generate some issue, I think we need to add also this test case in the TAP parse class.

[mavimo]

@REBELinBLUE I add a new test on TAP parser and seems there are no problem, can attach the full config are you using?

My test:

// ...
class TapParserTest extends \PHPUnit_Framework_TestCase
{
    // ...

    public function testMultipleReports()
    {
        $content = <<<TAP
    TAP version 13
ok 1 - ExampleTest::testBasicExample
1..1
Generating code coverage report in Clover XML format ... done
Generating code coverage report in HTML format ... done
TAP;
        $parser = new TapParser($content);
        $parser->parse();

        $this->assertEquals(0, $parser->getTotalFailures());
    }
    // ...
}

[REBELinBLUE]

phpunit.xml

<?xml version="1.0" encoding="UTF-8"?>
<phpunit backupGlobals="false"
         backupStaticAttributes="false"
         bootstrap="bootstrap/autoload.php"
         colors="true"
         convertErrorsToExceptions="true"
         convertNoticesToExceptions="true"
         convertWarningsToExceptions="true"
         processIsolation="false"
         stopOnFailure="false"
         syntaxCheck="false">
    <testsuites>
        <testsuite name="Application Test Suite">
            <directory>./tests/</directory>
        </testsuite>
    </testsuites>
    <logging>
        <log type="coverage-html" target="storage/coverage" charset="UTF-8" yui="true" highlight="true" lowUpperBound="35" highLowerBound="70" />
        <log type="coverage-clover" target="storage/coverage/index.xml" />
        <log type="tap" target="storage/logs/tap.log" logIncompleteSkipped="false" />
    </logging>
    <php>
        <env name="APP_ENV" value="testing" />
        <env name="APP_DEBUG" value="true" />
        <env name="APP_KEY" value="cHKwVTJCF75DKo9bta72ZNrkS6Q0tqYy" />
        <env name="CACHE_DRIVER" value="array" />
        <env name="SESSION_DRIVER" value="array" />
        <env name="QUEUE_DRIVER" value="sync" />
    </php>
</phpunit>

phpci.yml

build_settings:
    clone_depth: 1
    verbose: false
    ignore: 
        - "bootstrap"
        - "database"
        - "node_modules"
        - "public"
        - "resources"
        - "storage"
        - "tests"
        - "vendor"

setup:
    wipe:
        directory: "/var/www/PHPCI/public/coverage/deployer/"
    composer:
        action: "install"
        prefer_dist: true
    env:
        APP_ENV: "testing"
        APP_DEBUG: "true"
        APP_KEY: "cHKwVTJCF75DKo9bta72ZNrkS6Q0tqYy"
        CACHE_DRIVER: "array"
        SESSION_DRIVER: "array"
        QUEUE_DRIVER: "sync"

test:
    lint:
        directory: "app/"
    php_unit:
        config:
            - "phpunit.xml"
        directory:
            - "tests/"
        coverage: "/var/www/PHPCI/public/coverage/deployer/"
    php_mess_detector:
        allowed_warnings: 0
        path: "app/"
        rules:
            - "design"
            - "naming"
            - "unusedcode"
    php_code_sniffer:
        path: "app/"
        standard: "PSR2"
        allowed_warnings: 0
        allowed_errors: 0
    php_cpd:
    php_loc:
    php_docblock_checker:
    technical_debt:
        allowed_errors: -1

failure:
    email:
        committer: true

complete:

success:

[cerw]

I am bit confused how to use coverage as well? Why do i need to store it outside of phpci? Does PHPCI support showing the report or not? Thanks

[Adirelle]

It doesn't. However, you can use on of the plugin to copy the report in a public area.

[mavimo]

@cerw PHPCI support report inside but only for some plugins, code coverage is not supported, so you can store the information outside or you can create a custom code to display the results in PHPCI (PR are welcome :D )

[Henk8]

I also would like a code coverage plugin. The most important is that a build fails if the percentage of code coverage is lower than x%.

Is this possible, or is this something for an upcoming release?