REST module tests report 0% coverage

[vitorhorta]

Hi,

I'm having the same problem that was reported on Codeception/Codeception#1622 but that solution didn't work for me. I'm using Yii2 basic template and i'm trying to run Api tests that uses the REST module. The tests passes and the html files are created but I get 0 reports from coverage.

As mentioned on Codeception/Codeception#1622 I tried to downgrade the PHPUnit and Codeception version but it didn't work. Is there any other way to fix this issue? If not, what version should I use for both packages?

Thanks in advance

[thecatontheflat]

+1

[greentopsecret]

+1

[zelf]

+1

[danielintloop]

+1

[Naktibalda]

So many people are interested. Could one of you reproduce and fix the issue?

[rogoit]

+1

[Naktibalda]

Please provide instructions how to reproduce this issue and I will reopen this ticket.

[rogoit]

Just try to make an api test and check the code coverage

[Naktibalda]

Which version of codeception are you using?

[rogoit]

2.2.0

[Naktibalda]

@samdark This is a Yii issue, could you test it?

[Naktibalda]

@rogoit have you enabled code coverage and whitelist (not white_list) in codeception.yml?

[rogoit]

Yes i also try remote true and false on a vagrant. Try to run it from mac and vagrant. Xdebug settings are running fine on development.

[valoricDe]

I had the REST module enabled and got 0/46 methods covered. A removed REST module and executed codeception build afterwards it gave me code coverage.

In one scenario $this->getTestResultObject()->getCodeCoverage() in codeception/codeception/src/Codeception/Test/Feature/CodeCoverage.php:16 returns null and in the other a SebastianBergmann\CodeCoverage\CodeCoverage class.

[Naktibalda]

@valorize What module does REST module use for connecting to API in your test suite?

[valoricDe]

        - PhpBrowser:
            url: http://mview.example.local/
        - REST:
            depends: PhpBrowser
            url: http://mview.example.local

[valoricDe]

In one scenario \Codeception\Coverage\SuiteSubscriber::applyFilter:84 is called in the other not.

[Naktibalda]

I will try to reproduce it.

[valoricDe]

I think I found the flaw. Without REST+PhpBrowser is the \Codeception\Coverage\Subscriber\Local module active (see \Codeception\Coverage\Subscriber\Local::beforeSuite:34) if so the applyFilter method directly after will $result->setCodeCoverage($this->coverage);. If REST is activated Local is not an active module, but \Codeception\Coverage\Subscriber\LocalServer or RemoteServer are. But these do not execute $result->setCodeCoverage($this->coverage);.

So I guress the fix would be: add $this->applyFilter($e->getResult()); to LocalServer::beforeSuite

[Naktibalda]

Please make that change and check if it fixes your issue.

[valoricDe]

The change activates successfully the code coverage, but the report is still empty. I also had to adapt the LocalServer::afterSuite to the behaviour of Local::afterSuite. Soo I had to add: $this->mergeToPrint($e->getResult()->getCodeCoverage()); in LocalServer. It know reports just fine.

Sidequestion: When is codeception.yml used and when api.suite.yml? Is this overwritten when I supply the run command with the -c param?

Edit: setAddUncoveredFilesFromWhitelist is never been called making the coverage: show_uncovered: false option / setting kind of useless. Therefore I will add self::$coverage->setAddUncoveredFilesFromWhitelist($this->settings['show_uncovered']); in the first line of \Codeception\Coverage\Subscriber\Printer::printConsole, but I guess that's not the best place for that.

[valoricDe]

So basically we shouldn't consider only remote coverage when activating the REST module. So LocalServer and RemoteServer should integrate all functionality of \Codeception\Coverage\Subscriber\Local.

[davidjeddy]

Code coverage would be really awesome for API only applications.

[arustamov]

@Naktibalda @valorize any updates on this? btw it looks like Yii label can be removed since the issue is not framework specific

[Naktibalda]

I don't use PhpBrowser in my tests, but I use REST a lot, so I tried to generate code coverage report for one of my applications and I got empty report.

In order to get non-empty report I had to whitelist application code with this configuration:

coverage:
    enabled: true
    include:
      - 'application/*'

[valoricDe]

@arustamov I do not maintain this project. I stepped through all relevant code of codeception and updated the code as described in the upper comments. For me the code worked quite well then.

[davidjeddy]

@valorize Would you mind providing a PR for your changes? Would be really awesome to have code coverage working for API/REST apps.

[valoricDe]

@davidjeddy I'm sorry but I won't be able to. Timewise and because I left the company where I used codeception and where I adapted the code.

[davidjeddy]

@valorize NNNNnnooooooo :(. Man, was really hoping for a fix :(

[valoricDe]

@davidjeddy the fix is described in https://github.com/Codeception/Codeception/issues/2220#issuecomment-305147001 and https://github.com/Codeception/Codeception/issues/2220#issuecomment-305151819 It is not to difficult to go into these classes with xdebug and check which parts need to be adapted once one knows where to look.

[igorabrandao]

Hello @valorize, I've applied the changes in LocalServer as you told, however my coverage test just grab the data related to the /tests folder and none from the Controllers.

Is really necessary to downgrade the PhpUnit and Codeception versions? Because when I do it, all my tests crash.

Do you have any idea what can be happening?

Note: I'm testing a REST in another server.

Thanks in advance!

[igorabrandao]

@davidjeddy here you are the changes @valorize mentioned above.

BEFORESUITE `public function beforeSuite(SuiteEvent $e) { $this->module = $this->getServerConnectionModule($e->getSuite()->getModules()); $this->applySettings($e->getSettings()); $this->applyFilter($e->getResult()); if (!$this->isEnabled()) { return; }

    $this->suiteName = $e->getSuite()->getBaseName();

    if ($this->settings['remote_config']) {
        $this->addC3AccessHeader(self::COVERAGE_HEADER_CONFIG, $this->settings['remote_config']);
    }

    $knock = $this->c3Request('clear');
    if ($knock === false) {
        throw new RemoteException(
            '
            CodeCoverage Error.
            Check the file "c3.php" is included in your application.
            We tried to access "/c3/report/clear" but this URI was not accessible.
            You can review actual error messages in c3tmp dir.
            '
        );
    }
}`

AFTERSUITE `public function afterSuite(SuiteEvent $e) { if (!$this->isEnabled()) { return; } $this->mergeToPrint($e->getResult()->getCodeCoverage()); $coverageFile = Configuration::outputDir() . 'c3tmp/codecoverage.serialized';

    $retries = 5;
    while (!file_exists($coverageFile) && --$retries >= 0) {
        usleep(0.5 * 1000000); // 0.5 sec
    }

    if (!file_exists($coverageFile)) {
        if (file_exists(Configuration::outputDir() . 'c3tmp/error.txt')) {
            throw new \RuntimeException(file_get_contents(Configuration::outputDir() . 'c3tmp/error.txt'));
        }
        return;
    }

    $contents = file_get_contents($coverageFile);
    $coverage = @unserialize($contents);
    if ($coverage === false) {
        return;
    }
    $this->mergeToPrint($coverage);
}`

[igorabrandao]

Somehow in RemoteServer::retrieveAndPrintHtml, I'm getting the following error that prevents from getting the coverage from Controllers:

[UnexpectedValueException]
phar error: "/tmp/C3syXhG1.tar" is a corrupted tar file (checksum mismatch of file <!DOCTYPE HTML PUBLIC -//W3C//DTD HTML 3.2 Final//EN>