Codeception Tests that contain @env do not execute due to an Allure import error message

[ivy00johns]

I'm submitting a ...

• [X] bug report
• [ ] feature request
• [ ] support request

What is the current behavior?

When you have a Codeception test that includes a @env Test Annotation, like the following, and you run it you receive an Allure related error message:

Example Test

use Magento\AcceptanceTestFramework\AcceptanceTester;
use Yandex\Allure\Adapter\Annotation\Features;
use Yandex\Allure\Adapter\Annotation\Stories;
use Yandex\Allure\Adapter\Annotation\Title;
use Yandex\Allure\Adapter\Annotation\Description;
use Yandex\Allure\Adapter\Annotation\Parameter;
use Yandex\Allure\Adapter\Annotation\Severity;
use Yandex\Allure\Adapter\Model\SeverityLevel;
use Yandex\Allure\Adapter\Annotation\TestCaseId;

/**
 * @env chrome
 * @env firefox
 * @env phantomjs
 */
class LogIntoAdminCest
{
    public function logIntoAdmin(AcceptanceTester $I)
    {
        $I->amOnPage("admin/admin");
        $I->fillField("#username", "user");
        $I->fillField("#login", "blahblah");
        $I->click(".actions .action-primary");
        $I->seeInCurrentUrl("admin/admin");
    }
}

Stacktrace

  [Semantical Error] The annotation "@env" in method 
  LogIntoAdminCest::logIntoAdmin() was never imported. Did you maybe 
  forget to add a "use" statement for this annotation?    

  Exception trace:
  () at /vendor/doctrine/annotations/lib/Doctrine/Common/Annotations/AnnotationException.php:54
  Doctrine\Common\Annotations\AnnotationException::semanticalError() at /vendor/doctrine/annotations/lib/Doctrine/Common/Annotations/DocParser.php:727
  Doctrine\Common\Annotations\DocParser->Annotation() at /vendor/doctrine/annotations/lib/Doctrine/Common/Annotations/DocParser.php:663
  Doctrine\Common\Annotations\DocParser->Annotations() at /vendor/doctrine/annotations/lib/Doctrine/Common/Annotations/DocParser.php:354
  Doctrine\Common\Annotations\DocParser->parse() at /vendor/doctrine/annotations/lib/Doctrine/Common/Annotations/AnnotationReader.php:284
  Doctrine\Common\Annotations\AnnotationReader->getMethodAnnotations() at /vendor/doctrine/annotations/lib/Doctrine/Common/Annotations/IndexedReader.php:71
  Doctrine\Common\Annotations\IndexedReader->getMethodAnnotations() at /vendor/allure-framework/allure-php-api/src/Yandex/Allure/Adapter/Annotation/AnnotationProvider.php:43
  Yandex\Allure\Adapter\Annotation\AnnotationProvider::getMethodAnnotations() at /vendor/allure-framework/allure-codeception/src/Yandex/Allure/Adapter/AllureAdapter.php:250
  Yandex\Allure\Adapter\AllureAdapter->testStart() at n/a:n/a
  call_user_func() at /vendor/symfony/event-dispatcher/EventDispatcher.php:212
  Symfony\Component\EventDispatcher\EventDispatcher->doDispatch() at /vendor/symfony/event-dispatcher/EventDispatcher.php:44
  Symfony\Component\EventDispatcher\EventDispatcher->dispatch() at /vendor/codeception/codeception/src/Codeception/PHPUnit/Listener.php:90
  Codeception\PHPUnit\Listener->startTest() at /vendor/phpunit/phpunit/src/Framework/TestResult.php:386
  PHPUnit_Framework_TestResult->startTest() at /vendor/codeception/codeception/src/Codeception/Test/Test.php:74
  Codeception\Test\Test->run() at /vendor/phpunit/phpunit/src/Framework/TestSuite.php:722
  PHPUnit_Framework_TestSuite->run() at /vendor/codeception/codeception/src/Codeception/PHPUnit/Runner.php:106
  Codeception\PHPUnit\Runner->doEnhancedRun() at /vendor/codeception/codeception/src/Codeception/SuiteManager.php:157
  Codeception\SuiteManager->run() at /vendor/codeception/codeception/src/Codeception/Codecept.php:184
  Codeception\Codecept->runSuite() at /vendor/codeception/codeception/src/Codeception/Codecept.php:175
  Codeception\Codecept->run() at /vendor/codeception/codeception/src/Codeception/Command/Run.php:376
  Codeception\Command\Run->runSuites() at /vendor/codeception/codeception/src/Codeception/Command/Run.php:303
  Codeception\Command\Run->execute() at /vendor/symfony/console/Command/Command.php:264
  Symfony\Component\Console\Command\Command->run() at /vendor/symfony/console/Application.php:874
  Symfony\Component\Console\Application->doRunCommand() at /vendor/symfony/console/Application.php:228
  Symfony\Component\Console\Application->doRun() at /vendor/symfony/console/Application.php:130
  Symfony\Component\Console\Application->run() at /vendor/codeception/codeception/src/Codeception/Application.php:103
  Codeception\Application->run() at /vendor/codeception/codeception/codecept:36

  run [-o|--override OVERRIDE] [-e|--ext EXT] [--report] [--html [HTML]] [--xml [XML]] [--tap [TAP]] [--json [JSON]] [--colors] [--no-colors] [--silent] [--steps] [-d|--debug] [--coverage [COVERAGE]] [--coverage-html [COVERAGE-HTML]] [--coverage-xml [COVERAGE-XML]] [--coverage-text [COVERAGE-TEXT]] [--coverage-crap4j [COVERAGE-CRAP4J]] [--no-exit] [-g|--group GROUP] [-s|--skip SKIP] [-x|--skip-group SKIP-GROUP] [--env ENV] [-f|--fail-fast] [--no-rebuild] [--] [<suite>] [<test>]

What is the expected behavior?

I should be able to list a non-allure tag in the codeception.yml file and have Allure ignore those tags.

    enabled:
        - Codeception\Extension\RunFailed
        - Yandex\Allure\Adapter\AllureAdapter
    config:
        Yandex\Allure\Adapter\AllureAdapter:
            deletePreviousResults: true
            outputDirectory: allure-results
            ignoredAnnotations:
                - zephyrId
                - env

Other Information

After the following commit, 5088ee4f7a6f76dfc4d664d4d2ac8ebd2697de01, was made our Codeception project no longer runs when the @env annotation is included in a Test.

After reverting some of the changes that were made to the _initialize function our Tests began running without the above error.

[ivy00johns]

If we add this line $ignoredAnnotations = $this->tryGetOption('ignoredAnnotations', []); below the comment // Add custom ignored annotations on line 83 of the AllureAdapter.php file everything starts working again.

    /**
     * Extra annotations to ignore in addition to standard PHPUnit annotations.
     * 
     * @param array $ignoredAnnotations
     */
    public function _initialize(array $ignoredAnnotations = [])
    {
        parent::_initialize();
        Annotation\AnnotationProvider::registerAnnotationNamespaces();
        // Add standard PHPUnit annotations
        Annotation\AnnotationProvider::addIgnoredAnnotations($this->ignoredAnnotations);
        // Add custom ignored annotations
        $ignoredAnnotations = $this->tryGetOption('ignoredAnnotations', []);
        Annotation\AnnotationProvider::addIgnoredAnnotations($ignoredAnnotations);
        $outputDirectory = $this->getOutputDirectory();
        $deletePreviousResults =
            $this->tryGetOption(DELETE_PREVIOUS_RESULTS_PARAMETER, false);
        $this->prepareOutputDirectory($outputDirectory, $deletePreviousResults);
        if (is_null(Model\Provider::getOutputDirectory())) {
            Model\Provider::setOutputDirectory($outputDirectory);
        }
    }

[Stybbe123]

config:
                 Yandex\Allure\Adapter\AllureAdapter:
                     deletePreviousResults: false
                     outputDirectory: allure-results
                     ignoredAnnotations:
                         - env
                         - dataprovider

const IGNORED_ANNOTATION_PARAMETER = 'ignoredAnnotations';

        $ignoredAnnotations = $this->tryGetOption(IGNORED_ANNOTATION_PARAMETER, []);
        codecept_debug(">>>\n");
        codecept_debug($ignoredAnnotations);
        codecept_debug("<<<\n");

  >>>

  Array
  (
  )

  <<<

does not work in my case

[Stybbe123]

and dont see how it's going to work

private function tryGetOption($optionKey, $defaultValue = null)
    {
        if (array_key_exists($optionKey, $this->config)) {
            return $this->config[$optionKey];
        } 
        return $defaultValue;
    }

Array
  (
      [_initialized] => 1
      [config] => Array
          (
              [Yandex\Allure\Adapter\AllureAdapter] => Array
                  (
                      [deletePreviousResults] => 
                      [outputDirectory] => allure-results
                      [ignoredAnnotations] => Array
                          (
                              [0] => env
                              [1] => dataprovider
                          )

                  )

          )

  )

config structure should be something like : or tryGetOption is not going to get you anything.

       config:
             deletePreviousResults: false
             outputDirectory: allure-results
             ignoredAnnotations:
                - env
                - dataprovider

Array
  (
      [_initialized] => 1
      [config] => 
              [deletePreviousResults] => 
              [outputDirectory] => allure-results
              [ignoredAnnotations] => Array
          (
              [0] => env
              [1] => dataprovider
          )

  )

$ignoredAnnotations = $this->tryGetOption(IGNORED_ANNOTATION_PARAMETER, []);

        codecept_debug(">>>\n");
        codecept_debug($ignoredAnnotations);
        codecept_debug(">>>\n");

 >>>

  Array
  (
      [0] => env
      [1] => dataprovider
  )

  >>>

[baev]

Summon @kouratoras

[kouratoras]

@Stybbe123 just tested and works fine. Did you get the latest master branch?

[Stybbe123]

"allure-framework/allure-codeception": "~1.2.4" latest available release, plus your changes.

[kouratoras]

There is no release with that fix yet, only available to master branch. You can test it by setting:

"allure-framework/allure-codeception": "dev-master"

[Stybbe123]

tested again, works fine @ dev-master Is it going to be released soon ?

[baev]

https://github.com/allure-framework/allure-codeception/releases/tag/1.2.5