search

Codeception / Codeception

Full-stack testing PHP framework
http://codeception.com
MIT License
4.78k stars 1.3k forks source link

Suite isolation of acceptance tests with different actors not working as expected #6246

Closed 7118path closed 3 years ago

7118path commented 3 years ago

What are you trying to achieve?

I am trying to use selenium and phpbrowser in on project. To achive this I use two suites with different suite configs and Actor Instances. They share code in form of a static function in a Helper Module, but the problem also exits when this is not the case.

This is the codeception.yml:

paths:
    tests: .
    output: ./_output
    data: ./_data
    support: ./Support
    envs: ./_envs
actor_suffix: Tester

This is the config of the first suite 'Acceptance':

actor: AcceptanceTester
path: .
modules:
    enabled:
        - \Helper\Acceptance
        - \TYPO3\TestingFramework\Core\Acceptance\Helper\Login
        - Cli
        - Db:
            dsn: 'mysql:host=127.0.0.1;port=3311;dbname=***'
            user: '***'
            password: ***'
            populate: true
            cleanup: false
            dump: '_data/clear_eid_from_db.sql'
        - WebDriver:
            url: https://foobar.de
            wait: 5
            pageload_timeout: 5
            debug_log_entries: 200
            browser: chrome
            capabilities:
                chromeOptions:
                    args: ["--ignore-certificate-errors", "--disable-gpu"]
step_decorators:
    - Codeception\Step\ConditionalAssertion
    - Codeception\Step\TryTo
    - Codeception\Step\Retry

This is the config of the second suite 'Metadata':

actor: MetadataTester
modules:
    enabled:
        - \Helper\Acceptance
        - Cli
        - Asserts
        - PhpBrowser:
            url: https://foobar.de

All of this reside with the codeception.ymlin a dir Tests. I now expect to run the test from the first suite only when I do this command:

./vendor/bin/codecept run -c Tests/codeception.yml Acceptance

What do you get instead?

If I run this command to run the tests from the first suite 'Acceptance'

./vendor/bin/codecept run -c Tests/codeception.yml Acceptance

also the tests from the suite Metadata are run. They fail with errors like this:

 [RuntimeException] Action 'seeResponseCodeIs' can't be called

As seeResponseCodeIs is an action provided by the module phpbrowser I is not available in the suite Acceptance.

Interessting though: If I run this command to run the tests from the second suite 'Metadata'

./vendor/bin/codecept run -c Tests/codeception.yml Acceptance

only the tests from the suite Metadata are run.

Details

Naktibalda commented 3 years ago

I think that you had this issue because of path: . setting in acceptance.suite.yaml file.

Naktibalda commented 3 years ago

You had path: . in configuration because you created a "single suite" setup by running `codecept init acceptance"

7118path commented 3 years ago

Thank you for your answer! I removed the path: . entry from the acceptance.suite.yaml file and everything now works as expected!