Implement test task `setup` and `teardown`

[AlexanderMoskovkin]

Provide a capability to execute code before all tests started (setup) and after all tests completed (teardown).

[inikulin]

Just thought it would be better to implement global setup and tearDown functions, like global before and after in Mocha

[AlexanderMoskovkin]

Implemented in #1047

[inikulin]

Implemented in #1047

No, it's not

[AlexanderMoskovkin]

TestCafe should spawn a process with this script before tests are started and kill it when the tests are finished (if it alive yet).

It looks, the -app does the same. Maybe we should update the initial comment of the issue then?

[inikulin]

It looks, the -app does the same. Maybe we should update the initial comment of the issue then?

No, it's not. We've discussed it. You need a way to setup and teardown using async function, so if you have some complicated long-running setup you'll be able to just return promise for it.

[AlexanderMoskovkin]

ok, I've got it

[BuruY]

Any updates?

[JoshuaKGoldberg]

👋 @AlexanderMoskovkin @inikulin I'm interested in contributing this feature! It seems like this is necessary for having user-defined functions run before & after all tests, right? (my use case)

Questions:

• Would you accept a PR adding this feature in?
• If so, do you have any advice or preferences on API or implementation details?

As a rough proposal, how about something like:

import { allFixtures, allTests } from "testcafe";

allFixtures.before(async f => { /* ... */ });
allFixtures.beforeEach(async f => { /* ... */ });

allFixtures.after(async f => { /* ... */ });
allFixtures.afterEach(async f => { /* ... */ });

allTests.before(async t => { /* ... */ });
allTests.beforeEach(async t => { /* ... */ });

allTests.after(async t => { /* ... */ });
allTests.afterEach(async t => { /* ... */ });

[AndreyBelym]

Thank you for your interest in TestCafe. Of course, your contribution will be warmly welcomed.

The API design is the most difficult part here. Could you please elaborate on the place from where we should call these methods. We can't call them in any test file because of problems with overriding hooks. We need a special CLI switch like --app (and corresponding options in API and configuration files) for specifying a script that will be allowed to set up global hooks.

I am also thinking about methods for configuring global hooks. Right now I think that it's better to simply export global hooks from a script.

[JoshuaKGoldberg]

Could you please elaborate on the place from where we should call these methods.

A couple of equivalents in other test frameworks:

• Jasmine: helpers
• Jest: setupFilesAfterEnv

For TestCafe, my ideal vision would be something equivalent. How about a --testSetup CLI witch equivalent to -app (and corresponding testSetup A optionPI & configuration file)? The above code snippet would go in that file; each test file would then be run with the expectation that the code in any testSetup file(s) was run in that environment.

A little more concrete of an example:

// tests/setup.js
import { allTests } from "testcafe";

allTests.beforeEach(() => {
    console.log = () => {
        throw new Error("Don't use the console, silly!");
    };
});

testcafe chrome tests/**/*.test.js --testSetup tests/setup.js

[miherlosev]

@AndreyBelym

For this feature, I propose to use only programmatic API (such as for fixture and test hooks)

Proposed API

runner
    .beforeEachTest(async t => {})
    .afterEachTest(async t => {})
    .beforeEachFixture()
    .afterEachFixture()

[AndreyBelym]

You can use fixture and test hooks in any test, it doesn't matter how do you start it via CLI or Runner API. So I don't like the idea of limiting this feature to the Runner API only. Also, it still introduces the problem with overriding hooks.

As I said, it's better to use a dedicated script that will export hooks:

hooks.js

export beforeAll () {

}

export afterAll () {

}

export beforeFixture () {

}

export afterFixture () {

}

export beforeTest () {

}

export afterTest () {

}

And use the corresponding API/CLI flags to connect them to tests:

testcafe --hooks hooks.js

runner.hooks('hooks.js');

runner.hooks({
    beforeAll () {

    }
});

[NickCis]

Will these tasks allow me to setup requestHooks for all fixtures?

[Dmitry-Ostashev]

The API for this feature still needs to be discussed, but it should allow using any Test API functions including requestHooks.

[NickCis]

Is there any estimated date for that discussion?. In my work, we need this feature, and we are willing to send the necessary PRs to implement it.

[arubtsov]

Thank you for your interest in our tool and for your willingness to help. We are going to discuss it at the planning of the next sprint (at the beginning of the next week). Your patience is greatly appreciated.

[icfantv]

@arubtsov I'd like to inquire as to the results of your sprint planning with respect to this feature. Thanks.

[Farfurix]

@icfantv

Hello,

We are still researching this issue. I cannot give you any estimates as to when it will be implemented.

[akopper]

In my company we'd require this functionality too!

We want to test the target application independently of the deployment location. Could be localhost or an integration test environment. This functionality could help with a solution so we can specify the target url in the setup task.

Are there any updates about the progress?

[AlexSkorkin]

Once we get any results, we will post them here.

[benmonro]

@AlexSkorkin any update on this? we could use this at Walmart too

[AlexSkorkin]

No updates yet.

[AutomationTester143]

This is a very important feature. It would be a great help if you can implement this asap. Thanks in advance.

import { allFixtures, allTests } from "testcafe";

allFixtures.before(async f => { / ... / }); allFixtures.beforeEach(async f => { / ... / });

allFixtures.after(async f => { / ... / }); allFixtures.afterEach(async f => { / ... / });

allTests.before(async t => { / ... / }); allTests.beforeEach(async t => { / ... / });

allTests.after(async t => { / ... / }); allTests.afterEach(async t => { / ... / });

[benmonro]

@AlexSkorkin Think of how much benefit this can add to things like accessibility & mocking. Here are some examples:

allTests.afterEach(async t => {
  await checkForViolations(); // (using @testcafe-community/axe)
})

allTests.beforeEach(async t => {
  myMockApi.resetState();
});

In the current version of testcafe, there's a lot of redundant boilerplate that each test file would have to add for that. This would also help testcafe library projects like @testing-library/testcafe as it would give the ability to automatically inject client scripts for users.

Perhaps in a global startup file:

allTests.clientScripts({module:"@testing-library/testcafe"})

this would be a big advantage for a lot of teams using testcafe.

[dlangerenken]

100% in need of this feature. Right now, this is our workaround for not having to add common functionality in the .after methods:

function enhancedTestcafeFn(name: string | TemplateStringsArray) {
    const fix = fixture(name);
    const afterEach = fix.afterEach;
    const beforeEach = fix.beforeEach;

    fix.beforeEach = (fn: (t: TestController) => Promise<any>): FixtureFn => {
      return beforeEach(async (t: TestController) => {
        if (logRequests) {
          t.ctx.logger = RequestLogger({
            logRequestHeaders: true,
            logResponseHeaders: true
          });
          t.addRequestHooks(t.ctx.logger);
        }
        /* place here functions that should execute before every test */
        await fn(t);
      });
    };
    fix.beforeEach(async () => null);

    fix.afterEach = (fn: (t: TestController) => Promise<any>): FixtureFn => {
      return afterEach(async (t: TestController) => {
        await fn(t);
        /* place here functions that should execute after every test */
        await analyzeConsoleMessages(t);
        if (codeCoverageEnabled) {
          await collectCodeCoverage(name.toString());
        }
      });
    };
    fix.afterEach(async () => null);
    return fix;
  }

  (enhancedTestcafeFn as FixtureFn).only = enhancedTestcafeFn as FixtureFn;
  (enhancedTestcafeFn as FixtureFn).skip = enhancedTestcafeFn as FixtureFn;

Then import {enhancedTestcafeFn as fixture} in your code to replace the default fixture.

[AutomationTester143]

function enhancedTestcafeFn(name: string | TemplateStringsArray) {
    const fix = fixture(name);
    const afterEach = fix.afterEach;
    const beforeEach = fix.beforeEach;

    fix.beforeEach = (fn: (t: TestController) => Promise<any>): FixtureFn => {
      return beforeEach(async (t: TestController) => {
        if (logRequests) {
          t.ctx.logger = RequestLogger({
            logRequestHeaders: true,
            logResponseHeaders: true
          });
          t.addRequestHooks(t.ctx.logger);
        }
        /* place here functions that should execute before every test */
        await fn(t);
      });
    };
    fix.beforeEach(async () => null);

    fix.afterEach = (fn: (t: TestController) => Promise<any>): FixtureFn => {
      return afterEach(async (t: TestController) => {
        await fn(t);
        /* place here functions that should execute after every test */
        await analyzeConsoleMessages(t);
        if (codeCoverageEnabled) {
          await collectCodeCoverage(name.toString());
        }
      });
    };
    fix.afterEach(async () => null);
    return fix;
  }

  (enhancedTestcafeFn as FixtureFn).only = enhancedTestcafeFn as FixtureFn;
  (enhancedTestcafeFn as FixtureFn).skip = enhancedTestcafeFn as FixtureFn;

Can we use the same in Javascript instead of Typescript? Could you please share the import packages of this file as well?

[aleks-pro]

@AutomationTester143 @benmonro @dlangerenken

Hello,

Thank you for your ideas and feedback. We will update this thread as soon as we have any progress.

[miherlosev]

@AutomationTester143 @benmonro @dlangerenken

Originally, this thread was devoted to another issue. I've updated the original thread description to make it clearer. I've created a separate issue for the use-case that you specified.

[arigarcia]

Hello! I wanna support this idea. This functionality is critical to prepare the environment within the test framework and ease its execution later on in different envs.

[adamskeeled]

Hey! If I'm not mistaken, according to the changelog of v1.17.0, this has already been implemented and released, with the only thing missing being the docs on how to implement this.

But I tried using the basic config template described here https://github.com/DevExpress/testcafe/issues/6656, but it's not working. What could I be missing?

I'm on a fresh project with only one test and this simple config with a console log inside each hook. If I change testRun to fixture then the logs start showing.

[Aleksey28]

Hi, @adamskeeled We implemented and merged only the part of this issue with "test" and "fixture". "testRun" was implemented but is not merged yet and it is not in the changelog. The changelog contains information only about "test" and "fixture". The documentation also has this information on page https://testcafe.io/documentation/403435/guides/advanced-guides/hooks#global-hooks.

[adamskeeled]

Hey @Aleksey28 ,

Thanks for getting back to me so fast! Any idea for when "testRun" will be merged then?

[Aleksey28]

PR is under review and documentation.