Question: Why isn't "Before" run before ALL tests and have a "beforeEach" for the other cases?

[dthisner]

Is there a way to run something specific before all the tests? I am looking for logging into a website and taking that local storage to be re-used for the other tests.

Login in takes too long to do it every time, instead I would like to login before ALL the other tests are run and use that local storage in the other tests.

Right now, it seems that "gherkin-testcafe" can only really use Before, which runs before each and every test. Which is not eliminating my issue.

There is a BeforeAll, which you can't use t to call TestCafe calls.

Is there a way around this? Why not using before to run before ALL tests are being run and then have beforeEach to run before each test? :)

Thanks for your great work with Gherkin-Testcafe

[Lukas-Kullmann]

Update:

You can use user roles for this.

--- original comment ---

Hi @dthisner,

unfortunately this is not possible. TestCafé creates a completely new test controller (the t variable) before each test. And a test is mapped to a scenario in gherkin testcafe.

Mapping a test to a scenario brings the benefit that if a scenario fails, other scenarios will be run. If we mapped a test to a feature, the whole feature would fail immediately after the first scenario fails. You would then not get an information on other scenarios that run after the failed one.

Another benefit of mapping a test to a scenario is that TestCafé generates a test report entry for each scenario. If we mapped a test to a feature, it would only report if all scenarios in that feature passed or if at least one failed. You would not get an information which scenario failed an which passed. This would make finding the bugs very hard.

And lastly (and this is the reason why TestCafé is creating a new test controller for each test) it is very desirable to isolate tests as much as possible. If tests were not isolated, things you do in one test would affect other tests. Example: You implemented the cookie notice that you see everywhere on the internet nowadays. In one test, you just casually clicked it away to see everything on the screen. In the next test, you want to test that cookie notice (e. g. testing if you can open the privacy policy). But then it's gone since you already clicked it away.

I hope this makes it clear, why TestCafé implemented the isolation of tests the way they did and why we map tests and fixtures to scenarios and features respectively.

I understand that logging in is a hassle, but waiting the extra 10 seconds or so is worth it in terms of reproducibility and bug hunting. I would even recommend having the system as clean as possible before each test. Internally we create a new user for each test and clear all data that a user may see just to keep the test results consistent. This adds about five seconds to each of our tests, which in turn adds up for dozens of test. But it's worth it. If a bug occurs, we can isolate it pretty quickly before it hits production.

So in conclusion: No it's not possible to log a user in before all tests and the use this user for all tests. This is expected behavior and there are some strong arguments for this.

[dthisner]

Hello @Lukas-Kullmann, Thank you for taking the time and writing up the long answer, it is really appreciated.

The problem is that, sure, the login takes 10sec, but... when going through over 500 test scenarios, those 10sec adds up really quick. =)

I will see if I can do a workaround

Sadly, I am not able to run them in parallel, due to that I am not able to generate users, which would increase the speed immensely as well.

Thanks again for your answer and have an amazing weekend :)

[Lukas-Kullmann]

Maybe you can speed things up by defining a subset of "crutial" tests. Say 100 of them and only run those frequently. And then, on a regular basis (say once a week), you run all the tests. Maybe at nighttime so it does not block regular test runs.

With gherkin tags and filtering of tags, this would be very wasy to implement with gherking-testcafe.

[dthisner]

yeah, I am using tags like crazy! love those guys :)

I will dig into and talk to my project manager how we can set this up. My lead was talking about setting up a Mac Mini for running E2E tests on.

[Lukas-Kullmann]

I just googled a bit and it turns out, that you can do what you are trying to archive with user roles. https://devexpress.github.io/testcafe/documentation/test-api/authentication/user-roles.html

[dthisner]

thats awesome! thank you :)