search

thefrontside / bigtest

Ensure your React/Vue/Ember/anything app works perfectly across browsers.
https://frontside.com/bigtest
99 stars 14 forks source link

The secret sauce of reliability and understanding architecture of testing tools in the JavaScript ecosystem #786

Open taras opened 3 years ago

taras commented 3 years ago

There are many tools available in our industry that are designed for browser automation which some use for testing, for example, Playwright. There are also testing tools that manage DOM but are not browsers like Jest. Jest can also be used to execute Playwright commands. This approach is taken by a bunch of tools - like for example QAWolf.

There are many different permutations of this. This variety looks great on the surface but it can lead to choice paralysis for someone who's not familiar with their architecture. Sometimes this leads to people following their peers to choose something that might not work long term.

For BigTest, this means that people are more likely to first consider Playwright rather than BigTest. Alternatively, people who like what BigTest has to offer might be curious how Playwright fits into the picture. We need to make it easier for people who're considering using BigTest to feel assured in their choice and understand the relationship between BigTest and other options.

We don't want to keep having to return to this topic every time a new tool emerges. Instead, we should arm our community with a mechanism to evaluate other tools themselves. For people to do this, they need to be able to answer the following questions

  1. Why the BigTest agent is architected the way that it is?
  2. How do interactors work?
  3. How does convergence fit into this?
  4. How does the fact that tests run in the same runtime as the application affect the reliability of tests?
  5. How do Playwright, Selenium, Cypress, Jest, and TestCafe work?
  6. What do people need to look for in a tool to understand how it's architected?
  7. How can one tell if Interactors might work with their testing infrastructure?

Some bonus questions for us to answer could be,

  1. How could Playwright fit into BigTest?
  2. ...

We need to write a document that we can share the people can use to learn to understand BigTest architecture but also evaluate the architecture of other tools, including their own existing test harnesses. One of the best outcomes would be that people could start to recognize challenges that they had with previous test suites in their newfound understanding of the architecture of a browser test harness and the approach.

taras commented 3 years ago

Here is a good list of existing tools https://testguild.com/top-8-essential-javascript-automation-frameworks/

Automation Testing Made Easy Tools Tips and Training
The Top 11 Essential JavaScript Automation Frameworks
There has been explosive growth in JavaScript Framework usage over the past few years. Discover what are the top javascript automation frameworks
cowboyd commented 3 years ago

How could Playwright fit into BigTest?

@taras Well, since you asked :) https://github.com/thefrontside/bigtest/pull/788