Open Keith-CY opened 1 year ago
Cypress and Playwright have some commonalities and differences in the e2e test scenario:
common ground:
difference:
In general, Cypress is more suitable for use in single-browser e2e test scenarios, and can provide more intuitive and easy-to-understand test reports and a visual test runner. Playwright is more suitable for use in e2e test scenarios of multiple browsers and pages, and can provide more flexible test script writing and execution.
Cypress and Playwright have some commonalities and differences in the e2e test scenario:
common ground:
- Both support various user operations and custom scripts, which can be used to test various aspects of web applications, such as UI, functionality, performance, etc.
- Both provide easy-to-use API and documentation.
- Both support headless browser mode and visual mode to run tests.
- Both provide rich test reports and logs.
difference:
- The supported browsers are different: Cypress only supports Chrome, while Playwright supports Chrome, Firefox and WebKit.
- Different restrictions on test scripts: Cypress can only run test scripts in the same browser window, and Playwright can run test scripts in multiple browsers and pages.
- Assertions and test reports are different: Cypress has a built-in assertion library and a visual test runner, and the test report is more intuitive and easy to understand. Playwright requires the use of third-party libraries to implement assertions and test reports.
- Cypress is free and open source, while Playwright is a paid tool.
In general, Cypress is more suitable for use in single-browser e2e test scenarios, and can provide more intuitive and easy-to-understand test reports and a visual test runner. Playwright is more suitable for use in e2e test scenarios of multiple browsers and pages, and can provide more flexible test script writing and execution.
Only chrome/chromium is used in Neuron because it's the built-in webview of electron so the difference in browsers can be ignored.
Ref:
1.In consideration of a great advantage between Cypress and PlayWright is PlayWright can test multiple browsers in parallel,but Cypress can not.If the difference in browsers can be ignored,Cypress may be the better choice. 2.I have tried to run some easy UI examples successfully through Cypress and PlayWright on my local computer,Cypress is easier for me to understand and use,beacause its documents is more detailed 3.I have found some good videos talking about Cypress Vs PlayWright ,introduce differences in detail https://www.youtube.com/watch?v=fncL63KRA-0 https://www.youtube.com/watch?v=cixkQHNkoZ4
Let's make a decision. From the consideration above and the that we only care about chromium in practice, cypress should be a better choice.
Now we can schedule how and when to move e2e tests from https://github.com/nervosnetwork/neuron-automation to https://github.com/nervosnetwork/neuron/
Now we can schedule how and when to move e2e tests from https://github.com/nervosnetwork/neuron-automation to https://github.com/nervosnetwork/neuron/
Later, Neuron's E2E will be developed by using cypress.
First of all, familiarize yourself with and sort out the use cases in the https://github.com/nervosnetwork/neuron-automation
warehouse. In theory, these use cases will be extended as long as the original functions of Neuron are not adjusted.
After sorting out the cases above, combine the characteristics of Cypress to sort out and output the automation cases.
https://github.com/nervosnetwork/neuron/
is Neuron's existing warehouse. It is more reasonable to open a new warehouse for the automation project later.
After reading the project, I think we need to consider the following points when writing a new project with cypress:
My suggestion is that we can split the task into several parts similar to test plan firstly ,then assign in QA team.
I prefer to add e2e
inside https://github.com/nervosnetwork/neuron/
so each commit of developers will be tested by e2e
.
test each commit
can be achieved by github action
between two repositories too, so it's OK to keep them separate.
The difference is that e2e inside neuron repo
prevents merging if tests fail, while e2e triggered by neuron repo
will not prevent it, only an alert will be reported.
This feature requires introducing E2E test code into Neuron's Repo and synchronizing it with Github Action runs.
Neuron currently has an automated test service at https://github.com/nervosnetwork/neuron-automation, but it is a timed service that is not triggered by every commit. It is also not able to keep up with the updates to https://github.com/nervosnetwork/neuron, which puts a burden on maintenance.
If we can maintain E2E tests in Neuron's Repo, we can improve efficiency and reduce maintenance costs.
Following are the test frameworks suggested in community: