Migration from Protractor to other E2E solutions

[kkangs0226]

Protractor is deprecated and will reach end-of-life in the summer of 2023.

We should migrate to another E2E testing platform soon.

[kkangs0226]

Seems like Angular recommends Cypress for e2e tests, but because Cypress does not use WebDriver, it only supports certain browsers (Chromium-based browsers like Chrome & Edge, and Firefox) and currently only has experimental support for Safari. https://testing-angular.com/end-to-end-testing/ @CATcher-org/2223s2 Would like to know your recommendations for a new E2E platform (if any) and your thoughts about using Cypress.

[cheehongw]

https://blog.angular.io/the-state-of-end-to-end-testing-with-angular-d175f751cb9c

It seems that the Angular Team added support for Cypress, Nightwatch and WebdriverIO. Playwright is also acknowledged by them.

I think it is worth looking into WebdriverIO and Playwright for that Safari support.

[chunweii]

https://blog.angular.io/the-state-of-end-to-end-testing-with-angular-d175f751cb9c

It seems that the Angular Team added support for Cypress, Nightwatch and WebdriverIO. Playwright is also acknowledged by them.

I think it is worth looking into WebdriverIO and Playwright for that Safari support.

Cypress also supports Safari. See https://docs.cypress.io/guides/guides/cross-browser-testing

Cypress seems to be more popular, and it also has a dedicated documentation page for migration. Also see the migration tool

[cheehongw]

I tried migrating the Login page e2e tests to Cypress using the migration tool. It doesn't work very well.

For example, the following line of code in protractor

  async getTitle() {
    return element(by.css('app-root')).element(by.css('app-layout-header')).getText();
  }

should be

return cy.get('app-root').get('app-layout-header')

and not this (as suggested by the migrator tool):

return cy.get('app-root').element(cy.get('app-layout-header')).getText()

Regardless, i tried my best to make the tests work. Here is the spec file: https://github.com/cheehongw/CATcher/blob/try-cypress-migration/cypress/e2e/login.cy.ts

Cypress spawns a electron window and provides a dashboard to use, but does not offer much functionality other than viewing the tests. We can also view recorded runs and debug failed CI runs, but requires a Cypress cloud to store recorded runs. Cypress cloud has a limited free tier

https://user-images.githubusercontent.com/72195240/229367885-9708b4d8-a212-4359-a9e9-71c2804a732b.mp4

I also had occasional runs of cypress failing silently on me, though it might a skill issue on my part. It says the tests has passed but when you expand the test, the test didn't run at all and gave up on an uncaught exception.

[cheehongw]

I tried migrating the login page e2e to Playwright.

https://github.com/cheehongw/CATcher/blob/try-playwright-migration/e2e/login.spec.ts

---

I considered playwright because it is a new promising e2e solution released by Microsoft, and other internet users have suggested it over Cypress. Here are some of the top results from googling cypress vs playwright:

https://news.ycombinator.com/item?id=30105284 https://www.reddit.com/r/QualityAssurance/comments/yqhg55/cypress_vs_playwright_some_comparison_elements/

---

Playwright has an awesome vscode extension, which I found to be very useful in debugging and writing tests with.

The pick locator helps to provide suggestions for picking DOM elements. We can see the e2e test running if the Show Browser option is ticked.

---

Coding style wise, Playwright uses JS promises/ async-await, similar to the async/await we are currently using in Protractor, making it easier to migrate.

Protractor has an alternative coding style, which is known as the WebDriver Control Flow. Similarly, Cypress uses a style that is similar to Protractor's control flow. Since we do not use Protractor's control flow in our e2e tests, it is harder to migrate our code over to cypress.