Cypress 12.0 Features

[alexfinnarn]

While setting up Cypress for this repo, the cypress open command now has helpful release notes displayed before you get to the test runner screens.

I really like this update and how it makes it easier to follow the main changes. From my perspective, I thought two updates are worth making examples of:

• [ ] cy.origin() - I once was trying to test Salesforce with Cypress and that login process goes to another domain to authenticate which of course breaks the Cypress test. I wonder if this can be used for MiniOrange 2FA (used on WECMS) or other CA projects.
• [ ] "clear the page, cookies, localStorage, and sessionStorage before each test" - This could screw up some tests if people are not aware before updating Cypress. I think I've relied on authentication carrying over to tests within a spec file, but now using cy.session() would be the way to go.

Feel free to add other features that would be good to capture in examples and docs. I haven't tried the component testing out so I'm not sure what's going on with non-e2e Cypress testing features.

https://docs.cypress.io/guides/references/changelog#12-0-0

Dang...they are already on 12.2.0...I can't keep up.

[brentonkelly1982]

Hey @alexfinnarn, I work on the Provider Data Catalog (PDC) product. We recently upgraded Cypress from v10.16.0 to v12.3.0. We have 914 tests in our suite right now.

Some of the most notable things that affected us were

• I heavily referred to migration guides and change logs throughout the process.
• In 11.2.0, Cypress re-introduced the 'Run All Specs' function which we utilize. You turn it on via experimentalRunAllSpecs option in the Cypress config file.
• In 12.0.0, the Cookies.defaults and Cookies.preserveOnce APIs were removed. You now have to utilize the cy.session() command to preserve data between tests.
• In 12.0.0, the .within() command requires a single element and throws an error if given more than one.
• in 12.0.0, testIsolation is turned on by default. This is a HUGE hit to test performance as Cypress resets the browser context before each test by clearing the DOM state by visiting about:blank, clearing cookies in all domains, clearing localStorage in all domains and clearing sessionStorage in all domains. Every single test must now be completely set up from scratch. So if your tests are written to build off of each other, this could be a big problem for you too. Before this, our tests completed in 2.5 minutes but after, they completed in 10.5 minutes! There are flexible options though. What we did was set testIsolation to false in the Cypress config file and then used { testIsolation: true } inside some context() blocks where we needed it. You can see how this is used in the 12.0.0 migration guide. In the end, I was able to get the tests back down to running at around 3.5 minutes.
• In 12.0.0, Cypress dropped support for Node.js 12, 15 and 17. Cypress now requires Node.js 14, 16 or 18+.
• After I finished updating all of the specs, we couldn't run all of the tests without Cypress crashing. The issue was to much memory consumption. We added the numTestsKeptInMemory option to the Cypress config and just set it to 1. This fixed the issue but the side effect is that we don't get snapshot data. More can be read about this option here.

It took me several days to work through this update. I hope this helps.

[alexfinnarn]

Thanks for the update @brentonkelly1982 ! Very good notes to have.

Every single test must now be completely set up from scratch...

So would you say the test isolation change caused the biggest impact while upgrading the test suite?

I haven't gotten a chance to look at the Open Data projects' test suites yet, but for the Drupal sites, since most data is persisted in the Drupal database and cy.session() can preserve the login cookies/info so I don't anticipate much impact from that test isolation config change, except extra time running the whole test suite.

[brentonkelly1982]

You bet @alexfinnarn!

So would you say the test isolation change caused the biggest impact while upgrading the test suite?

Yes, this was definitely the biggest impact. It caused me to edit every spec we have to add at least a cy.visit() to each test. For several tests, I had to also add actions in order to set the page up for the assertions given how our tests previously built off of each other.

[alexfinnarn]

@brentonkelly1982 You might want to look at the experimentalMemoryManagement configuration option in 12.4.0 https://docs.cypress.io/guides/references/changelog#12-4-0 to help with your memory issue.

[brentonkelly1982]

Thanks @alexfinnarn. I upgrade our Cypress to 12.5.0 last week and added that in.