Testing framework decisions

[rendall]

The website needs a testing framework. Eventually it will (likely) have a portal through which our creatives interact with our engine, world, pitching system, and BackStory audiences interact with stories and other content.

Let us use this discussion as a forum to come to a consensus about the best one(s) to use for our needs. Here are some to consider:

• Cypress: An end-to-end testing framework designed for modern web applications. Cypress makes testing easier by running tests in a real browser, allowing you to see exactly what happens as your tests run. https://www.cypress.io/

• Jest: A delightful JavaScript Testing Framework with a focus on simplicity. It works with projects using: Babel, TypeScript, Node, React, Angular, Vue, and more. https://jestjs.io/

• Vitest: A Vite-native test framework that's fast and lightweight, aiming to provide a Jest-like experience without the overhead. It supports native ESM, TypeScript, and Vue out of the box. https://vitest.dev/

• Testing Library: Offers a set of helpers to test UI components in a user-centric way. Rather than dealing with instances of rendered components, it works with actual DOM nodes. https://testing-library.com/

• Storybook: An open-source tool for developing UI components in isolation for React, Vue, Angular, and more. It allows developers to create components independently and showcase components interactively in an isolated development environment. https://storybook.js.org/

We want to think about ease of use and support. I was on a team that used Testing Library and Storybook for the Frontend and Jest for the backend. Worked quite well together, but I want to invite thoughts.

[SundayTyper]

I've used Cypress before and it is dead-easy. Although their analytics and multi-processing features are locked behind a paywall, so the free tier is not desperately fast. Especially as it will render a site and interact via the UI.

Storybook is also a good choice for creating and storing components to use in multiple places (i.e. banners, logos, table stylings)

A different test framework I liked was Playwright. It has asynchronous waits (like Cypress), but runs headlessly by default, so it will execute tests faster. I think it's also got some decent integrations with jQuery and Jest for load testing. As well as exporting its results to a file whch we can create an internal dashboard for

[rendall]

I've used Cypress too and quite like it. However, it can take a lot of time to maintain. We might consider using it, but only for vital user flows, e.g. log in and out, pitching

Storyboard is pretty great, especially for component-heavy sites. You test the front-end components in isolation and can test each state independently. Also, you can see each component visually.

Jest is an industry standard and has lots of support. I've been using vitest for the engine and it's a drop-in replacement for jest. So far, no complaints and it is definitely faster. On the other hand, it was not as informative when there were inefficiencies in the code.

Testing Library is an enhancement of Jest, and I think we should use it for frontend work. Storybook is more of a visual testing framework and development tool, while Testing Library tests internal logic and state.

I have not used Playwright and will take a look. We will be moving away from jQuery, though. Load testing is 👍🏻