search

UnlyEd / next-right-now

Flexible production-grade boilerplate with Next.js 11, Vercel and TypeScript. Includes multiple opt-in presets using Storybook, Airtable, GraphQL, Analytics, CSS-in-JS, Monitoring, End-to-end testing, Internationalization, CI/CD and SaaS B2B multi single-tenancy (monorepo) support
https://unlyed.github.io/next-right-now/
MIT License
1.26k stars 111 forks source link

Pages Unit Test #156

Open samuelcastro opened 4 years ago

samuelcastro commented 4 years ago

It'd be great to add pages tests, we could use: https://testing-library.com/docs/react-testing-library/intro

Vadorequest commented 4 years ago

It would be great to know what you mean by "page tests". If you mean integration testing and navigation workflows, then it's better to use Cypress to perform those. (partly because recording of videos and screenshots helps debugging failing tests)

It really comes down to what the pages do, if they are used to display static information, there is not much point testing those, a simple test about checking a random part of the page should suffice to know whether the whole page works or not. For pages with complex navigation workflow, Cypress seems better suited. For pages with complex internal workflow (lots of actions, states), they can be tested by testing the underlying components one by one (component unit tests). Testing the thing as a whole can also be done, Cypress that help with that too.

The problem with automated testing is that "add pages tests" is way too vague and must be really thought of, writing tests takes quite a lot of time, maintaining them is a burden too. Only important stuff should be tested.

For the record, we already have some tests that test several pages using Cypress, including the homepage (which is the most important, IMHO) and those tests are executed automatically using GH Actions upon deployment.

smblee commented 4 years ago

I think @samuelcastro may be referring to your quote

For pages with complex internal workflow (lots of actions, states), they can be tested by testing the underlying components one by one (component unit tests).

Correct me if I am wrong, but I believe for frontend, you only have Cypress for end-to-end frontend testing use case? react-testing-library would be a good library to perform the test you mentioned ^.

Maybe you have some other thoughts on testing ^?

Vadorequest commented 4 years ago

Actually, Cypress is meant to test user workflows (like, navigation).

We also have UI component testing, using react-test-renderer. But only a few components are actually being tested ATM.

https://github.com/UnlyEd/next-right-now/blob/fc357f165abf9ae6d9c452d09d931a907126c5e6/src/components/i18n/I18nLink.test.tsx#L9

With those, I believe we've got enough tooling for testing UI components. Now, it's more a matter of writing the actual tests.

Also, the react-testing-library didn't exist when I chose the tools for UI component testing, maybe it offers more features? I haven't been limited by react-test-renderer so far.

samuelcastro commented 4 years ago

It would be great to know what you mean by "page tests". If you mean integration testing and navigation workflows, then it's better to use Cypress to perform those. (partly because recording of videos and screenshots helps debugging failing tests)

I meant unit tests of DOM elements of the pages and their components, I know we have cypress for e2e/integration and I know it's great for general navigation and the experience as a whole, however, unit testing the elements will give us a better spectrum of testing improving the project quality and reliability, react-testing-library's goal is to unit test components at a lower level, making sure your components work as expected.

Also cypress has a limitation on its free tier price, so as the project grows up, you'll need to pay more.

The best approach for me would be work with both.

react-testing-library uses the simulation of a real DOM through js-dom to find and test elements while react-test-renderer works with pure JavaScript objects, without depending on the DOM, so it gives react-testing-library a lot more power of capabilities. However react-testing-library was not meant to replace react-test-renderer they kind of complete each other, it's a great replacement for Enzyme though which we're not using here anyway.

Also, the react-testing-library didn't exist when I chose the tools for UI component testing, maybe it offers more features? I haven't been limited by react-test-renderer so far.

I don't know when you set it up but the first release of React Testing Library was in March, 2018, it's not a new library.

Vadorequest commented 4 years ago

Thanks for the thorough explanation.

What's the testing use case here? What would you like to test using react-testing-library that cannot be tested by react-test-renderer?

One thing to keep in mind is that the latter in more lightweight, it does less, so that makes sense.

I'm all for adding react-testing-library, once I understand exactly how it will benefit the project. 😉 Adding more lib for the sake of adding more lib/capabilities is pointless if it doesn't help achieve something better. And it makes the project more complicated to understand and maintain too. (I just spent a full day on upgrading dependencies 😢)

samuelcastro commented 4 years ago

react-testing-library is intended for unit/integration/e2e testing directly from the DOM's perspective. It uses React renderer and ReactTestUtils internally, requires real DOM because it's component's output that is asserted in tests, not internals.

react-test-renderer contain a subset of functionality, react-testing-library was built upon it. API is scarce and requires to write boilerplate code or custom utility functions for full-grown testing. React officially promotes react-testing-library as better alternatives.

From the official react page I quote:

We recommend using React Testing Library which is designed to enable and encourage writing tests that use your components as the end users do.

Also read the intro to understand the problem and the solution they propose.

Vadorequest commented 4 years ago

Thanks. I guess we'll add it once we have an actual use case for it.

samuelcastro commented 4 years ago

Btw, we already have many cases that we can use it.

Vadorequest commented 4 years ago

That's true, there are components that could use a few tests. Maybe we should put together a list of those we wish to test and what's to be tested? I don't have time to write too many tests myself, I try to focus on the most important components so far.

samuelcastro commented 4 years ago

Looks like a good plan, I think appBootstrap and pageLayout are the most important ones at this point, I don't think we should test all pages since they're just example pages.

Vadorequest commented 4 years ago

Example pages are already tested, 1 of each (SSG, SSR, etc.) to avoid undetected regression depending on the rendering mode.

SSG with revalidate could also be tested actually, I don't think it is, currently.

The app bootstrap part is definitely a critical bit. Maybe a bit hard to mock, though. Especially the part around the "Component" being passed down by the Next.js framework itself.

Vadorequest commented 3 years ago

Just came across https://www.cypress.io/blog/2020/06/11/introducing-the-cypress-real-world-app/ which is a very useful resource. I hope I find the time to play around and might implement my findings in NRN afterwards.

samuelcastro commented 3 years ago

Great resource indeed.

Vadorequest commented 3 years ago

Cypress has been updated to v6 and now theoretically supports Component testing. (I haven't tried it myself yet)

CopyJosh commented 3 years ago

I've been using react-testing-library for everything but e2e, and have been looking into cypress to fill in that gap. Been looking at this project to hopefully be my "how to" guide of doing this well in nextjs. (No pressure lol).

In any case since this issue caught my eye, with v7 and cypress' component test runner, I feel like that would move the needle toward cypress for everything. I have yet to see a good breakdown article of anyone trying to pit that and react-testing-library head to head but I'm sure the results would be interesting..

Vadorequest commented 3 years ago

Interesting, I don't have much feedback to give on Cypress "as a replacement for react-testing-library", I haven't used it for that purpose and I don't know the pros/cons.

Cypress is moving towards "one engine for all tests", or at least it tries to be the test runner not only for E2E but for components as well. Definitely sounds interesting, although it'd be great to have a "good breakdown article" about the benefits/differences, as you stated.

Cypress 7 brought Component testing, there are a few related issues:

lmiller1990 commented 3 years ago

Hello @CopyJosh! I'll try to answer your questions about Cypress and Testing Library. FYI: I work on Cypress and I maintain Testing Library's Vue integration, so I don't really have a bias either way.

Testing Library provides:

As for the rest, you need to bring your own:

Cypress provides:

You can also switch our the driver part, interestingly enough. More info. Then you can write cy.findByText, for example - using the Testing Library driver, but the test of the Cypress infrastructure (test runner, browser, etc). This is useful if you like the idea of Cypress (see your tests in a browser as they run) but like the testing library driver better.

A summary:

The main thing that enjoy about Cypress is you can see what's going on:

image

Obviously this very useful when building a UI component (similar to Storybook, but you can actually make assertions).

I recorded a video about the different frameworks available for component testing, you might find it useful.

CopyJosh commented 3 years ago

@lmiller1990 wow what an incredibly thorough response. I sincerely appreciate this break down. I suppose that is why there aren’t a lot of head to head comparisons, it is really the kitchen sink vs the entire kitchen we’re talking about here. I can’t wait to get started!

lmiller1990 commented 3 years ago

Let me know if you get stuck on anything - initial configuration can be a little challenging (depending on your stack). I'm around most days to help out :)

Vadorequest commented 3 years ago

Thanks @lmiller1990, really useful to split the responsabilities of each tool, it wasn't clear to me until now!

You mentioned "Cypress runs in a real browser", which made me think about a recent discussion I had with a professional tester. he said to me he didn't want to use Cypress, because of how it's designed. I don't remember the technicals (it has to do with "running in a browser" thing), but he stated Cypress isn't suited for complex E2E, because it cannot leave the current page, meaning it's not usable for testing multi-pages workflow, like testing payments (checkout page > payment page > confirmation code (3DS) > checkout page). I'm not sure how much of that is true, and it's not really the question at hand, but I'd love to learn more about the Cypress limitations.

He compared Cypress to Playwright, basically stating Playwright can do those things, while Cypress cannot, and it has to do with how they were designed in the first place.

eric-burel commented 3 years ago

For the record because it has't been quoted in this discussion: https://github.com/toomuchdesign/next-page-tester This is not for unit testing but integration testing, as it will also trigger data fetching methods. I intend to set this up in my apps later on, and I am digging right now how to use a similar approach in storybook.

Cypress is more often relevant for those test but Jest tends to be faster if you want to unit test a page. Typical use case is checking how query parameters are handled, because this is a responsibility of the page.