[PROPOSAL] Use cypress-testing-library

[ruanyl]

What/Why

What are you proposing?

cypress-testing-library offers bunch of useful Cypress commands which simplified the selector query, and it also encourages developers to write robust, implementation-detail-agnostic tests. That helps developers to write tests that are more user-centric and accessible.

Link to the project: https://github.com/testing-library/cypress-testing-library

What users have asked for this feature?

No applicable

What problems are you trying to solve?

You want tests for your UI that avoid including implementation details and rather focus on making your tests give you the confidence for which they are intended.

You want your tests to be maintainable so refactors (changes to implementation but not functionality) don't break your tests and slow you and your team down.

It mitigates the problems of writing brittle tests that break easily when we refactor our code or change the UI. It also helps to solve the problems of writing tests that don’t reflect how the user actually uses our software or how accessible it is.

What is the developer experience going to be?

The Testing Library family of libraries is a very light-weight solution for testing without all the implementation details. The main utilities it provides involve querying for nodes similarly to how users would find them. In this way, testing-library helps ensure your tests give you confidence in your UI code.

The developer experience is going to be improved by using testing-library. Testing-library provides simple and consistent APIs for querying the DOM elements by their text content, labels, roles, etc.

The testing-library encourages you to write tests that query the DOM elements in the same way that a user would interact with them, rather than relying on implementation details of your components, such as css class names which might change as the implementation changes. This makes the tests easier to understand and maintain.

Are there any security considerations?

No

Are there any breaking changes to the API

No

What is the user experience going to be?

Not applicable

Are there breaking changes to the User Experience?

Not applicable

Why should it be built? Any reason not to?

No applicable

What will it take to execute?

No

Any remaining open questions?

No

[ruanyl]

@kavilla @tianleh ^^

[seraphjiang]

Thanks @ruanyl to make this proposal. a few quetions

1. could we add a few examples to demonstrates the benefit
2. where are the top area from you perspective, we should start to make incremental progress
3. we see the benefit, could we also discuss about concerns, e.g. increased maintenance responsibility to keep it up to date

[ruanyl]

Hi @seraphjiang, thanks for you reply!

1. could we add a few examples to demonstrates the benefit

1. Unit tests and E2E tests will share the consistent API styles which can improve the productivity.

testing-library provides consistent API between react-testing-library and cypress-testing-library, people who wrote unit tests with react-testing-library could start writing E2E tests quickly with less mental efforts.

The query API includes: ByRole ByLabelText ByPlaceholderText ByText ByDisplayValue ByAltText ByTitle ByTestId

2. The testing-library API is semantic and "user-centric", it encourages to write tests in the same way as the end user interact with the product. While cypress APIs are open-ended, people could write different styles of tests.

For example, considering the following html

  <div id="app">
    <label for="username-input-id">Username</label>
    <input id="username-input-id" name="username" class="username-input" />
  </div>

Options with Cypress:

// By class name
cy.get('.username-input').type('testuser')

// By input name
cy.get('input[name="username"]').type('testuser')

Now suppose we refactored the code because of: naming convention changed .username-input -> .usernameInput, internal terminology changed username -> userLogin, or even fixed a typo, etc. The above tests will fail.

With testing-library:

cy.getByLabelText('Username').type('testuser')

The test will always work unless what user see on the screen didn't change.

2. where are the top area from you perspective, we should start to make incremental progress

Since introducing cypress-testing-library is a non-breaking change. I'd encourage people to start writing new E2E tests with it(I'm sure people used react-testing-library will love it :D). Then the team could also consider to refactor existing tests with testing-library on their own pace.

3. we see the benefit, could we also discuss about concerns, e.g. increased maintenance responsibility to keep it up to date

I think one of the challenge is the adoption of cypress-testing-library across different projects. Though the good thing is we are all using react-testing-library, people should be pretty familiar with the API as they are conceptually the same thing. Still this has to be properly communicated, as cypress-testing-library is just a tool that enhanced the cypress default API. But I'm optimistic on this, people familiar with testing-library should be able to pick it up quickly as people could "reuse" what they did with writing unit tests with testing-library.

[tianleh]

take a look

[ashwin-pc]

I like this, but if we do adopt it, we should also add these instructions to /docs/writing_tests.md so that we can document its usage