Closed stuckless closed 3 years ago
I definitely think option 2 seems like a good idea and easier to maintain and implement. I will create an issue to track this effort.
I passed along this issue to the Carbon team to see what their thoughts are.
need to submit a PR to show what I am asking for from the Carbon team
We should not underestimate the effort in achieving this for our more complex components. The testable location of a part of a component, i.e. a button somewhere in the table, could require:
I assume the above must also be part of the contract. Since we need to write tests to maintain the actual contract, perhaps we can somehow also export the code that delivers the part that should be tested (will be testing framework specific). Kind if like the wrapper suggestion, but less effort from ours side, since we need to write the tests anyway.
PR created for Carbon https://github.com/carbon-design-system/carbon/pull/9652
Carbon says they will support the ability to add a data-testid on all top level components but will not be implementing that in their codebase. Closing this issue as we have completed the contract for our repo and got a conclusion from Carbon. If we want to discuss how we can further improve our testing stability let's open a new issue.
Goal: For any given component provide a contract on how that component can be tested. It's a contract like an API contract that will remain consistent from build to build and ideally between major releases.
Semantic versioning is a good tool to convey that are you are not introducing breaking changes in minor and patch releases, and this works great for libraries since they are non-visual and provide a single entry point, the API.
Can you change the DOM? Yes, absolutely, provided that it doesn't break your testability contract. eg, if you say, to find a dialog you'll use
role=dialog
andtitle=Your Title
but then you change the dom to use<dialog>
element and norole
, then that is a breaking change. The testing contract is broken.To have more flexibility around the component DOM, there are a couple of options: 1) define a set of component testing wrappers for a component. ie, Dialog("title").click("Send") or Dialog("title").isPrimaryAction("Send").
2) define a testing interface as a set of locators. ie,
Regression fix contract
For Example Button
role = button
andtext=TEXT
click( role = button and text =TEXT)
role=button
anddata-kind=primary
role=button
and iconalt text=ABC
click(role=button and alt text=TEXT)
Dialog
role=dialog
andtext=Dialog Title
role=button
andtext = Button Text
anddata-kind = primary
click(role=button and text=Text)
Dropdown
datatest-id=dropdownid
textcontent=text
and hasdata-selected=true
click(textcontents=text)
When a component is created the testing contract should be created as well. That contract should be validated in testing-library or cyrpress. The contract would help validate the components, and it would serve as documentation to consumers on how they can build resilient tests around the components that they consume.
Ideally Carbon would adopt the testing contract as well.