search

plone / volto

React-based frontend for the Plone Content Management System
https://demo.plone.org/
MIT License
427 stars 575 forks source link

Fix cypress test "As editor, I can unlock a locked page" #5933

Closed wesleybl closed 1 month ago

wesleybl commented 1 month ago

Sometimes this test fails because it does not find the alert with the message that the content is locked. What may be happening is that the test runs too quickly, and Plone does not consider that the content has been locked. Then we wait a bit for Plone to consider it locked.

This fix: https://github.com/plone/volto/actions/runs/8403447599/job/23013937367#step:10:649

netlify[bot] commented 1 month ago

Deploy Preview for volto canceled.

Name Link
Latest commit 22425c88fbc337b7eb25911e973cee2fad44b0d2
Latest deploy log https://app.netlify.com/sites/volto/deploys/66067530d08a6500084affd6
netlify[bot] commented 1 month ago

Deploy Preview for plone-components failed.

Name Link
Latest commit 22425c88fbc337b7eb25911e973cee2fad44b0d2
Latest deploy log https://app.netlify.com/sites/plone-components/deploys/66067530ae2f020008da8bd4
ichim-david commented 1 month ago

@wesleybl thx, there is also a flaky one on the listing block tests which I will have to look at and add a pull request myself. It's annoying for me that we have to add these wait timeouts, I feel that we are not using Cypress to the best of its abilities but I guess it is what it is and we can always improve in the future.

ichim-david commented 1 month ago

@wesleybl unfortunately the problem still occurs and I have a branch where I had to get rid of the check for the toast with lock info since every so often the toast didn't show up. https://github.com/plone/volto/commit/984757505f17c00b3f32c481a0e88526427f7a9a#diff-fcada51db602a0197ba4aa1b737618004ca91e9d00e8fbdca4c0bff93b2b8eafL104

I myself have made pull requests with test fixes only to discover that they still crashed every so often due to a flaky test scenario. I have started to look into this topic and have found this user on youtube which has a series of tutorials about flaky tests scenarios and how to fix them. This video shows how you can wrap the test and duplicate it so that you see how it behaves after several runs. https://www.youtube.com/watch?v=bWR6zFGywMI

In my own experience you need to run the test 20 times and several times to see if the test passes or not. Here is a screen recording of my test trial where I ran it 20 times 2 times and I finally got no failures after 2 commits in this branch.

I write this as a recipe for you to have in mind to use as well if you write tests or want to fix tests and ensure that they are fixed for good. From now on I will apply this recipe to run it several times before accepting a pull request as fixed as running it just once or twice by running all tests again is not a good test bench on the effectiveness of the fix.

https://github.com/plone/volto/assets/152852/3ceb2af8-6276-4e63-b18b-b42102e34067

wesleybl commented 1 month ago

@ichim-david great! I often cannot simulate an intermittent error locally. With your tip it seems possible. Thank you for sharing it!