search

serenity-bdd / serenity-cucumber

Cucumber integration for the Serenity BDD Reporting library
Other
78 stars 74 forks source link

Seem serenity.take.screenshots = FOR_FAILURES make test fail #215

Open happy1610 opened 4 years ago

happy1610 commented 4 years ago

Dear all, I met this strange case T_T Test always pass, mean that all elements are shown as expected when NOT USING serenity.take.screenshots = FOR_FAILURES/AFTER_EACH_STEP... in serenity.conf

When HAVE serenity.take.screenshots = ..., that case always fail cause element not showing. I also try to debug, and see that scripts're stable So I guess root cause from value serenity.take.screenshots Does any one met this issue before? and what do you do?

Thanks so much

happy1610 commented 4 years ago

@wakaleo please help, thanks so much

wakaleo commented 4 years ago

I can't reproduce this behaviour - do you have a stack trace?

happy1610 commented 4 years ago

I can't reproduce this behaviour - do you have a stack trace?

I'm working on my company project hic, the fail screen likes this image

wakaleo commented 4 years ago

NoSuchElementException - your test is failing to find an element. This isn't related to the screenshot mechanism.

happy1610 commented 4 years ago

NoSuchElementException - your test is failing to find an element. This isn't related to the screenshot mechanism.

first look seems like this, I review the scripts, it's stable. And I do many times to re-run test with having screenshot mechanism => always fail test with NoSuchElementException But when I remove screenshot mechanism, test always pass with correct flow T.T hix

wakaleo commented 4 years ago

Screenshots can slow down test execution and reveal timing issues in the tests that wouldn’t usually be apparent.

happy1610 commented 4 years ago

Screenshots can slow down test execution and reveal timing issues in the tests that wouldn’t usually be apparent.

I also try to add wait time but not work, seem this issue is serious, so what should we do now? hichic. now, I workaround by remove screenshot mechanism.

wakaleo commented 4 years ago

You will need to fix the timing issues in your tests. You can disable screenshots but that is not really a solution to your problem

happy1610 commented 4 years ago

timing issues

timing issues => you mean wait time for element showing?

wakaleo commented 4 years ago

Waiting for the element to be accessible yes, but there is more to timing than just wait times

happy1610 commented 4 years ago

Waiting for the element to be accessible yes, but there is more to timing than just wait times

thank you @wakaleo

nbarrett commented 4 years ago

Here's a thought @happy1610 - I wonder whether your test relies on the very slightly slower performance of the test when running in 'screenshot everything' mode? If so, then you could: