search

Lombiq / Orchard-Base-Theme

An Orchard Core base theme with reusable mixins, components, etc.
BSD 3-Clause "New" or "Revised" License
4 stars 3 forks source link

UI test is flaky, randomly failing with "stale element not found" (OSOE-661) #81

Closed Piedone closed 1 year ago

Piedone commented 1 year ago

BehaviorBaseThemeTests in OSOCE, and specifically TestBaseThemeSiteSettingsAsync is flaky, often failing in GHA with the following exception:

OpenQA.Selenium.StaleElementReferenceException: stale element reference: stale element not found
  (Session info: headless chrome=114.0.5735.199); For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors#stale-element-reference-exception
   at OpenQA.Selenium.WebDriver.UnpackAndThrowOnError(Response errorResponse, String commandToExecute)
   at OpenQA.Selenium.WebDriver.Execute(String driverCommandToExecute, Dictionary`2 parameters)
   at OpenQA.Selenium.WebDriver.PerformActions(IList`1 actionSequenceList)
   at OpenQA.Selenium.Interactions.Actions.Perform()
   at Atata.IWebDriverExtensions.Perform[T](T driver, Func`2 actionsBuilder)
   at Lombiq.Tests.UI.Extensions.NavigationWebElementExtensions.<>c__DisplayClass0_0.<<ClickReliablyAsync>b__0>d.MoveNext() in D:\a\Open-Source-Orchard-Core-Extensions\Open-Source-Orchard-Core-Extensions\test\Lombiq.UITestingToolbox\Lombiq.Tests.UI\Extensions\NavigationWebElementExtensions.cs:line 44
--- End of stack trace from previous location ---
   at Lombiq.Tests.UI.Extensions.ExtendedLoggingExtensions.<>c__DisplayClass23_0.<<ExecuteSectionAsync>b__0>d.MoveNext() in D:\a\Open-Source-Orchard-Core-Extensions\Open-Source-Orchard-Core-Extensions\test\Lombiq.UITestingToolbox\Lombiq.Tests.UI\Extensions\ExtendedLoggingExtensions.cs:line 100
--- End of stack trace from previous location ---
   at Lombiq.Tests.UI.Extensions.ExtendedLoggingExtensions.ExecuteSectionAsync[TResult](UITestContext context, LogSection section, Func`1 functionAsync) in D:\a\Open-Source-Orchard-Core-Extensions\Open-Source-Orchard-Core-Extensions\test\Lombiq.UITestingToolbox\Lombiq.Tests.UI\Extensions\ExtendedLoggingExtensions.cs:line 111
   at Lombiq.BaseTheme.Tests.UI.Extensions.TestCaseUITestContextExtensions.TestBaseThemeSiteSettingsAsync(UITestContext context, Func`1 selectFromMediaLibraryAsync, By byIcon) in D:\a\Open-Source-Orchard-Core-Extensions\Open-Source-Orchard-Core-Extensions\src\Themes\Lombiq.BaseTheme\Lombiq.BaseTheme.Tests.UI\Extensions\TestCaseUITestContextExtensions.cs:line 101
   at Lombiq.OSOCE.Tests.UI.Tests.ThemeTests.BehaviorBaseThemeTests.<>c.<<ThemeFeaturesShouldWork>b__1_0>d.MoveNext() in D:\a\Open-Source-Orchard-Core-Extensions\Open-Source-Orchard-Core-Extensions\test\Lombiq.OSOCE.Tests.UI\Tests\ThemeTests\BehaviorBaseThemeTests.cs:line 27
--- End of stack trace from previous location ---
   at Lombiq.Tests.UI.Services.UITestExecutionSession`1.ExecuteAsync(Int32 retryCount, String dumpRootPath) in D:\a\Open-Source-Orchard-Core-Extensions\Open-Source-Orchard-Core-Extensions\test\Lombiq.UITestingToolbox\Lombiq.Tests.UI\Services\UITestExecutionSession.cs:line 112

Example builds:

This behavior is exhibited mostly under Windows GHA builds, but it also happens under Linux, just less frequently.

Jira issue