Fix test instability related to timers

[wjgerritsen-0001]

Sometimes tests are failing because the build server is too slow/fast e.g. https://github.com/yashaka/NSelene/actions/runs/10847342249/job/30102217890?pr=103

Failed DoubleClick_Waits_For_NoOverlay_IfCustomized [2 s] Error Message: Expected: less than 2024-09-13 10:22:17.1450627 But was: 2024-09-13 10:22:18.2704144 Stack Trace:
at NSelene.Tests.Integration.SharedDriver.SeleneSpec.SeleneElement_DoubleClick_Specs.DoubleClick_Waits_For_NoOverlay_IfCustomized() in D:\a\NSelene\NSelene\NSeleneTests\Integration\SharedDriver\SeleneElement_DoubleClick_Specs.cs:line 238

Failed Hover_Waits_For_NoOverlay_IfCustomized [1 s] Error Message: Expected: less than 2024-09-13 10:22:18.1612626 But was: 2024-09-13 10:22:18.2699995

Stack Trace: at NSelene.Tests.Integration.SharedDriver.SeleneSpec.SeleneElement_Hover_Specs.Hover_Waits_For_NoOverlay_IfCustomized() in D:\a\NSelene\NSelene\NSeleneTests\Integration\SharedDriver\SeleneElement_Hover_Specs.cs:line 238

Solution direction I am thinking of is to make use of ITimeProvider (.Net 8 feature): https://andrewlock.net/exploring-the-dotnet-8-preview-avoiding-flaky-tests-with-timeprovider-and-itimer/ (Backported to netstandard2.0 with nuget package Microsoft.Bcl.TimeProvider)

I understand that we cannot programmatically proceed the time in the browser, but still it might solve our stability issues.

Comments?

[yashaka]

Hm... How the solution would look like? if use ITimeProvider?

[yashaka]

Actually... specifically that failure, from your example above – is pretty weird

The general timeout is 1.0 The polling is 0.05 - almost zero, that means that we will recheck almost instantly... The UI will be ready "for action to pass" - after 300 ms (0.3s) – this means, that specifically in this example, the only reason for test failure of such type – should be the "duration of double click request" to be near 1s... That's pretty weird... that a simple doubleclick can take so long... but who knows... Maybe it can be...

But than, the simplest fix – would be simply increase the timeout from 1.0 to 2.0 in such type of tests.... This would be the most KISS solution...

Maybe in some other cases ITimeProvider would be more useful... What do you think? Maybe utilzing ITimeProvider will not be so complicated that I think...

[wjgerritsen-0001]

My guess is the the CI server is pretty heavy loaded, so that the time to load the new script and double click and all the interactions using selenium can easily take +1 sec.

Let me try to do a sample implementation and discuss whether stuff becomes too complicated with that. ;-)

[yashaka]

Good good! Looking forward!

On Fri, Sep 13, 2024 at 3:42 PM wjgerritsen-0001 @.***> wrote:

My guess is the the CI server is pretty heavy loaded, so that the time to load the new script and double click and all the interactions using selenium can easily take +1 sec.

Let me try to do a sample implementation and discuss whether stuff becomes too complicated with that. ;-)

— Reply to this email directly, view it on GitHub https://github.com/yashaka/NSelene/issues/104#issuecomment-2348867590, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAO6ZHSUFFCKREAUFK65QUDZWLMSFAVCNFSM6AAAAABOFCSOJCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNBYHA3DONJZGA . You are receiving this because you commented.Message ID: @.***>

[wjgerritsen-0001]

The FakeTimeProvider is not rich enough for our purpose, I requested an interface addition:

• https://github.com/dotnet/extensions/issues/5441

This will take time, so for now we just can increase the timeouts.

[wjgerritsen-0001]

@yashaka Please have a look at the above solution direction. It encapsulates testing the timeouts, so a lot of duplicate code can be removed while enforcing that the assertions are everywhere the same.

Another benefit is that this approach is a preparation for using the FakeTimeProvider in the future.