WinAppDriver crashes when using XPaths in prolonged executions

[wildcat111]

When running WinAppDriver for prolonged periods in instances where primarily xpaths are used as element locator strategy, WinAppDriver crashes.

We automated our internal application using WinAppDriver and we mainly use xpaths for locators. After about 18 hours of continuous execution of these tests WinAppDriver kept crashing. We weren’t sure if the issue was with WinAppDriver or with our application. After doing some research I found out other folks had similar issues like the one here: https://github.com/microsoft/WinAppDriver/issues/547.

I am not entirely sure if this crash is happening because of a memory leak. As mentioned in https://github.com/microsoft/WinAppDriver/issues/547 issue, this crash could entirely be related to application.

So, I ran tests against the Windows Calculator App, using both xpaths as Locator Strategy and then just Automation Id as Locator Strategy. The test is very simple, it adds one 9-digit number to another 9-digit number so that every number key is pressed. I used Data Driven approach, using the library here: https://pypi.org/project/robotframework-datadriver/, to run about 4000 of the tests I mentioned using xpaths. After about 3400 test cases or about 16 hours of execution WinAppDriver Crashes consistently.

When using Automation Id as a strategy for the exact same test and execution there is NO crash. I even ran 8000 tests using Automation Id and WinAppDriver did not crash.

It seems to be some issue with xpath Locator strategy that’s causing WinAppDriver to crash.

I’ve attached the Calculator test suite and data support files and the Result set of both xpath execution and Automation Id Execution.

Calc_Tests_Results_AutomationIds_4000.zip Calc_Tests_Results_AutomationIds_8000.zip Calc_Tests_Results_Xpaths_4000.zip CalculatorTests.zip

[Tree55Topz]

What is the actual error that WinAppDriver gives you when it crashes?

Additionally, do you use a lot of Thread.Sleeps in your code? If so, that may be the cause as this is usually a bad solution for dynamic waits when you scale your tests. It looks like from one of those posts that you linked, they were using Thread.Sleep. When you have those being called multiple times, especially when they are nested in method calls, you can easily step on your own toes.

If you find that your solution is using several Thread.Sleep calls I have some code I can share with you where we do not use this at all.

Instead, try making a generic WaitForObject method that can even return an element. Something like below (this is C#)..

// Wait for an Object to be accessible
        public WindowsElement WaitForObject(Func<WindowsElement> element)
        {
            return WaitForObject(element, 30);
        }

        // Wait for an Object to be accessible, use a custom timeout
        public WindowsElement WaitForObject(Func<WindowsElement> element, int timeout)
        {
            var wait = new DefaultWait<WindowsDriver<WindowsElement>>(_session)
            {
                Timeout = TimeSpan.FromSeconds(timeout),
                PollingInterval = TimeSpan.FromSeconds(1)
            };

            wait.IgnoreExceptionTypes(typeof(WebDriverException));
            wait.IgnoreExceptionTypes(typeof(InvalidOperationException));
            wait.IgnoreExceptionTypes(typeof(StaleElementReferenceException));
            wait.IgnoreExceptionTypes(typeof(NoSuchElementException));
            wait.IgnoreExceptionTypes(typeof(NotFoundException));
            wait.IgnoreExceptionTypes(typeof(WebException));

            WindowsElement waitElement = null;

            wait.Until(driver =>
            {
                waitElement = element();

                return waitElement != null && waitElement.Enabled && waitElement.Displayed;
            });

            return waitElement;
        }

[wildcat111]

ProtocolError: ('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)) - This is the error that I am seeing, it's happening when it is trying to click on an element, this is at test number 3355.

There are no sleeps in this code that I've attached and the results attached are from this code as well. It's just straight forward clicks on elements.

I probably should have mentioned we are using Robot Framework for our scripts.

[anunay1]

Can you try the same using Appium desktop

[anunay1]

See the same thing happens there also

[anunay1]

What is the actual error that WinAppDriver gives you when it crashes?

Additionally, do you use a lot of Thread.Sleeps in your code? If so, that may be the cause as this is usually a bad solution for dynamic waits when you scale your tests. It looks like from one of those posts that you linked, they were using Thread.Sleep. When you have those being called multiple times, especially when they are nested in method calls, you can easily step on your own toes.

If you find that your solution is using several Thread.Sleep calls I have some code I can share with you where we do not use this at all.

Instead, try making a generic WaitForObject method that can even return an element. Something like below (this is C#)..

// Wait for an Object to be accessible
        public WindowsElement WaitForObject(Func<WindowsElement> element)
        {
            return WaitForObject(element, 30);
        }

        // Wait for an Object to be accessible, use a custom timeout
        public WindowsElement WaitForObject(Func<WindowsElement> element, int timeout)
        {
            var wait = new DefaultWait<WindowsDriver<WindowsElement>>(_session)
            {
                Timeout = TimeSpan.FromSeconds(timeout),
                PollingInterval = TimeSpan.FromSeconds(1)
            };

            wait.IgnoreExceptionTypes(typeof(WebDriverException));
            wait.IgnoreExceptionTypes(typeof(InvalidOperationException));
            wait.IgnoreExceptionTypes(typeof(StaleElementReferenceException));
            wait.IgnoreExceptionTypes(typeof(NoSuchElementException));
            wait.IgnoreExceptionTypes(typeof(NotFoundException));
            wait.IgnoreExceptionTypes(typeof(WebException));

            WindowsElement waitElement = null;

            wait.Until(driver =>
            {
                waitElement = element();

                return waitElement != null && waitElement.Enabled && waitElement.Displayed;
            });

            return waitElement;
        }

It will be good if you can share how to use these two function.

[Tree55Topz]

@anunay1 seems like this may not matter for them since they are not using Thread.Sleeps which do commonly cause memory issues. Also, I am not too familiar with the Robot Framework so not sure how it would work for you. However, I can share how this function works for my solution.

First, I created my page objects like in this tutorial https://www.automatetheplanet.com/winappdriver-page-objects/

The issue we were having was that whenever a element was instantiated or was being referenced to look for it, the test would fail pretty quickly. Instead of using Thread.Sleep, we created a WaitForObject method that would handle this pretty effectively. I have seen the same approach used in multiple other help forums as well.

You can use the WaitForObject like this

WindowsElement profileTab = WaitForObject(() => (WindowsElement)_session.FindElementByClassName("TabControl").FindElementByName("Profile"));

Additionally, it can be used on an existing WindowsElement. In this example, editView is an element I have defined on the page object

WaitForObject(() => editView);

This would be called before trying to do any click actions or things like that.

[anunay1]

Thanks @Tree55Topz .