microsoft / WinAppDriver

Windows Application Driver
MIT License
3.68k stars 1.4k forks source link

Finding elements according to inspect.exe #2034

Open Kevv55 opened 1 week ago

Kevv55 commented 1 week ago

I can find the element names, on both inspect.exe and Accessibility Insights but when I use FindElementByName() function it returns the following error:

{"status":7,"value":{"error":"no such element","message":"An element could not be located on the page using the given search parameters."}}

My assumption is because I want to click on an element that is a combo box and is the child of other elements (element tree screenshot from Accessibility Insights below). When I try to use the FindElementByName() on the parent elements I get the same error, FindElementByClass does not produce anything for the UI elements that inspect.exe/Accessibility Insight display the class name, and I wouldn't know how to find the Xpath of the UI element.

Shot1

Element I am trying to access: Screenshot 2024-10-07 160223

image

My first question is: In a WindowsDriver session is there a way to display the names of the available elements, so finding them by trial and error can be reduced.

Below is my code, it is written to Automate a Win32 application, everything works until the point where I have to find the "cbocode" combo box element:

public class Tests
{
    public const string DriveURL = "http://127.0.0.1:4723/";
    // It works dont mind the error
    public WindowsDriver<WindowsElement> DesktopSession;

    [SetUp]
    public void Setup()
    {
        Console.WriteLine("Execution Started");
        AppiumOptions Options = new AppiumOptions();
        Options.AddAdditionalCapability("app", @"App_location");
        Options.AddAdditionalCapability("newCommandTimeout", "100");
        Options.AddAdditionalCapability("platformName", "Windows");
        Options.AddAdditionalCapability("deviceName", "WindowsPC");
        DesktopSession = new WindowsDriver<WindowsElement>(new Uri(DriveURL), Options);
        Assert.IsNotNull(DesktopSession);
        var splashHandle = DesktopSession.CurrentWindowHandle;
        Console.WriteLine(splashHandle);
        Thread.Sleep(TimeSpan.FromSeconds(7));

        // To deal with the splash screen
        var handles = DesktopSession.WindowHandles;
        DesktopSession.SwitchTo().Window(handles[0]);
        Console.WriteLine("Current Window handle");
        Console.WriteLine(DesktopSession.CurrentWindowHandle);
        DesktopSession.Manage().Timeouts().ImplicitWait = TimeSpan.FromSeconds(1.5);
    }

    [Test]
    public void Test1()
    {

        WindowsElement loginWindow = DesktopSession.FindElementByName("Login");
        Assert.IsNotNull(loginWindow);
        loginWindow.Click();

        loginWindow.SendKeys("type into message box");
        loginWindow.SendKeys(Keys.Enter);

        Console.WriteLine("Window handle after the login prompt");
        Console.WriteLine(DesktopSession.WindowHandles[0]);
        Console.WriteLine(DesktopSession.WindowHandles.Count);

        WindowsElement custButton = DesktopSession.FindElementByName("cmdCust");
        Assert.IsNotNull(custButton);
        custButton.Click();
        Thread.Sleep(TimeSpan.FromSeconds(5));

        WindowsElement custWindow = DesktopSession.FindElementByName("Customer");
        Assert.IsNotNull(custWindow);
        custWindow.Click();
        custWindow.SendKeys("xxxxxx");
        custWindow.SendKeys(Keys.Enter);
        Thread.Sleep(TimeSpan.FromSeconds(5));

        // Produces an error {status: 7, Cannot find the element with the given parameters}
        WindowsElement cmboBox = DesktopSession.FindElementByName("cbocode");
        Assert.IsNotNull(cmboBox);
    }

}

}

I was having problems with the splashscreen so I debugged it all to ensure the WindowHandle only changes after the splashscreen.

My main problem is having something clearly described by inspect.exe/Accessibility insight but not being able to find it via FindElementbyName/ID/Class. Why is the Windows driver able to find some elements but not others, and how can I go around this to be able to click and move into those elements?

I know there are probably more effective ways to write this code, so if you could also leave a link where I could find all the documentation needed with the functions available for OpenQA.Selenium.Appium.Windows api I would really appreciate it.

Shakevg commented 2 days ago

Kevv55 Inspect.exe often does not show the real tree available for WinAppDriver. You can get the actual element using DesktopSession.PageSource (It will get XML), then format it in some tool (I use xpather; it can also verify XPath) or save it to a file and view the actual controls.