How to do Hybrid Windows/ Web automation?

[tomw93]

I am wanting to do a hybrid of Windows and Web based automation, where the Windows app contains a web view.

This would involve switching to the web view as described in appium docs for mobile, and then we can use selenium to interact with the webview.

I have seen similar posts in the past but no ETA on it actually being added / a working approach for this.

The best approach at the moment is to use the inspect.exe to find element names - but given our web view is created in react which does most things with CSS names, they don't show up in inspect.

[anunay1]

yes they will not show up in inspect, you will have to use tools like selenium, webdriver io to achieve this.

[kviji]

@anunay1 Is it possible to combine both Windows driver and web driver in same codebase ? Like refer web driver code while clicking web content alone ?

[tomw93]

@anunay1 Is it possible to combine both Windows driver and web driver in same codebase ? Like refer web driver code while clicking web content alone ?

This is what I want to do. Also I can't use selenium because the App itself spawns the browser window, and AFAIK there is no way to 'attach' to a browser.

[kviji]

Thats true @tomw93 . I have the exact situation with my application and would love solutions to help achieve this

[anunay1]

yes it is possible, (using chrome as an example) create a selenium session, it will open chrome, now go to your windows application and click on the button which opens the web browser, I am sure if chrome is selected as a default browser it will open the link in a new tab, and then switch to that tab(its a hunch but it might work) I do not have any application which behaves like this, if you can give me a sample app I can give it s shot.

[tomw93]

@anunay1 It's an application that is a gateway to the main web application which is presented via a WebView. I think this is done by default with Internet Explorer, and is presented through the application - meaning it doesn't just open as a free standing browser. @kviji I've found out that if you give Web elements IDs, those get exposed through to Inspect as 'AutomationIDs' which you can then interact with. But that is not ideal, and really CSS and other attrivbutes of the web element should pass through to the UI automation library.

[kviji]

Hi @tomw93 My web application contains some Web automation ids. However I am not able to detect any of them through Inspect.

[PandaMagnus]

If your devs are willing to modify their code, you can have them launch Chrome set up to accept remote debugging sessions, then you can attach to said session using Selenium. If you're using C#, I've got a blog on the topic: https://intellitect.com/selenium-chrome-csharp/

If you're NOT using C#, this is the blog that I used to learn how to do this before converting to C#: https://www.teachmeselenium.com/how-to-connect-selenium-to-an-existing-browser-that-was-opened-manually/

[matanshapira]

When a proper support for elements inside WebView is expected?

[deepali-hub]

I am also facing the same issue i cannot locate any element in Webview in standard browser application. Is there any solution?

[roymaitri]

I'm trying to automate similar webview in windows application. Did anyone find any solution?

[anunay1]

have you tried the solution that @pandamagnus suggested, Unfortunately I do not have any application which is hybrid, if anyone can share any application I can give it a shot.

[sujaysie]

@anunay1 you can try automating simple Print option of google chrome.

[zahraazezo]

@tomw93 @matanshapira @@anunay1 @roymaitri @kviji Did you find a solution for this case ? hoe to interact with web elements in window app ?

[liljohnak]

We are currently using a hybrid with chromedriver to test this. It works. We are considering moving to WebView2 https://docs.microsoft.com/en-us/microsoft-edge/webview2/howto/webdriver

[junaid-ansari312]

@anunay1 You can try similar example on Notepad>Help>View Help Button It opens a link on Browser