Cannot interact with elements on Azure Pipelines

[benzionyunger]

I'm testing out Azure DevOps Server (on-prem) as a solution to run WAD tests while NOT connected to an RDP session, so I had WAD running off a task on a Microsoft hosted agent 2019.

While executing in an RDP session there are no issues, but if a user tries to execute a pipeline from the web app without an RDP session, although WAD does launch and connects to the application, it cannot perform certain actions like send keys. It's quite interesting, because the WAD server does find elements and verifies it's enabled, even performs a 'clear' on a field, but returns 500 on sending keys.

I also tested while in a console session (disconnected the RDP session with tscon), then it was able to perform fine - except for a screen resolution issue which hid some elements; the 'Set Screen Resolution' task did resolve that, but it only works in a console session.

If someone can help resolve why WAD cannot interact with the elements, that would be great!

The WAD logs are attached (its not very long) for further inspection. Azure WAD logs.txt

[benzionyunger]

After more troubleshooting I found that using the Actions class instead of class driver.sendkeys() does work as follows:

 Actions actions = new Actions(driver);
        actions.moveToElement(e);
        actions.perform();
        actions.sendKeys(value);
        actions.click().perform();

But overall I this didn't resolve by issue because there were still many elements that WAD could not find and threw a 404 Not Found error- just reiterating these elements were found np in an RDP session or console session.

Can anyone resolve or explain this?

[Shakevg]

@benzionyunger Are you using a Self-hosted Agent? Looks like the screen is lock... Did you configure agent according to instruction?

[anunay1]

I too faced issues with send keys and send keys has issues. it's not disposed even after driver.quit(). which language binding are you using.

[benzionyunger]

It is a Miscroft hosted agent, running on interactive mode, with auto-sign on, the script is written in Java using maven and testng. Although I don't have screenshots, I do not believe the issue is with the screen lock because the WAD logs show the script logged into the application (after using a different sendKeys method) which would not have happened had there been a screen locking issue.

But, the inconsistency is continuing after login and WAD cannot find elements -not just send keys, as the script moves on. Again, I dont have any of these issues in an RDP session or console session, just when executing the script from the web app without any session with the host machine.

On Fri, Jun 11, 2021 at 8:46 AM anunay1 @.***> wrote:

I too faced issues with send keys and send keys has issues. it's not disposed even after driver.quit(). which language binding are you using.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/microsoft/WinAppDriver/issues/1540#issuecomment-859286563, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF2M2RUHNMSFXBZBG77VBTDTSGPMJANCNFSM46OGFE7Q .

[Shakevg]

@benzionyunger Your case strange, looks like you are probably confused with Agent types. It is not possible to make an RDP session with Microsoft hosted agent because it is available only as a service, but not a real VM. If you set up Agent manually on Microsft VM - it is a self-hosted Agent.

I still suppose it is an issue with Agent according to error. Could provide a code sample, make a screenshot or video?

[benzionyunger]

@Shakevg Thank you for taking your time to help me out!

Regarding the RDP, I meant that when I have an RDP session with the machine that is hosting the Azure DevOps Server then everything works out alright, when I am disconnected from that session and I access the Azure DevOps app from another machine on the network, then everything goes sour.

And regarding the type of agent, I meant that it's a Microsoft hosted agent, not a self hosted agent as described here

My personal hypothesis on this situation is that although WAD can create a session on an agent, certain basic driver events are touching part of the OS that Microsoft doesn't like since there is no user with authorization. Is your experience only with Azure DevOps Cloud version or also with on-premise Server version? I cannot use the Cloud version due to IT security protocols, maybe there it works better.

Unfortunately I cannot take a screenshot since that also causes a HTTP 500 "unknown error". But here is the relevant lines of code from the beginning until the session errs.

private void setUpDriver(){
        try {
            DesiredCapabilities capabilities = new DesiredCapabilities();
            capabilities.setCapability("app", "Root");
            capabilities.setCapability("platformName", "Windows");
            capabilities.setCapability("deviceName","WindowsPC");
            driver = new WindowsDriver<WindowsElement>(new URL("http://127.0.0.1:4723"), capabilities);
            sleep(3);

            WebElement session = driver.findElementByName("my app name");
            String hex = session.getAttribute("NativeWindowHandle");
            int handle = Integer.parseInt(hex);
            String windowHandle = Integer.toHexString(handle);
            windowHandle = "0x" + windowHandle;

            DesiredCapabilities caps = new DesiredCapabilities();
            caps.setCapability("appTopLevelWindow", windowHandle);
            caps.setCapability("platformName", "Windows");
            caps.setCapability("deviceName","WindowsPC");
            driver = new WindowsDriver<WindowsElement>(new URL("http://127.0.0.1:4723"), caps);
        } catch (MalformedURLException | InterruptedException e) {
            e.printStackTrace();
        }
    }
 public Base login() throws InterruptedException {
        Assert.assertTrue(isPresent(LoginBtn));
        clearField(ServerField);
        find(ServerField).sendKeys(".");
        clickElement(LoginBtn);
        return this;
    }
public boolean isPresent(By by){

        WebElement element = null;
        try {
            element = driver.findElement(by);
        } catch (NoSuchElementException e) {
            e.printStackTrace();
        }
        return element!=null;
    }
    public void clearField(By by){
        WebDriverWait wait = new WebDriverWait(driver, 10);
        WebElement element = wait.until(ExpectedConditions.elementToBeClickable(by));
        element.clear();
    }
    public WebElement find(By by){
        WebDriverWait wait = new WebDriverWait(driver, 20);
        WebElement element = wait.until(ExpectedConditions.presenceOfElementLocated(by));
        return element;
    }