[HOW-TO] Anyone with success Running EasyRepro Tests using Microsoft Agents on Azure DevOps?

[tipsey]

Question

It seems like EasyRepro really needs the use of an Interactive Agent when using Azure. Anyone with success using a Microsoft hosted agent? If I run EasyRepro tests on Azure DevOps Express using Interactive Agent, there is no problem but if I use vm agents by Microsoft , some tests fails (Usually ones that don't run headless).

Thanks in advance!

[Shakevg]

@tipsey I am using Microsoft Hosted Agents (2019) and haven't had any issues. Could you describe your issue in detail with code examples and errors

[tipsey]

Also using windows-2019 agent, tried also vs2017-win2016 agent and windows-latest with same issues. I can predict which will fail in Azure if they do not run headless locally in Visual Studio:

Most tests fail with this error and scenario:

System.ArgumentNullException: input cannot be null Parameter name: input

BrowserCommand1.Execute[T1,T2,T3,T4,T5,T6,T7,T8,T9](IWebDriver driver, T1 p1, T2 p2, T3 p3, T4 p4, T5 p5, T6 p6, T7 p7, T8 p8, T9 p9) line 138 BrowserCommand1.Execute(IWebDriver driver) line 35 BrowserPage.Execute[TResult](BrowserCommandOptions options, Func`2 delegate) line 182 WebClient.SetValue(LookupItem control, FormContextType formContextType) line 2429 Entity.SetValue(LookupItem control) line 494

Scenarios: Usually open a menu tab, then clicking a subgrid command

xrmApp.Entity.SelectTab("TabName"); xrmApp.Entity.SubGrid.ClickCommand("grid__active", "Generate Report");

Thank you!

[thathi]

@tipsey we are running in headless too. when I try running with a full browser there were issues around finding the elements. Some of the elements won't be rendered unless you use scrolling. I haven't seen any problems using headless mode with Microsoft agents. But I got your point that Interactive Agent for running Dynamics UI tests in build agents.

[jaithunsheik]

Hi @thathi , @tipsey, @Shakevg I'm trying to deploy the test cases using yaml in azure pipelines but some test cases are passed and other are failed with the above mentioned errors. I'm new to Azure pipelines can somebody help me how to deploy the automated test cases successfully without these issues. Can you please tell me what are these interactive agents and how these can be used? your help is highly appreciated

Thanks

[Shakevg]

jaithunsheik I suppose it is just flaky tests and you should investigate and fix it as any othe selenium test: add additional waiting or in the worst case use sleep. Firstly you can try to add sleep before setvalue and look is it help or not. There is no universal approch here. May be some cases were not take into account durig method impementation.

[jaithunsheik]

Hi @Shakevg Thanks for replying back. I think the issue is caused because of the screen resolution which I tried to set up , some of the issues liking setting up the lookup worked luckily. But setting the date field value to today's date is not working in azure pipelines. Please find the below images

[tipsey]

I think you need to set the DateTime.Today to the expected String format i.e. DateTime.Today.ToString("yyyy/MM/dd"));

Setting up an interactive agent on On-Premise Azure is relatively easy. Setting up in Cloud much more difficult: How to configure an Interactive Agent with a Microsoft Hosted Virtual Machine https://medium.com/@stukalin/self-hosted-azure-devops-agent-on-azure-virtual-machine-70483c4d1526

[thathi]

@jaithunsheik hope you are running it in headless mode and with fixed width & height settings in pipeline? if not try with that.

[Shakevg]

Hi @Shakevg Thanks for replying back. I think the issue is caused because of the screen resolution which I tried to set up , some of the issues liking setting up the lookup worked luckily. But setting the date field value to today's date is not working in azure pipelines. Please find the below images

To change Azure Agent vm resolution you can use next

[jaithunsheik]

Thanks everyone, I will try all the above option and let you know if there are any issues.

[jaithunsheik]

Hi @Shakevg Thanks for replying back. I think the issue is caused because of the screen resolution which I tried to set up , some of the issues liking setting up the lookup worked luckily. But setting the date field value to today's date is not working in azure pipelines. Please find the below images

To change Azure Agent vm resolution you can use next

I did change the screen resolution but still some of the test cases failing while setting field value. Some times it gets failed by setting option set field, sometimes date, look up. Any idea about this issue?

[jaithunsheik]

I think you need to set the DateTime.Today to the expected String format i.e. DateTime.Today.ToString("yyyy/MM/dd"));

Setting up an interactive agent on On-Premise Azure is relatively easy. Setting up in Cloud much more difficult: How to configure an Interactive Agent with a Microsoft Hosted Virtual Machine https://medium.com/@stukalin/self-hosted-azure-devops-agent-on-azure-virtual-machine-70483c4d1526

Thanks @tipsey that fixed date field but still some test cases are failing as mentioned in the above comment. All the test cases works properly in visual studio but throws errors when executed in azure pipeline

[tipsey]

I think you need to set the DateTime.Today to the expected String format i.e. DateTime.Today.ToString("yyyy/MM/dd")); Setting up an interactive agent on On-Premise Azure is relatively easy. Setting up in Cloud much more difficult: How to configure an Interactive Agent with a Microsoft Hosted Virtual Machine https://medium.com/@stukalin/self-hosted-azure-devops-agent-on-azure-virtual-machine-70483c4d1526

Thanks @tipsey that fixed date field but still some test cases are failing as mentioned in the above comment. All the test cases works properly in visual studio but throws errors when executed in azure pipeline

@jaithensheik

I finally figured it out with help on this board. You need to run all tests headless in Azure. I forced tests to run headless with these window options and it worked! Plus to boot they execute a tiny bit faster in headless mode:

using System.Drawing;

var client = new WebClient(TestSettings.Options); if (TestSettings.Options.Headless) client.Browser.Driver.Manage().Window.Size = new Size(1920, 1080);

TestSettings.cs

Headless = true,

A quick note: Test your testcase in Visual Studio with these headless settings. IF working headless locally, it should work on Azure. If doesn't work headless locally, but works in headed, it's probably some other flakiness issue. (timing etc)

[jaithunsheik]

I think you need to set the DateTime.Today to the expected String format i.e. DateTime.Today.ToString("yyyy/MM/dd")); Setting up an interactive agent on On-Premise Azure is relatively easy. Setting up in Cloud much more difficult: How to configure an Interactive Agent with a Microsoft Hosted Virtual Machine https://medium.com/@stukalin/self-hosted-azure-devops-agent-on-azure-virtual-machine-70483c4d1526

Thanks @tipsey that fixed date field but still some test cases are failing as mentioned in the above comment. All the test cases works properly in visual studio but throws errors when executed in azure pipeline

@jaithensheik

I finally figured it out with help on this board. You need to run all tests headless in Azure. I forced tests to run headless with these window options and it worked! Plus to boot they execute a tiny bit faster in headless mode:

using System.Drawing;

var client = new WebClient(TestSettings.Options); if (TestSettings.Options.Headless) client.Browser.Driver.Manage().Window.Size = new Size(1920, 1080);

TestSettings.cs

Headless = true,

A quick note: Test your testcase in Visual Studio with these headless settings. IF working headless locally, it should work on Azure. If doesn't work headless locally, but works in headed, it's probably some other flakiness issue. (timing etc)

Hi @tipsey ,

Thanks for the approach. I tried to execute the test cases using above approach in visual studio and see that when setting the date field element click is intercepted. I have added delay but still same. Don't know how to resolve this issue :\ I did increased the screen resolution and running in headless mode as you said.

Test method threw exception: OpenQA.Selenium.ElementClickInterceptedException: element click intercepted: Element is not clickable at point (532, 622). Other element would receive the click: ... (Session info: headless chrome=99.0.4844.51)

[jaithunsheik]

Hi Everyone, The issue is resolved, I changed the way all the field values are set in form by setting the data field first instead of setting at end. Thanks for everyone support. Much appreciated.

[jaithunsheik]

Hi @tipsey , @Shakevg , @thathi Thanks for your support. Can you please tell me how to use azure pipeline variables to store the credentials and url , and also use them in the test cases while running ? Currently I have hardcoded them in test setting file and would like to azure pipeline variables for more security.

Your guidance will be highly appreciated!!

Thanks, Jaithun

[Shakevg]

@jaithunsheik You can find detail infromation here

[jaithunsheik]

@jaithunsheik You can find detail infromation here

Hi @Shakevg , I followed the app.config method to change to different environments. Is there any way to test how this transform is working locally in visual studio before doing it in devops?

[cookyrag]

Ive started using and extending locally and also found a problem setting fields in NOT headless when they were in a section that was off the screen. I changed the setfield source to call the hover function to force easyrepro to scroll to the field, sections that are not initially displayed are rendered differently

[Shakevg]

@jaithunsheik You can find detail infromation here

Hi @Shakevg , I followed the app.config method to change to different environments. Is there any way to test how this transform is working locally in visual studio before doing it in devops?

I don't know such way. As solution I can suggest test in AzureDevOps without running tests: transform file, then print it content using powershell