Inconsistent Results in Pipeline

[arpalmares]

Actions:

• Test Studio test are working
• Created test user for accessing the tests with access to app and dataverse (CRUD)
• MFA and Security defaults disabled
• Repository cloned in Azure DevOps
• Updated TestURLs.json
• Service connection setup
• Pipeline setup with variables
• Disabled Firefox task (only running Test on Chrome)

Issue:

• Running the Test from the pipeline produces inconsistent results (Same tests in test studio, same commit in repo)
• Sometimes it is successful, but mostly fails. (only 21.43% pass rate)

• Majority of failed runs report the following prompt:

  
  Error Message:
  Test method Microsoft.PowerApps.TestAutomation.Tests.TestAutomation.RunTestAutomation threw exception: 
  OpenQA.Selenium.JavaScriptException: javascript error: Cannot read properties of undefined (reading 'Log')
  (Session info: chrome=99.0.4844.74)
  Stack Trace:
    at OpenQA.Selenium.WebDriver.UnpackAndThrowOnError(Response errorResponse)
  at OpenQA.Selenium.WebDriver.Execute(String driverCommandToExecute, Dictionary`2 parameters)
  at OpenQA.Selenium.WebDriver.ExecuteScriptCommand(String script, String commandName, Object[] args)
  at OpenQA.Selenium.WebDriver.ExecuteScript(String script, Object[] args)
  at Microsoft.PowerApps.TestAutomation.Browser.SeleniumExtensions.ExecuteScript(IWebDriver driver, String script, Object[] args) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Browser\Extensions\SeleniumExtensions.cs:line 165
  at Microsoft.PowerApps.TestAutomation.Api.TestAutomation.InitiateTest(IWebDriver driver, Uri uri) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Api\Pages\TestAutomation.cs:line 142
  at Microsoft.PowerApps.TestAutomation.Api.TestAutomation.<>c__DisplayClass1_0.<ExecuteTestAutomation>b__0(IWebDriver driver) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Api\Pages\TestAutomation.cs:line 35
  at Microsoft.PowerApps.TestAutomation.Browser.DelegateBrowserCommand`1.ExecuteCommand(IWebDriver driver, Object[] params) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Browser\DelegateBrowserCommand.cs:line 28
  at Microsoft.PowerApps.TestAutomation.Browser.BrowserCommand`1.Execute[T1,T2,T3,T4,T5,T6,T7,T8,T9](IWebDriver driver, T1 p1, T2 p2, T3 p3, T4 p4, T5 p5, T6 p6, T7 p7, T8 p8, T9 p9) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Browser\BrowserCommand.cs:line 136
  at Microsoft.PowerApps.TestAutomation.Browser.BrowserCommand`1.Execute(IWebDriver driver) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Browser\BrowserCommand.cs:line 36
  at Microsoft.PowerApps.TestAutomation.Browser.BrowserPage.Execute[TResult](BrowserCommandOptions options, Func`2 delegate) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Browser\BrowserPage.cs:line 182
  at Microsoft.PowerApps.TestAutomation.Api.TestAutomation.ExecuteTestAutomation(Uri uri, Int32 testRunNumber, Int32 maxWaitTimeInSeconds) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Api\Pages\TestAutomation.cs:line 32
  at Microsoft.PowerApps.TestAutomation.Tests.TestAutomation.RunTestAutomation() in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Tests\TestAutomation\RunTestAutomation.cs:line 193
  Standard Output Messages:
  Power Apps  Test Automation Execution Starting Test #1 of 1
  An error occurred during Test Run #1 of 1: OpenQA.Selenium.JavaScriptException: javascript error: Cannot read properties of undefined (reading 'Log')
  (Session info: chrome=99.0.4844.74)
  at OpenQA.Selenium.WebDriver.UnpackAndThrowOnError(Response errorResponse)
  at OpenQA.Selenium.WebDriver.Execute(String driverCommandToExecute, Dictionary`2 parameters)
  at OpenQA.Selenium.WebDriver.ExecuteScriptCommand(String script, String commandName, Object[] args)
  at OpenQA.Selenium.WebDriver.ExecuteScript(String script, Object[] args)
  at Microsoft.PowerApps.TestAutomation.Browser.SeleniumExtensions.ExecuteScript(IWebDriver driver, String script, Object[] args) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Browser\Extensions\SeleniumExtensions.cs:line 165
  at Microsoft.PowerApps.TestAutomation.Api.TestAutomation.InitiateTest(IWebDriver driver, Uri uri) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Api\Pages\TestAutomation.cs:line 142
  at Microsoft.PowerApps.TestAutomation.Api.TestAutomation.<>c__DisplayClass1_0.<ExecuteTestAutomation>b__0(IWebDriver driver) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Api\Pages\TestAutomation.cs:line 35
  at Microsoft.PowerApps.TestAutomation.Browser.DelegateBrowserCommand`1.ExecuteCommand(IWebDriver driver, Object[] params) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Browser\DelegateBrowserCommand.cs:line 28
  at Microsoft.PowerApps.TestAutomation.Browser.BrowserCommand`1.Execute[T1,T2,T3,T4,T5,T6,T7,T8,T9](IWebDriver driver, T1 p1, T2 p2, T3 p3, T4 p4, T5 p5, T6 p6, T7 p7, T8 p8, T9 p9) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Browser\BrowserCommand.cs:line 136
  at Microsoft.PowerApps.TestAutomation.Browser.BrowserCommand`1.Execute(IWebDriver driver) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Browser\BrowserCommand.cs:line 36
  at Microsoft.PowerApps.TestAutomation.Browser.BrowserPage.Execute[TResult](BrowserCommandOptions options, Func`2 delegate) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Browser\BrowserPage.cs:line 182
  at Microsoft.PowerApps.TestAutomation.Api.TestAutomation.ExecuteTestAutomation(Uri uri, Int32 testRunNumber, Int32 maxWaitTimeInSeconds) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Api\Pages\TestAutomation.cs:line 32
  at Microsoft.PowerApps.TestAutomation.Tests.TestAutomation.RunTestAutomation() in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Tests\TestAutomation\RunTestAutomation.cs:line 162
  Current results directory location: D:\a\_temp\..\TestResults\TestResults\Deploy_VssAdministrator 2022-03-24 12_50_24\In

[arpalmares]

This is already fixed. Added code to extend timeouts on TestURLs.