search

jediwhale / fitsharp

Functional testing tools for .NET
http://fitsharp.github.io
Other
152 stars 73 forks source link

WebDriver fixture tests raise exception in FitSharp 2.7.1 #162

Open essenius opened 5 years ago

essenius commented 5 years ago

I've created a C# fixture to drive web tests using Selenium Webdriver (see https://github.com/essenius/FitNesseFitSharpSelenium) using FitSharp (SLIM). This fixture works fine until FitSharp 2.7.0. However, when I use FitSharp 2.7.1, I get an exception with the fixture function that starts the browser and initiates the connection.

This is the stack trace I get when running any FitNesse test using that fixture:

__EXCEPTION__:ABORT_SLIM_TEST: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> SeleniumFixture.StopTestException: Could not start browser: chrome ---> OpenQA.Selenium.WebDriverException: Cannot start the driver service on http://localhost:15703/
at OpenQA.Selenium.DriverService.Start()
at OpenQA.Selenium.Remote.DriverServiceCommandExecutor.Execute(Command commandToExecute)
at OpenQA.Selenium.Remote.RemoteWebDriver.Execute(String driverCommandToExecute, Dictionary`2 parameters)
at OpenQA.Selenium.Remote.RemoteWebDriver.StartSession(ICapabilities desiredCapabilities)
at OpenQA.Selenium.Remote.RemoteWebDriver..ctor(ICommandExecutor commandExecutor, ICapabilities desiredCapabilities)
at OpenQA.Selenium.Chrome.ChromeDriver..ctor(ChromeDriverService service, ChromeOptions options, TimeSpan commandTimeout)
at SeleniumFixture.Model.BrowserDriverFactory.CreateLocalChromeDriver(Boolean headless)
at SeleniumFixture.Model.BrowserDriverFactory.<.ctor>b__5_0()
at SeleniumFixture.Model.BrowserDriverFactory.CreateLocalDriver(String browserName)
at SeleniumFixture.Model.BrowserDriver.NewDriver(String browserName)
--- End of inner exception stack trace ---
at SeleniumFixture.Model.BrowserDriver.NewDriver(String browserName)
at SeleniumFixture.Selenium.NewBrowser(String browserName)
at SeleniumFixture.Selenium.SetBrowser(String browserName)
--- End of inner exception stack trace ---
at fitSharp.Machine.Model.TypedValue.ThrowExceptionIfNotValid()
at fitSharp.Slim.Operators.InvokeInstructionBase.InvokeMember(Tree`1 parameters, Int32 memberIndex)
at fitSharp.Slim.Operators.ExecuteCall.ExecuteOperation(Tree`1 parameters)
at fitSharp.Slim.Operators.InvokeInstructionBase.Invoke(TypedValue instance, MemberName memberName, Tree`1 parameters)

The fixture raises a StopTestException because Selenium raises a WebDriverException, and that seems to indicate a timeout, as also the execution log hints at:

Unable to connect to the remote server
Unable to connect to the remote server
Unable to connect to the remote server
Unable to connect to the remote server
Unable to connect to the remote server
Unable to connect to the remote server
Unable to connect to the remote server
Unable to connect to the remote server
Unable to connect to the remote server
Unable to connect to the remote server
Starting ChromeDriver 76.0.3809.126 (d80a294506b4c9d18015e755cee48f953ddc3f2f-refs/branch-heads/3809@{#1024}) on port 15703
Only local connections are allowed.
Please protect ports used by ChromeDriver and related test frameworks to prevent access by malicious code.

So the strange thing is that the driver service (ChromeDriver) has been started, while the exception says it could not be started. As said, this works fine in FitSharp 2.7.0: the browser gets started and the test runs correctly. Also, when I use RunnerW in 2.7.1 and attach the process in Visual Studio (without setting any breakpoints), I don't get the exception. If I run it in RunnerW without attaching the process, I do get the issue. This makes the suspicion of a timing related issue even stronger.

Is there anything in the 2.7.1 changes that could explain this?

jediwhale commented 5 years ago

The changes in 2.7.1 were to fix a problem with Slim using stdin and stdout when the messages contained non-English UTF-8 characters. (http://www.fitnesse.org/FitNesse.UserGuide.WritingAcceptanceTests.SliM.SlimProtocol.PortManagement). The default in Slim now is to use stdin and stdout. If you !define SLIM_PORT=0 or 1 or 8085, does the result change?

essenius commented 5 years ago

Using port 0, 8085 or 8475 (the last one is the one I normally use) doesn't make a difference. Using Port 1 doesn't work either, but gives a different error: Could not complete testing: fitnesse.testsystems.slim.SlimCommunicationException: Could not send/receive data with SUT. By the way, I get this same error with FitSharp 2.7.0 for port 1 (i.e. stdin/stdout); that's the reason I normally use a free port. Note this only happens with fixtures using Webdriver. I haven't seen it happen with any other fixtures, including those calling REST APIs or UI Automation functions.

jediwhale commented 4 years ago

I downloaded your repo and tried your GoogleTest with 2.7.1 and it worked. Is there any different test in your demos that fails with 2.7.1? image

essenius commented 4 years ago

If I use 2.7.1, none of my tests using Selenium Webdriver work, With 2.7.0 they work if I use another port than 1.