Override Selenium Configuration using Java System Properties

[aquddos123]

I am trying to override selenium configuration using java system properties in my project (see project configuration below) but it is not working. Instead of invoking SeleniumGridWithCapabilitiesDriverFactoryFactoryBase() it is still invoking LocalDriverFactory() for one of SeleniumDriverSetup methods. Any idea how can I achieve this in my project?

Please note, instead of using JUnit to launch a FitNesse test suite (like used in sample-fitnesse-project) I am using Java Task using "maven-antrun-plugin" to start FitNesse server. Since it is starting three different jvm's in my case (one for maven process, second for fitnesse server, and third for slim engine) I am suspecting the java system properties are going into a different jvm than the one I am trying to access from. See my project configuration below:

<dependencies>
    <dependency>
        <groupId>org.fitnesse</groupId>
        <artifactId>fitnesse</artifactId>
        <version>${fitnesse.version}</version>
        <scope>runtime</scope>
    </dependency>
    <dependency>
        <groupId>nl.hsac</groupId>
        <artifactId>hsac-fitnesse-fixtures</artifactId>
        <version>5.2.9</version>
        <scope>runtime</scope>
    </dependency>
    <dependency>
        <groupId>org.fitnesse.plugins</groupId>
        <artifactId>maven-classpath-plugin</artifactId>
        <version>1.9</version>
        <scope>runtime</scope>
    </dependency>
</dependencies>

<profile>
    <id>integration</id>
    <build>
        <plugins>
            <plugin>
                <artifactId>maven-antrun-plugin</artifactId>
                <version>1.6</version>
                <executions>
                    <execution>
                        <id>start-fitnesse-integration</id>
                        <phase>integration-test</phase>
                        <configuration>
                            <tasks>
                                <echo taskname="FitNesse" message="Starting FitNesse ${fitnesse.version}..."/>
                                <java classname="fitnesseMain.FitNesseMain" classpathref="maven.runtime.classpath" fork="true" failonerror="true">
                                    <arg line="-p ${fitnesse.port}"/>
                                    <arg line="-c ${suiteFilter}"/>
                                    <arg line="-d ."/>
                                    <jvmarg value="${slim.port}"/>
                                    <jvmarg value="${appBaseUrl}"/>
                                    <jvmarg value="${appContextPath}"/>
                                    <jvmarg value="${appUserName}"/>
                                    <jvmarg value="${appPassword}"/>

                                    <jvmarg value="${seleniumGridUrl}"/>
                                    <jvmarg value="${seleniumCapabilities}"/>
                                    <jvmarg value="${fitnesseSuiteToRun}"/>

                                    <jvmarg value="-DseleniumGridUrl=http://<username>:<accessKey>@ondemand.saucelabs.com:80/wd/hub"/>
                                    <jvmarg value="-DseleniumCapabilities=browserName:chrome, platform:Windows 10, version:latest"/>
                                    <jvmarg value="-DfitnesseSuiteToRun=HsacExamples.SlimTests.BrowserTests.ScenarioTest"/>

                                    <argLine>-DseleniumGridUrl=http://<username>:<accessKey>@ondemand.saucelabs.com:80/wd/hub</argLine>
                                    <argLine>-DseleniumCapabilities="browserName:chrome, platform:Windows 10, version:latest</argLine>
                                    <argLine>-DfitnesseSuiteToRun=HsacExamples.SlimTests.BrowserTests.ScenarioTest</argLine>

                                    <sysproperty key="seleniumGridUrl" value="http://<username>:<accessKey>@ondemand.saucelabs.com:80/wd/hub"/>
                                    <sysproperty key="seleniumCapabilities" value="browserName:chrome, platform:Windows 10, version:latest"/>
                                    <sysproperty key="fitnesseSuiteToRun" value="HsacExamples.SlimTests.BrowserTests.ScenarioTest"/>
                                </java>
                            </tasks>
                        </configuration>
                        <goals>
                            <goal>run</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</profile>

I am using the following command to start test suite in integration mode:

mvn clean -Pintegration integration-test -Dfitnesse.port=8001 -DseleniumGridUrl=http://<username>:<accessKey>@ondemand.saucelabs.com:80/wd/hub -DseleniumCapabilities="browserName:chrome, platform:Windows 10, version:latest"

With the above property settings, I am able to access the values in the wiki pages but not at the code level which is required to override selenium settings in the hsac-fitnesse-plugin.

So I am looking for either of the two solutions. Please advise.

1. First (preferred), as discussed in "HsacExamples.SlimTests.BrowserTests.OverridingSeleniumSetup", a way to be able to override using command line arguments and/or by setting properties within Ant task.
2. Second, if the above is NOT possible, set the properties in the code through wiki pages and then write some custom code to override the configuration in hsac-plugin code before calling any SuiteSetUp. Something like below:

new SeleniumGridDriverFactoryFactory().willOverride();

[fhoeben]

To start off with the most question: Why don't you just switch to running the tests using the jUnit approach in the sample project? What's the advantage of spawning 2 processes instead of 1 and requiring ports to be available?

In an antrun setup I don't think you should have to specify fitnesseSuiteToRun as that is a system property picked up specifically by the jUnit runner used by the jUnit setup, and you are passing that using the -c command line option.

This also brings us to you proper question. Allowing the selenium override in this setup will not be possible using the override setup based on the properties you are passing. Reading the property values and using these to configure the selenium driver used by the fixtures (in the Slim process) and then locking this configuration so the wiki page content can no longer change it, is all built into the jUnit runner also (see https://github.com/fhoeben/hsac-fitnesse-fixtures/blob/560e4b346a4c8dd109ca88957808fec1ffe4ca37/src/main/java/nl/hsac/fitnesse/junit/HsacFitNesseRunner.java#L304). That code will never be invoked in your setup.

You would need to build something similar to the code in that runner and ensure it is executed (dealing with the fact you have a separate FitNesse and Slim process) before (or instead of) any 'selenium driver setup' table in the wiki.

Alternatively I believe you should be able to create separate suites for different connection parameters (either fixed or using system properties to supply value at startup) each using completely different SuiteSetUp wiki pages using FitNesse's symbolic links feature. You would have a single page hierarchy containing just the test pages and then separate suites (next to the top level page of the test hierarchy) that contain just a property to link the test hierarchy and a single SuiteSetUp page. (This would be similar to setting up different database connections as described in answer to this Stackoverflow question: https://stackoverflow.com/questions/64765404/can-fitnesse-page-tags-be-used-in-page)

I imagine you could have one hierarchy configured for 'local' selenium use, which one would use when running FitNesse on a local machine with a browser installed. And possibly one 'remote' where the parameters would based on values passed as system properties to the FitNesse process (which can then be used as ${MY_PROP} in the SuiteSetUp page) which would be flexible in a similar way to what the jUnit runner does: well suited for have a single config with varying settings in a CI pipeline. But you can also add additional suites for specific configurations to make it easy and clear what these configurations should be (e.g. test using the Selenium grid using Safari on MacOS, test using the grid using Edge on Windows, test using the grid using Chrome on Linux). Switching between these different configs is than a matter of running the correct suite (each suite will have all test pages).

[aquddos123]

Thanks @fhoeben for detailed response. We are not using JUnit test runner for two reasons: First, main reason is that this is not a new system we are configuring. We have our test suites already configured locally and in CI setup. We just decided to extend our test system and take advantage of this plugin fixtures in our future test suites. I think we are quite successful with this upgrade so far, just need to deal with this selenium override and next is container execution. Second, we don't like JUnit reports. In fact, this was the main reason we didn't adopt JUnit runner in the first place. We like native FitNesse reports which we are getting by embedding an argument "format=html" in our command line arguments and then rendering html report as seen in the below command which runs inside container

java -cp plugins.properties:fixtures/*:libs/* fitnesseMain.FitNesseMain -p 8002 -o -c HsacExamples.SlimTests.BrowserTests?suite"&"format=html"&"runTestsMatchingAnyTag=demo

Switching to JUnit runner is going to be little expensive and challenging for us at this time therefore we are looking into an easy BUT reliable workaround for our future test suites.

We already have a mechanism of overriding environment configuration in our test system and that we are doing using command line arguments and some fitnesse variables at SuiteSetUp level.

I'll spend some time to explore more about the options you've suggested and get back to you.

Thanks!

[fhoeben]

Ok.

But please note that my jUnit runner also ensures the FitNesse HTML are generated as part of the jUnit run.... :-)

So I would really recommend to take a closer look at running via jUnit. In my experience it is just a matter of adding one jUnit class to your codebase (or just copying the one from the sample project). And updating the pom.xml. I really think it will be much less work than building your own override. And the runner also generates some extra files which can be used to rerun failed tests and generate overview reports over multiple suite runs (that you could be doing in parallel)...

[aquddos123]

"But please note that my jUnit runner also ensures the FitNesse HTML are generated as part of the jUnit run.... :-)" Did you mean the HTML reports which look like this?

When I say native FitNesse report, we are generating something like below. Is this type of reporting possible with JUnit runner in sample project?

Also, can you please point out which jUnit class I might need to copy to enable jUnit runner in my setup? Is it same HsacFitNesseRunner.java or a different one?

[fhoeben]

@aquddos123 you are right the top level page will look like your first screenshot. There will not be a single page containing all test HTML pages like the page one sees in the wiki, instead you have to click on individual test pages to see their generated HTML. For larger suites having a separate pages seems much better to be. If the suites get larger I recommend using the 'combine' tool in my fixtures to generate a nicer top-level overview (see for instance https://fhoeben.gitlab.io/-/hsac-fitnesse-fixtures/-/jobs/2569203324/artifacts/test-results/index.html or https://fhoeben.gitlab.io/-/hsac-fitnesse-fixtures/-/jobs/2569203409/artifacts/example-results/index.html)

The 'combine' command is meant to combine multiple suite results into a single overview but can also be used for a single suite. They immediately give an overview whether all tests passed, first create a separate section with all the failed test and then list all tests in order of descending time it took to execute the page. It will be a much more concise and useful overview than either of the two screenshots you posted.

The 'combine' command is actually just running nl.hsac.fitnesse.junit.reportmerge.HtmlReportIndexGenerator the sample project has this setup in its pom.xml using the maven-exec-plugin 'combine-reports'

[aquddos123]

In my experience it is just a matter of adding one jUnit class to your codebase (or just copying the one from the sample project). And updating the pom.xml

Can you please elaborate this a little bit?

[fhoeben]

Well: you add a class similar (or even exactly the same) to https://github.com/fhoeben/sample-fitnesse-project/blob/master/src/test/java/nl/hsac/fitnesse/sample_project/FixtureDebugTest.java to your project's test-sourcedir. (If you also have regular jUnit tests that should be executed before the FitNesse suite, you should exclude this class from normal test run, see https://github.com/fhoeben/sample-fitnesse-project/blob/9091e4a002681413b04cfe40b167a4c616c893db/pom.xml#L83-L91)

The value in the @FitNesseRunner.Suite annotation is the test/suite that will be run, but that can be overridden using the system property fitnesseSuiteToRun I usually bind executing the FitNesse test to the failsafe plugin, see https://github.com/fhoeben/sample-fitnesse-project/blob/9091e4a002681413b04cfe40b167a4c616c893db/pom.xml#L93-L133

Then if you want to get the run overview from the combine tool you can have your CI server generate that use the exec plugin, see https://github.com/fhoeben/sample-fitnesse-project/blob/9091e4a002681413b04cfe40b167a4c616c893db/pom.xml#L155-L163

Using this setup you can then using any selenium configuration you want by either setting properties in the pom.xml itself, or adding them to the command to run the failsafe test via maven. This is what is described in 'HsacExamples.SlimTests.BrowserTests.OverridingSeleniumSetup'

Actually in my pipelines I prefer to partition my test suite based on pages' last execution time and then have each partition be executed in separate docker containers and finally merging the results. Setting that up takes a bit more work :-). A full example (even building the docker containers, ideally you just pull the project's standard images from docker hub) can be found in the projects GitLab pipeline (running tests in 8 partitions and combining the results) is in https://github.com/fhoeben/hsac-fitnesse-fixtures/blob/560e4b346a4c8dd109ca88957808fec1ffe4ca37/.gitlab-ci.yml#L135-L264. The best thing about this approach is that you don't even have to setup, and maintain, a Selenium grid. The test just use a Chrome inside their own docker container (the chrome and chromedriver are locked in the container so no issues with chrome(driver) updates either). A Selenium grid is than only required for cross-browser-platform builds, which probably should use a smaller/less frequent test setup.

So that is what I recommend for a great test setup, but just switching your maven based setup from antrun to jUnit should really not be big deal: one extra Java class, a few changes to pom.xml and a small update of the build job on the CI server.

[aquddos123]

Just worked on some basic setup as you explained. Running into an issue. Here are the steps performed:

1. FixtureDebugTest.java copied to ssimilar location
2. junit dependency added to pom.xml
3. maven-failsafe-plugin copied to pom.xml
4. maven-surefire-plugin copied to pom.xml to exclude this class from normal jUnit test run
5. Make sure all necessary fixtures, plugins, drivers, etc are present under ${project.basedir}/wiki/ folder
6. Executed the following command: mvn clean test-compile failsafe:integration-test -DfitnesseSuiteToRun=HsacExamples.SlimTests.BrowserTests.ScenarioTest -DseleniumGridUrl=http://<username>:<access-key>@ondemand.saucelabs.com:80/wd/hub -DseleniumCapabilities="browserName:chrome, platform:Windows 10, version:latest" Outcome:

[fhoeben]

It looks like you are almost there. I have to admit I haven't tried to connect to a remote Selenium in a while. This seems there is some classpath issue. If you run the tests from the wiki and edit the SuiteSetUp to connect to SauceLabs, does that work (I assume it does but just checking)?

When you run via maven this way and try to connect to local chrome, or firefox, does that work? And if you try to connect the sample project to you Saucelabs grid, does that work? Do you use an IDE where you could run the FixtureDebugTest, does that work?

Either there is something wrong with the fixtures build, and we are missing some jars. Or something else I forgot to mention is needed in your pom.xml to ensure all dependencies are on the classpath when you start the integration tests.

I haven't had any other reports about a problem with the fixtures themselves, so I'm guessing I forgot a step...

[fhoeben]

I just tried to reproduce your problem in my sample project. And even though I have no SauceLabs account and cannot actually run any tests I get an error from them. So I managed to get beyond getting all classes found.

I just noticed you still have maven-classpath-plugin in your project. That is probably the cause of your problem, it has really ancient jars as dependencies and that messes up Selenium.

In this setup you probably don't need it (I stopped bundeling it some time ago). So can you try to remove that plugin from your dependencies and see whether that makes things go better?

If you do need/want it when running the wiki locally you should just copy it into the wiki/plugins that way it will be available when the wiki server is run, but it will not be on the classpath of the slim process or the jUnit run.

[aquddos123]

Sorry for late reply. As far as maven-classpath-plugin is concerned, I agree with you 200% that this is an ancient plugin and that is one of the reasons we are looking for a better solution for future automation, while keeping existing ones up and running. I'll have to exclude/include many jars in my POM to deal with such conflicts and they are working fine so far. Actually, the above snippet is not a full picture of my POM and I apologize for that.

I've also tried using maven-classpath-plugin as plugin instead of maven dependency but that didn't work. That said, I am one step more closer to adopting jUnit config instead of Antrun config as you described.

So, I started with a very fresh project using archetype nl.hsac:fitnesse-project and good news is that I am able to pass through the issue reported earlier. While doing that, I came across few issues/questions:

1. Issue: I've used latest archetype (v1.7.0) to create a sample project. The webdrivers executables it downloads by default are very old therefore not working with latest Chrome and Edge browsers. I'll have to copy over new webdriver exe's to make it working. Probably, the archetype code may need to be revisited.
2. I've tried using tag based filters both as system property at CLI and annotation in the FixtureDebugTest but fail. Please provide some examples on the filters use. See below what I tried: mvn clean test-compile failsafe:integration-test -DfitnesseSuiteToRun=HsacExamples.SlimTests.BrowserTests -DsuiteFilter="?responder=suite&runTestsMatchingAllTags=smoke"

   @FitNesseRunner.SuiteFilter("runTestsMatchingAllTags=smoke")
   @FitNesseRunner.Suite("HsacExamples.SlimTests.BrowserTests")

   @FitNesseRunner.SuiteFilter("?responder=suite&runTestsMatchingAllTags=smoke")
   @FitNesseRunner.Suite("HsacExamples.SlimTests.BrowserTests")

3. Question 2: Consider I have a project created using archetype nl.hsac:fitnesse-project and I have Docker Desktop installed on my local machine. Then I tried copying and running buildChrome.sh from my local project but no luck. I've also copied https://github.com/fhoeben/hsac-fitnesse-docker/tree/master/test folder into my local project and tried building and running my own docker image but no luck. It would be much appreciated if you can explain the steps to configure and run some tests from my local project inside hsac/fitnesse-fixtures-test-jre8-chrome container.
4. If I have my own custom code for different tests, what is the best way to package and install in the classpath? Currently, if I use mvn clean compile dependency:copy-dependencies exec:exec it is not package my code at all. And if I use mvn clean install with the same command, it is running integration tests first and then start fitnesse in wiki mode. I have some ideas of using maven-jar-plugin to package and install test-jar but I don't want to go off track with this project so please advise. The scenario is, in addition to hsac-fixtures we would be writing our own fixtures to support some decision and other custom script tables and we would need to recompile and install our code every time we make some changes.

[fhoeben]

I’m sorry I’m off on holiday at the moment so I don’t have access to my dev environment. Some initial thoughts on your issues:

1. You are right I don’t actively maintain the archetype. It’s very similar to the sample project and I personally prefer a simple copy paste to the magic of an archetype.
2. I don’t remember exactly what the correct config was. But I’m sure you don’t need a responder. Have you tried just using ‘smoke’ as annotation value?
3. I have a separate docker project. I believe its readme has some extra info
4. This surprises me. I would expect both the sample project or archetype to give you a pom that allows you to use your own classes too. They would not be in a jar, but their class files should be on the wiki class path. When running with junit runner there are no 2 jvms slim, that’s what makes debugging much easier

[aquddos123]

Ok, for #2 I've tried your suggestion but it didn't work. I think I'll come back to it later. And for # 4 I'll have to tweak maven phases between exec-maven and maven-failsafe plugins to make it working and it is working fine now. Recently I've been working on containerizing my test builds and got some success but have some questions too.

First of all, I am able to use both hsac/fitnesse-fixtures-test-jre8 and hsac/fitnesse-fixtures-test-jre8-chrome images to execute UI tests from BrowserTests BUT on BrowserStack hub. When I tried running BrowserTests in container's local chrome it crashing right away. The error log looks something like below. I googled the issue and tried running container in normal user instead sudo but no luck. Also tried argument no-sandbox as -DseleniumJsonProfile="{'args': ['disable-dev-shm-usage'],['no-sandbox']}" but no luck.

Aug 10, 2022 9:51:39 PM nl.hsac.fitnesse.HsacPluginFeatureFactory registerSlimCoverageIfPossible
INFO: Registered test system slimcoverage
2022-08-10 21:51:39.574 testRun Started ('4' tests)
slim:in-process

2022-08-10 21:51:39.623 testSuite Started 'nl.hsac.fitnesse.HsacFitNesseSuiteStarter'
2022-08-10 21:51:39.623 test Started 'HsacExamples.SlimTests.SuiteSetUp' (1 / 4)
2022-08-10 21:51:39.677 test Ignored 'HsacExamples.SlimTests.SuiteSetUp'
2022-08-10 21:51:39.677 test Started 'HsacExamples.SlimTests.BrowserTests.SuiteSetUp' (2 / 4)
Starting ChromeDriver 103.0.5060.134 (8ec6fce403b3feb0869b0732eda8bd95011d333c-refs/branch-heads/5060@{#1262}) on port 56434
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations f[1660168300.093][SEVEREor suggestions on] :k ebeipnidn(g)  failed: CCannhorto maesDsriigvne rr esqaufees.t
ChromeeDridv eard dwraess ss t(a9r9t)e
d successfully.
2022-08-10 21:51:40.467 test Failed 'HsacExamples.SlimTests.BrowserTests.SuiteSetUp' message='failed' details='java.lang.AssertionError: ABORT_SLIM_TEST:nl.hsac.fitnesse.fixture.slim.StopTestException: Could not start a new session. Response code 500. Message: unknown error: Chrome failed to start: exited abnormally.
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Build info: version: '4.1.3', revision: '7b1ebf28ef'
System info: host: '5fd3770b00fc', ip: '172.17.0.2', os.name: 'Linux', os.arch: 'amd64', os.version: '5.10.102.1-microsoft-standard-WSL2', java.version: '11.0.15'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Command: [null, newSession {capabilities=[Capabilities {browserName: chrome, goog:chromeOptions: {args: [], extensions: []}, unhandledPromptBehavior: ignore}], desiredCapabilities=Capabilities {browserName: chrome, goog:chromeOptions: {args: [], extensions: []}, unexpectedAlertBehaviour: ignore, unhandledPromptBehavior: ignore}}]
        at nl.hsac.fitnesse.fixture.util.selenium.driverfactory.DriverManager.getSeleniumHelper(DriverManager.java:52) [hsac-fitnesse-fixtures-5.2.13-SNAPSHOT.jar]
        at nl.hsac.fitnesse.fixture.slim.web.SeleniumDriverSetup.setAndUseDriverFactory(SeleniumDriverSetup.java:288) [hsac-fitnesse-fixtures-5.2.13-SNAPSHOT.jar]
        at nl.hsac.fitnesse.fixture.slim.web.SeleniumDriverSetup.startDriverForWithProfile(SeleniumDriverSetup.java:100) [hsac-fitnesse-fixtures-5.2.13-SNAPSHOT.jar]
        at nl.hsac.fitnesse.fixture.slim.web.SeleniumDriverSetup.startDriverFor(SeleniumDriverSetup.java:105) [hsac-fitnesse-fixtures-5.2.13-SNAPSHOT.jar]
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [n/a]
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [n/a] 

Here are steps to reproduce error:

1- This is my Dockerfile i use to build a custom image of mine because I have my own FitNesseRoot and some other custom fixtures to add

ARG  TEST_IMAGE=hsac/fitnesse-fixtures-test-jre8-chrome:5.2.12
ARG  SELENIUM_VERSION=4.1.3
FROM ${TEST_IMAGE} as hsac-fixtures

FROM selenium/standalone-chrome:${SELENIUM_VERSION}
RUN sudo mv /etc/supervisor/conf.d/selenium.conf /etc/supervisor/conf.d/selenium.conf.bak && \
    sudo mkdir -p /fitnesse/target && \
    sudo mkdir -p /fitnesse/wiki/webdrivers && \
    sudo ln -s /usr/bin/chromedriver /fitnesse/wiki/webdrivers/chromedriver-linux-64bit && \
    sudo chown -R 1200:1201 /fitnesse/

WORKDIR /fitnesse
COPY startGridAndRunTests.sh .
COPY fitnesse-hsac.conf /etc/supervisor/conf.d/
COPY wiki/FitNesseRoot wiki/FitNesseRoot
COPY wiki/fixtures/<somefixtures> wiki/fixtures 

VOLUME /fitnesse/target
VOLUME /fitnesse/wiki/FitNesseRoot

ENV FITNESSE_OPTS -Djava.security.egd=file:/dev/./urandom -Djava.awt.headless=true \
    -DseleniumBrowser=chrome \
    -DseleniumJsonProfile="{'args':['disable-dev-shm-usage']}"

#ENTRYPOINT ["/fitnesse/startGridAndRunTests.sh"]
#CMD []

COPY --from=hsac-fixtures --chown=1200:1201 /fitnesse /fitnesse
COPY wiki/fixtures/* wiki/fixtures

RUN sudo apt-get update && \
    sudo apt-get install -y vim

2- After building new image, I am running image by overriding the ENTRYPOINT to access container console for R&D purposes. Here is the command:

docker run -u root -it --entrypoint=/bin/bash -v C:\dev\code\test\results:/fitnesse/target/ hsac-chrome docker run -it --entrypoint=/bin/bash -v C:\dev\code\test\results:/fitnesse/target/ hsac-chrome

Here is how my container looks like:

3- Once I am inside container, I'm running the following commands and getting the error mentioned above. Full error log is captured below:

java -DseleniumBrowser=chrome -DfitnesseSuiteToRun=HsacExamples.SlimTests.BrowserTests.ScenarioTest -cp wiki/fixtures:wiki/fixtures/* nl.hsac.fitnesse.junit.JUnitConsoleRunner nl.hsac.fitnesse.HsacFitNesseSuiteStarter

java -DfitnesseSuiteToRun=HsacExamples.SlimTests.BrowserTests.ScenarioTest -cp wiki/fixtures:wiki/fixtures/* nl.hsac.fitnesse.junit.JUnitConsoleRunner nl.hsac.fitnesse.HsacFitNesseSuiteStarter

I've also made sure the chromedriver-linux-64bit.exe is present under wiki/webdrivers folder in both image containers

Also note that the following command is running tests fine on BrowserStack hub and it is overriding Selenium configuration successfully. java "-DseleniumCapabilities=browserName:safari, platform:OS X, version:latest" -DseleniumGridUrl=https://<user>:<key>@hub-cloud.browserstack.com/wd/hub -DfitnesseSuiteToRun=HsacExamples.SlimTests.BrowserTests -cp wiki/fixtures:wiki/fixtures/* nl.hsac.fitnesse.junit.JUnitConsoleRunner nl.hsac.fitnesse.HsacFitNesseSuiteStarter

4- I've after tried running startGridAndRunTests.sh script inside container but it does nothing, just cursor blinking forever.

Here is the full error log:

root@5fd3770b00fc:/fitnesse# java -DseleniumBrowser=chrome -DfitnesseSuiteToRun=HsacExamples.SlimTests.BrowserTests.ScenarioTest -cp wiki/fixtures:wiki/fixtures/* nl.hsac.fitnesse.junit.JUnitConsoleRunner nl.hsac.fitnesse.HsacFitNesseSuiteStarter
Aug 10, 2022 9:49:57 PM fitnesse.ConfigurationParameter loadProperties
INFO: No configuration file found (/fitnesse/wiki/plugins.properties)
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.MonthsFromToday
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.YearsFromToday
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.WeekDaysFromToday
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.DayOfWeek$Monday
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.DayOfWeek$Tuesday
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.DayOfWeek$Wednesday
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.DayOfWeek$Thursday
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.DayOfWeek$Friday
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.DayOfWeek$Saturday
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.DayOfWeek$Sunday
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.LastDayOfMonth
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.RandomInt
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.RandomString
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.RandomBsn
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.RandomEmail
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.RandomIban
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.RandomPostalCode
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.RandomDutchLicensePlate
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.DefineDefault
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.DefineFromProperties
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.DefineDefaultFromProperties
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.RandomUuid
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.MonthsFromToday
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.YearsFromToday
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.WeekDaysFromToday
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.DayOfWeek$Monday
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.DayOfWeek$Tuesday
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.DayOfWeek$Wednesday
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.DayOfWeek$Thursday
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.DayOfWeek$Friday
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.DayOfWeek$Saturday
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.DayOfWeek$Sunday
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.LastDayOfMonth
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.RandomInt
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.RandomString
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.RandomBsn
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.RandomEmail
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.RandomIban
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.RandomPostalCode
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.RandomDutchLicensePlate
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.DefineDefault
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.DefineFromProperties
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.DefineDefaultFromProperties
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added symbol class nl.hsac.fitnesse.symbols.RandomUuid
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added Slim table type: table template: nl.hsac.fitnesse.slim.AutoArgScenarioTable
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added Slim table type: storyboard: nl.hsac.fitnesse.slim.StoryboardTable
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory add
INFO: Added Slim table type: all arg template: nl.hsac.fitnesse.slim.AllArgScenarioTable
Aug 10, 2022 9:49:57 PM nl.hsac.fitnesse.HsacPluginFeatureFactory registerSlimCoverageIfPossible
INFO: Registered test system slimcoverage
2022-08-10 21:49:57.745 testRun Started ('4' tests)
slim:in-process

2022-08-10 21:49:57.807 testSuite Started 'nl.hsac.fitnesse.HsacFitNesseSuiteStarter'
2022-08-10 21:49:57.807 test Started 'HsacExamples.SlimTests.SuiteSetUp' (1 / 4)
2022-08-10 21:49:57.857 test Ignored 'HsacExamples.SlimTests.SuiteSetUp'
2022-08-10 21:49:57.857 test Started 'HsacExamples.SlimTests.BrowserTests.SuiteSetUp' (2 / 4)
Starting ChromeDriver 103.0.5060.134 (8ec6fce403b3feb0869b0732eda8bd95011d333c-refs/branch-heads/5060@{#1262}) on port 39833
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for sug[ge1s6t6i0o1n6s8 1on keeping ChromeDriver safe.
ChromeDriver was started successfully.
98.211][SEVERE]: bind() failed: Cannot assign requested address (99)
2022-08-10 21:49:58.583 test Failed 'HsacExamples.SlimTests.BrowserTests.SuiteSetUp' message='failed' details='java.lang.AssertionError: ABORT_SLIM_TEST:nl.hsac.fitnesse.fixture.slim.StopTestException: Could not start a new session. Response code 500. Message: unknown error: Chrome failed to start: exited abnormally.
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Build info: version: '4.1.3', revision: '7b1ebf28ef'
System info: host: '5fd3770b00fc', ip: '172.17.0.2', os.name: 'Linux', os.arch: 'amd64', os.version: '5.10.102.1-microsoft-standard-WSL2', java.version: '11.0.15'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Command: [null, newSession {capabilities=[Capabilities {browserName: chrome, goog:chromeOptions: {args: [], extensions: []}, unhandledPromptBehavior: ignore}], desiredCapabilities=Capabilities {browserName: chrome, goog:chromeOptions: {args: [], extensions: []}, unexpectedAlertBehaviour: ignore, unhandledPromptBehavior: ignore}}]
        at nl.hsac.fitnesse.fixture.util.selenium.driverfactory.DriverManager.getSeleniumHelper(DriverManager.java:52) [hsac-fitnesse-fixtures-5.2.13-SNAPSHOT.jar]
        at nl.hsac.fitnesse.fixture.Environment.getSeleniumHelper(Environment.java:645) [hsac-fitnesse-fixtures-5.2.13-SNAPSHOT.jar]
        at nl.hsac.fitnesse.fixture.slim.web.SeleniumDriverSetup.getHelper(SeleniumDriverSetup.java:327) [hsac-fitnesse-fixtures-5.2.13-SNAPSHOT.jar]
        at nl.hsac.fitnesse.fixture.slim.web.SeleniumDriverSetup.driverDescription(SeleniumDriverSetup.java:176) [hsac-fitnesse-fixtures-5.2.13-SNAPSHOT.jar]
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [n/a]
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [n/a]
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [n/a]
        at java.base/java.lang.reflect.Method.invoke(Method.java:566) [rt.jar]
        at fitnesse.slim.fixtureInteraction.SimpleInteraction.methodInvoke(SimpleInteraction.java:322) [fitnesse-20220319.jar:20220319]
        at nl.hsac.fitnesse.fixture.slim.SlimFixture.invoke(SlimFixture.java:65) [hsac-fitnesse-fixtures-5.2.13-SNAPSHOT.jar]
        at nl.hsac.fitnesse.fixture.slim.SlimFixture.aroundSlimInvoke(SlimFixture.java:45) [hsac-fitnesse-fixtures-5.2.13-SNAPSHOT.jar]
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [n/a]
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [n/a]
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [n/a]
        at java.base/java.lang.reflect.Method.invoke(Method.java:566) [rt.jar]
        at fitnesse.slim.fixtureInteraction.SimpleInteraction.methodInvoke(SimpleInteraction.java:322) [fitnesse-20220319.jar:20220319]
        at fitnesse.slim.fixtureInteraction.SimpleInteraction.callMethod(SimpleInteraction.java:305) [fitnesse-20220319.jar:20220319]
        at fitnesse.slim.fixtureInteraction.SimpleInteraction.invokeMethod(SimpleInteraction.java:290) [fitnesse-20220319.jar:20220319]
        at fitnesse.slim.fixtureInteraction.SimpleInteraction.findAndInvoke(SimpleInteraction.java:217) [fitnesse-20220319.jar:20220319]
        at fitnesse.junit.JUnitRunNotifierResultsListener.firstFailure(JUnitRunNotifierResultsListener.java:175)
        at fitnesse.junit.JUnitRunNotifierResultsListener.testExceptionOccurred(JUnitRunNotifierResultsListener.java:97)
        at fitnesse.testsystems.CompositeTestSystemListener.testExceptionOccurred(CompositeTestSystemListener.java:61)
        at fitnesse.testrunner.MultipleTestsRunner$InternalTestSystemListener.testExceptionOccurred(MultipleTestsRunner.java:225)
        at fitnesse.testsystems.CompositeTestSystemListener.testExceptionOccurred(CompositeTestSystemListener.java:61)
        at fitnesse.testsystems.slim.SlimTestSystem.testExceptionOccurred(SlimTestSystem.java:201)
        at fitnesse.testsystems.slim.SlimTestSystem.evaluateTables(SlimTestSystem.java:151)
        at fitnesse.testsystems.slim.SlimTestSystem.processTable(SlimTestSystem.java:131)
        at fitnesse.testsystems.slim.HtmlSlimTestSystem.processAllTablesOnPage(HtmlSlimTestSystem.java:48)
        at fitnesse.testsystems.slim.SlimTestSystem.runTests(SlimTestSystem.java:93)
        at fitnesse.testrunner.run.PerTestSystemTestRun.executeTestSystemPages(PerTestSystemTestRun.java:64)
        at fitnesse.testrunner.run.PerTestSystemTestRun.startTestSystemAndExecutePages(PerTestSystemTestRun.java:47)
        at fitnesse.testrunner.run.PerTestSystemTestRun.executeTestPages(PerTestSystemTestRun.java:30)
        at fitnesse.testrunner.MultipleTestsRunner.executeTestPages(MultipleTestsRunner.java:67)
        at fitnesse.junit.FitNesseRunner.executeTests(FitNesseRunner.java:580)
        at fitnesse.junit.FitNesseRunner.runPages(FitNesseRunner.java:497)
        at nl.hsac.fitnesse.junit.HsacFitNesseRunner.runPages(HsacFitNesseRunner.java:216)
        at fitnesse.junit.FitNesseRunner.run(FitNesseRunner.java:478)
        at org.junit.runners.Suite.runChild(Suite.java:128)
        at org.junit.runners.Suite.runChild(Suite.java:27)
        at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
        at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
        at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:105)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:94)
        at nl.hsac.fitnesse.junit.JUnitConsoleRunner.runTests(JUnitConsoleRunner.java:38)
        at nl.hsac.fitnesse.junit.JUnitConsoleRunner.main(JUnitConsoleRunner.java:46)
'
2022-08-10 21:49:58.583 test Finished 'HsacExamples.SlimTests.BrowserTests.SuiteSetUp'

2022-08-10 21:49:58.584 test Finished 'HsacExamples.SlimTests.BrowserTests.SuiteSetUp'

2022-08-10 21:49:58.584 test Started 'HsacExamples.SlimTests.BrowserTests.ScenarioTest' (3 / 4)
2022-08-10 21:49:58.622 test Ignored 'HsacExamples.SlimTests.BrowserTests.ScenarioTest'
2022-08-10 21:49:58.623 test Started 'HsacExamples.SlimTests.BrowserTests.SuiteTearDown' (4 / 4)
Starting ChromeDriver 103.0.5060.134 (8ec6fce403b3feb0869b0732eda8bd95011d333c-refs/branch-heads/5060@{#1262}) on port 54545
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
[1660168198.636][SEVERE]: bind() failed: Cannot assign requested address (99)
2022-08-10 21:49:58.738 test Failed 'HsacExamples.SlimTests.BrowserTests.SuiteTearDown' message='failed' details='java.lang.AssertionError: ABORT_SLIM_TEST:nl.hsac.fitnesse.fixture.slim.StopTestException: Could not start a new session. Response code 500. Message: unknown error: Chrome failed to start: exited abnormally.
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Build info: version: '4.1.3', revision: '7b1ebf28ef'
System info: host: '5fd3770b00fc', ip: '172.17.0.2', os.name: 'Linux', os.arch: 'amd64', os.version: '5.10.102.1-microsoft-standard-WSL2', java.version: '11.0.15'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Command: [null, newSession {capabilities=[Capabilities {browserName: chrome, goog:chromeOptions: {args: [], extensions: []}, unhandledPromptBehavior: ignore}], desiredCapabilities=Capabilities {browserName: chrome, goog:chromeOptions: {args: [], extensions: []}, unexpectedAlertBehaviour: ignore, unhandledPromptBehavior: ignore}}]
        at nl.hsac.fitnesse.fixture.util.selenium.driverfactory.DriverManager.getSeleniumHelper(DriverManager.java:52) [hsac-fitnesse-fixtures-5.2.13-SNAPSHOT.jar]
        at nl.hsac.fitnesse.fixture.Environment.getSeleniumHelper(Environment.java:645) [hsac-fitnesse-fixtures-5.2.13-SNAPSHOT.jar]
        at nl.hsac.fitnesse.fixture.slim.web.SeleniumDriverSetup.getHelper(SeleniumDriverSetup.java:327) [hsac-fitnesse-fixtures-5.2.13-SNAPSHOT.jar]
        at nl.hsac.fitnesse.fixture.slim.web.SeleniumDriverSetup.runSummary(SeleniumDriverSetup.java:203) [hsac-fitnesse-fixtures-5.2.13-SNAPSHOT.jar]
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [n/a]
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [n/a]
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [n/a]
        at java.base/java.lang.reflect.Method.invoke(Method.java:566) [rt.jar]
        at fitnesse.slim.fixtureInteraction.SimpleInteraction.methodInvoke(SimpleInteraction.java:322) [fitnesse-20220319.jar:20220319]
        at nl.hsac.fitnesse.fixture.slim.SlimFixture.invoke(SlimFixture.java:65) [hsac-fitnesse-fixtures-5.2.13-SNAPSHOT.jar]
        at nl.hsac.fitnesse.fixture.slim.SlimFixture.aroundSlimInvoke(SlimFixture.java:45) [hsac-fitnesse-fixtures-5.2.13-SNAPSHOT.jar]
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [n/a]
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [n/a]
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [n/a]
        at java.base/java.lang.reflect.Method.invoke(Method.java:566) [rt.jar]
        at fitnesse.slim.fixtureInteraction.SimpleInteraction.methodInvoke(SimpleInteraction.java:322) [fitnesse-20220319.jar:20220319]
        at fitnesse.slim.fixtureInteraction.SimpleInteraction.callMethod(SimpleInteraction.java:305) [fitnesse-20220319.jar:20220319]
        at fitnesse.slim.fixtureInteraction.SimpleInteraction.invokeMethod(SimpleInteraction.java:290) [fitnesse-20220319.jar:20220319]
        at fitnesse.slim.fixtureInteraction.SimpleInteraction.findAndInvoke(SimpleInteraction.java:217) [fitnesse-20220319.jar:20220319]
        at fitnesse.junit.JUnitRunNotifierResultsListener.firstFailure(JUnitRunNotifierResultsListener.java:175)
        at fitnesse.junit.JUnitRunNotifierResultsListener.testExceptionOccurred(JUnitRunNotifierResultsListener.java:97)
        at fitnesse.testsystems.CompositeTestSystemListener.testExceptionOccurred(CompositeTestSystemListener.java:61)
        at fitnesse.testrunner.MultipleTestsRunner$InternalTestSystemListener.testExceptionOccurred(MultipleTestsRunner.java:225)
        at fitnesse.testsystems.CompositeTestSystemListener.testExceptionOccurred(CompositeTestSystemListener.java:61)
        at fitnesse.testsystems.slim.SlimTestSystem.testExceptionOccurred(SlimTestSystem.java:201)
        at fitnesse.testsystems.slim.SlimTestSystem.evaluateTables(SlimTestSystem.java:151)
        at fitnesse.testsystems.slim.SlimTestSystem.processTable(SlimTestSystem.java:131)
        at fitnesse.testsystems.slim.HtmlSlimTestSystem.processAllTablesOnPage(HtmlSlimTestSystem.java:48)
        at fitnesse.testsystems.slim.SlimTestSystem.runTests(SlimTestSystem.java:93)
        at fitnesse.testrunner.run.PerTestSystemTestRun.executeTestSystemPages(PerTestSystemTestRun.java:64)
        at fitnesse.testrunner.run.PerTestSystemTestRun.startTestSystemAndExecutePages(PerTestSystemTestRun.java:47)
        at fitnesse.testrunner.run.PerTestSystemTestRun.executeTestPages(PerTestSystemTestRun.java:30)
        at fitnesse.testrunner.MultipleTestsRunner.executeTestPages(MultipleTestsRunner.java:67)
        at fitnesse.junit.FitNesseRunner.executeTests(FitNesseRunner.java:580)
        at fitnesse.junit.FitNesseRunner.runPages(FitNesseRunner.java:497)
        at nl.hsac.fitnesse.junit.HsacFitNesseRunner.runPages(HsacFitNesseRunner.java:216)
        at fitnesse.junit.FitNesseRunner.run(FitNesseRunner.java:478)
        at org.junit.runners.Suite.runChild(Suite.java:128)
        at org.junit.runners.Suite.runChild(Suite.java:27)
        at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
        at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
        at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:105)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:94)
        at nl.hsac.fitnesse.junit.JUnitConsoleRunner.runTests(JUnitConsoleRunner.java:38)
        at nl.hsac.fitnesse.junit.JUnitConsoleRunner.main(JUnitConsoleRunner.java:46)
'
2022-08-10 21:49:58.739 test Finished 'HsacExamples.SlimTests.BrowserTests.SuiteTearDown'

2022-08-10 21:49:58.739 test Finished 'HsacExamples.SlimTests.BrowserTests.SuiteTearDown'

Starting ChromeDriver 103.0.5060.134 (8ec6fce403b3feb0869b0732eda8bd95011d333c-refs/branch-heads/5060@{#1262}) on port 57119
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
[1660168198.752][SEVERE]: bind() failed: Cannot assign requested address (99)
Error shutting down selenium
nl.hsac.fitnesse.fixture.slim.StopTestException: Could not start a new session. Response code 500. Message: unknown error: Chrome failed to start: exited abnormally.
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Build info: version: '4.1.3', revision: '7b1ebf28ef'
System info: host: '5fd3770b00fc', ip: '172.17.0.2', os.name: 'Linux', os.arch: 'amd64', os.version: '5.10.102.1-microsoft-standard-WSL2', java.version: '11.0.15'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Command: [null, newSession {capabilities=[Capabilities {browserName: chrome, goog:chromeOptions: {args: [], extensions: []}, unhandledPromptBehavior: ignore}], desiredCapabilities=Capabilities {browserName: chrome, goog:chromeOptions: {args: [], extensions: []}, unexpectedAlertBehaviour: ignore, unhandledPromptBehavior: ignore}}]
        at nl.hsac.fitnesse.fixture.util.selenium.driverfactory.DriverManager.getSeleniumHelper(DriverManager.java:52)
        at nl.hsac.fitnesse.fixture.Environment.getSeleniumHelper(Environment.java:645)
        at nl.hsac.fitnesse.fixture.slim.web.SeleniumDriverSetup.getHelper(SeleniumDriverSetup.java:327)
        at nl.hsac.fitnesse.fixture.slim.web.SeleniumDriverSetup.runSummary(SeleniumDriverSetup.java:203)
        at nl.hsac.fitnesse.fixture.slim.web.SeleniumDriverSetup.stopDriver(SeleniumDriverSetup.java:302)
        at nl.hsac.fitnesse.junit.HsacFitNesseRunner.shutdownSelenium(HsacFitNesseRunner.java:349)
        at nl.hsac.fitnesse.junit.HsacFitNesseRunner.runPages(HsacFitNesseRunner.java:220)
        at fitnesse.junit.FitNesseRunner.run(FitNesseRunner.java:478)
        at org.junit.runners.Suite.runChild(Suite.java:128)
        at org.junit.runners.Suite.runChild(Suite.java:27)
        at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
        at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
        at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:105)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:94)
        at nl.hsac.fitnesse.junit.JUnitConsoleRunner.runTests(JUnitConsoleRunner.java:38)
        at nl.hsac.fitnesse.junit.JUnitConsoleRunner.main(JUnitConsoleRunner.java:46)
Caused by: org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Response code 500. Message: unknown error: Chrome failed to start: exited abnormally.
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Build info: version: '4.1.3', revision: '7b1ebf28ef'
System info: host: '5fd3770b00fc', ip: '172.17.0.2', os.name: 'Linux', os.arch: 'amd64', os.version: '5.10.102.1-microsoft-standard-WSL2', java.version: '11.0.15'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Command: [null, newSession {capabilities=[Capabilities {browserName: chrome, goog:chromeOptions: {args: [], extensions: []}, unhandledPromptBehavior: ignore}], desiredCapabilities=Capabilities {browserName: chrome, goog:chromeOptions: {args: [], extensions: []}, unexpectedAlertBehaviour: ignore, unhandledPromptBehavior: ignore}}]
        at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:126)
        at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:84)
        at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:62)
        at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:156)
        at org.openqa.selenium.remote.service.DriverCommandExecutor.invokeExecute(DriverCommandExecutor.java:167)
        at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:142)
        at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:567)
        at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:248)
        at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:164)
        at org.openqa.selenium.chromium.ChromiumDriver.<init>(ChromiumDriver.java:108)
        at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:106)
        at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:93)
        at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:82)
        at nl.hsac.fitnesse.fixture.util.selenium.driverfactory.LocalDriverFactory.createDriver(LocalDriverFactory.java:54)
        at nl.hsac.fitnesse.fixture.util.selenium.driverfactory.DriverManager.getSeleniumHelper(DriverManager.java:46)
        ... 22 more
2022-08-10 21:49:58.906 testSuite Finished 'nl.hsac.fitnesse.HsacFitNesseSuiteStarter'

Any idea what I am doing wrong when running tests in local chrome on containers?

UPDATE: After googling it more, I realised this is a common isssue with headless chrome. Here is one blog explaining the issue I think this is a common issue as mentioned in blog https://bugs.chromium.org/p/chromedriver/issues/detail?id=2473. Question is, did it ever worked with the hsac sample project? Any workaround?

[fhoeben]

I believe the images work. I use very similar images in a GitLab pipeline that is run for every build and that is passing. BUT the example suite for BrowserTest is quite flaky the 'allmusic' site used in the tests often gives problems so the tests fail also. The acceptance tests also use the docker images with chrome, in headless and those tasks/jobs run without problem, see for instance https://gitlab.com/fhoeben/hsac-fitnesse-fixtures/-/jobs/2827401389 To verify whether your images are correct I would recommend using those tests (i.e. HsacAcceptanceTests.SlimTests.BrowserTest) instead of the example tests.

I'm not sure what you mean by:

First of all, I am able to use both hsac/fitnesse-fixtures-test-jre8 and hsac/fitnesse-fixtures-test-jre8-chrome images to execute UI tests from BrowserTests BUT on BrowserStack hub. When I tried running BrowserTests in container's local chrome it crashing right away. The error log looks something like below.

Do you mean that when running docker on your local machine you can use the standard images with BrowserStack, but not with local chrome? Unfortunately I'm no longer able to easily reproduce any of this locally as my Apple Mac is equipped with an M1 processor (i.e. an 'arm64' processor) and the selenium chromedriver image is for 'amd64' processors and I have not been able to run that image properly using emulation. Is that maybe a problem you are also facing?

If you are running docker on a 'amd64' machine, it seems to me there is something wrong with the custom images you create which prevents chrome from being launched. I suspect it might have something to do with the fact that you copy the files from my image and there might be some permission issue or the fact that my image defines the directories as volumes and you provide the content in the image. Also the fact that you do apt-get and the very end of the Dockerfile seems a bad idea to me. This makes the images non-reproducible as you change the executables in your images for every build instead of creating a stable base. (Maybe the chrome is updated to a version that is not compatible with the bundled chromedriver?)

Are you able to run the acceptance tests using the standard images I uploaded to Dockerhub (if you just add the wiki files for those tests to the image)?

I'm also not 100% clear why you create a Dockerfile that basically copies the Dockerfile I use with only minimal changes. I would think it would be more logical to just use my image, copy the compiled fixture classes you need and mount the FitNesseRoot you need as a volume (or copy the files into the container). Also the entry point and command you should be able to override from my image.

When you copy files into the container/image it is important to use the --chown=1200:1201 flag to ensure the files are readable by the user that will actually be running the entrypoint/command.

As a side note: In my actual test projects I do not always create custom images for the fitnesse jobs. Sometimes I feel it is easier just to have my pipeline-job first create a docker container from the standard image, copy any needed extra files into it and only then start it (so instead of using a single docker run command I have multiple lines: first a docker create, then a docker cp and then a docker start). This is especially nice if you have a test suite that needs no custom fixtures, or you just want to have a single test execution with the code/test and there no need to create an image to be used multiple times (and that might need to be cleaned up after the run).

[fhoeben]

I just managed to update the images to work for me locally (on my Mac M1/arm64). I pushed the changes to a branch of the sample docker project: `wip/arm64

I also pushed to resulting images to docker hub (with tags ...-arm64), e.g. 'hsac/fitnesse-fixtures-test-jre8-chrome:5.2.12-arm64'

[fhoeben]

Did you ever make any headway fixing your setup? As I stated above the images work in our gitlab pipeline, and I haven't had any other reports of the docker images not working

[aquddos123]

Hi @fhoeben I am so sorry I got busy with some other automation work, but I promise I'll try it one more time and get back to you in a day or two.

[aquddos123]

Are you able to run the acceptance tests using the standard images I uploaded to Dockerhub (if you just add the wiki files for those tests to the image)?

I'm also not 100% clear why you create a Dockerfile that basically copies the Dockerfile I use with only minimal changes. I would think it would be more logical to just use my image, copy the compiled fixture classes you need and mount the FitNesseRoot you need as a volume (or copy the files into the container). Also the entry point and command you should be able to override from my image.

Hi Fried, As you mentioned above, I've tried pulling your latest image from docker hub but no luck. Please note that I am running docker desktop on my local Windows 11 64 bit machine (Intel Core i7 x64 based processor). Using locally downloaded image, I am able to run tests with BrowserStack, but not with container's local chrome (in both headless and non-headless modes).

A few things before getting into steps:

1. please ignore if I am using example tests from BrowserTests or HsacAcceptanceTests. The problem is not with tests but with the chrome browser which is either not initiating in some cases and in some cases crashing right away.
2. most of the steps I am doing for this R&D are manual. There are no automated CI jobs configured yet.
3. At this time, I am not building any new image of mine. I am just pulling your latest image from docker hub, starting a container by overriding its entrypoint, and then executing tests manually in the container terminal.

Steps to reproduce:

1. Pull docker latest image using command docker pull hsac/fitnesse-fixtures-test-jre8-chrome
2. Start container with access to its terminal. docker run -it --entrypoint=/bin/bash -v C:/dev/code/fitnesse-test/wiki/FitNesseRoot:/fitnesse/wiki/FitNesseRoot -v C:/dev/code/fitnesse-test/target/results:/fitnesse/target hsac/fitnesse-fixtures-test-jre8-chrome At this time the docker desktop and running container looks like below:
3. Next, I am running an example test from container terminal in headless Chrome. See screenshot for error. java -DseleniumBrowser=chrome -DfitnesseSuiteToRun=HsacAcceptanceTests.SlimTests.BrowserTest.ClickOnLinkTest -cp wiki/fixtures:wiki/fixtures/* nl.hsac.fitnesse.junit.JUnitConsoleRunner nl.hsac.fitnesse.HsacFitNesseSuiteStarter
4. Then, I tried running another test from container terminal in a non-headless chrome mode. See screenshot for error. java -DseleniumBrowser=chrome -DfitnesseSuiteToRun=HsacExamples.SlimTests.BrowserTests.ScenarioTest -cp wiki/fixtures:wiki/fixtures/* nl.hsac.fitnesse.junit.JUnitConsoleRunner nl.hsac.fitnesse.HsacFitNesseSuiteStarter
5. Then, I ran similar test with BrowserStack in a non-headless Chrome and test worked fine. java "-DseleniumCapabilities=browserName:chrome, platform:Windows, version:latest" -DseleniumGridUrl=https://USER:KEY@hub-cloud.browserstack.com/wd/hub -DfitnesseSuiteToRun=HsacExamples.SlimTests.BrowserTests.ScenarioTest -cp wiki/fixtures:wiki/fixtures/* nl.hsac.fitnesse.junit.JUnitConsoleRunner nl.hsac.fitnesse.HsacFitNesseSuiteStarter

I've also tried a workaround --shm-size="2g" to avoid the browser crashing in the docker containers but no luck. docker run -it --entrypoint=/bin/bash --shm-size="2g" -v C:/dev/code/yc-test-advance/wiki/FitNesseRoot:/fitnesse/wiki/FitNesseRoot -v C:/dev/code/yc-test-advance/target/tests:/fitnesse/target hsac/fitnesse-fixtures-test-jre8-chrome

I've also tried setting up some additional Chrome options which are discussed as the cause of crashing Chrome in the docker containers but no luck

[fhoeben]

@aquddos123 First of all thanks for the extensive report of what you tried and encountered. I agree that something is wrong with the overall chrome setup. The problem does not seem related to the specific test as far as I can see.

So let's focus on your first example and see if we can get to work.

First let me point out that supplying selenium options on the command line overrides all setup in the .wiki files. So whatever is configured in the SuiteSetUp page is irrelevant. So in your first attempt you are not actually starting a headless but a normal chrome. On Mac and Linux I've seen that work but maybe it's not supported on Windows?

One thing you did not describe: how much memory can the docker daemon use? I don't know how it is setup on Windows, but on my Mac I can configure the docker daemon with memory (and CPU limits). I know chrome is quite a memory hungry application so lacking memory is something that I would imagine could make it crash. I currently have my docker setup in such a way to allow 6GB of memory.

You are now starting the tests from an interactive prompt, and I believe that should work and you manual command does not seem very strange, but have you tried the shell script I provide as part of the container? This is based on what I do in the gitlab pipeline from inside the container:

export JAVA_OPTS="-Xmx96m -Xms96m"
export FITNESSE_OPTS="-Djava.security.egd=file:/dev/./urandom -Djava.awt.headless=true -Xmx128m"
export SUITE=HsacAcceptanceTests.SlimTests.BrowserTest.ClickOnLinkTest
./fitnesse/startGridAndRunTests.sh -DfitnesseSuiteToRun=${SUITE} -DseleniumBrowser=chrome -DseleniumJsonProfile="{'args':['disable-dev-shm-usage']}"

Please note that the chrome options are specified as seleniumJsonProfile, what is in the wiki is ignored.

I've tried to add an option there to prevent shared memory usage, but maybe the --shm-size="2g" option when starting the container makes a difference?

Then if we would want to try headless I thing we need something like:

export JAVA_OPTS="-Xmx96m -Xms96m"
export FITNESSE_OPTS="-Djava.security.egd=file:/dev/./urandom -Djava.awt.headless=true -Xmx128m"
export SUITE=HsacAcceptanceTests.SlimTests.BrowserTest.ClickOnLinkTest
./fitnesse/startGridAndRunTests.sh -DfitnesseSuiteToRun=${SUITE} -DseleniumBrowser=chrome -DseleniumJsonProfile="{'args':['headless','disable-gpu','disable-dev-shm-usage']}"

And adding no-sandbox:

export JAVA_OPTS="-Xmx96m -Xms96m"
export FITNESSE_OPTS="-Djava.security.egd=file:/dev/./urandom -Djava.awt.headless=true -Xmx128m"
export SUITE=HsacAcceptanceTests.SlimTests.BrowserTest.ClickOnLinkTest
./fitnesse/startGridAndRunTests.sh -DfitnesseSuiteToRun=${SUITE} -DseleniumBrowser=chrome -DseleniumJsonProfile="{'args':['no-sandbox','headless','disable-gpu','disable-dev-shm-usage']}"

Do you still get the same error?

[aquddos123]

Fried, Thank you for highlighting the two issues (1) selenium options in the command line and (2) memory issue with container. So, I provided --shm-size="2g" at container startup, and then removed selenium options from command line and tried tests with wiki's setting (headless chrome) and it worked fine. As of now my own tests and test from HsacAcceptanceTests are working fine but browser is still crashing for HsacExamples which I am not too worried about. I also noticed the headless chrome is running on a very low resolution and we can see in some error screenshots. How can we configure high resolution in your fixtures?

I looked at docker daemon config and found defaultKeepStorage set to 20GB by default. I am not sure if this is what you were mentioning. I'll have to explore this further with our DevOps.

I've also tried your script below but there is some issue with shell scripts in Windows command prompt and my GitBash is also not working fine with Docker so I'll have to fix those issues first. I'm going to try all this on MacBook and see the difference.

export JAVA_OPTS="-Xmx96m -Xms96m"
export FITNESSE_OPTS="-Djava.security.egd=file:/dev/./urandom -Djava.awt.headless=true -Xmx128m"
export SUITE=HsacAcceptanceTests.SlimTests.BrowserTest.ClickOnLinkTest
. startGridAndRunTests.sh -DfitnesseSuiteToRun=${SUITE} -DseleniumBrowser=chrome -DseleniumJsonProfile="{'args':['headless','disable-gpu','disable-dev-shm-usage']}"

By the way, have you tried making your fixtures compatible with Selenium images for different browsers like https://hub.docker.com/r/selenium/node-chrome If not, I would request to please consider this as a new feature to this project because that will bring a lot more scalability and flexibility to your project. Thanks.

[fhoeben]

Regarding the docker config I meant one tab higher: 'Resources' When you select that you can indicate the amount of memory

The errors you see when trying to use startGridAndRunTests.sh to me suggest like the script has been given windows newlines (\r\n) instead of regular linux newlines \n. Very strange, given the command you start the container with I would expect you to be in the actual bash of container and not in your Windows GitBash or anything like that.

My chrome image is actually based on Seleniums standalone chrome image (instead of their chrome image to be used as a node in a grid). I see no reason why a similar image could not be made for Firefox and Edge (since the Selenium project also offers standalone images for those). But it is not something I need and at the moment I have other things the keep me busy, but feel free to make one and submit a PR. :-)

If you are using a Selenium grid for scalability my fixtures already support using that (it is essentially the same as BrowserStack you are using). You actually don't need my chrome image when connecting to a hub, you only need the basic test image which is MUCH more lightweight than the chrome image. Using the selenium settings on the command line it should be able to use any Selenium hub. The chrome image adds the browser and chromedriver, but those are not needed when using your own Selenium hub, BrowserStack or another SAAS selenium service.

[aquddos123]

Sorry I meant Selenium's standalone images and not the grid images. Because they have multiple options (chrome, firefox, edge) whereas you only have chrome. I'll give it a try and get back to you. If that works, it'll be more safer option to go for instead of building our own images on top of Selenium standalone images.

So, right now we are in the process of evaluating whether we should go for local headless browser option or third party grid option. Grid option is in the picture because of support for Safari which may be hard to find one pre-configured.

Well thank you so much for your time and support. I am thinking of closing this thread here. We can open a new ticket for future questions and/or issues. Thoughts?

[fhoeben]

One thing to note: Selenium's standalone chrome image (and therefore my chrome image as well) as well as their firefox image is NOT headless by default. They actually run a normal browser, only if you configure it has to be headless (as you did in the end they run truly headless). This is an important thing to consider because running truly headless is different than running normally. In the past I've had issues with running tests headless, so now I only run normally. Headless sometimes behaves differently (especially when it comes to handling user locales, e.g. date pickers, or things like showing PDF objects) and that is just extra noise you don't want to spend time on, you want to focus on testing the actual applications. Running non-headless inside a docker container generally works like a charm (that's one of the things the selenium images have all figured out). Why it didn't work for you I don't know, but my guess is the amount of memory available to the container.

I guess that for running tests on Safari you will need a grid setup, either locally or in the cloud. What I've done in the past is test everything first with chrome in docker images (a CI pipeline that runs for each push to each branch, split the test set over multiple docker containers to speed them up) and only run tests on other browsers if those are fine (only on the master branch, each night). That way you get quick, low cost, feedback that catches > 95% of the problems in my experience.

Firefox should be easy enough to add also via docker images, but in my experience these were always quite a bit slower than the same tests on chrome, and Safari even slower. That's why I always focussed by efforts on chrome.

If you have those you might consider setting up one, or a couple, of mac minis forming a local Selenium grid to run some tests on Safari. I've done that in the past and in my experience is that they are not that hard to setup and maintain. But a cloud service can of course be even more convenient.

And the final word is of course where do you want to spend your effort/money. These days with Edge also using the Chromium engine I find that the vast majority of actual/real users of applications is using that. And web standards support is better than before in all browsers (Safari probably giving the most issues). So if you test for Chrome automatically is the extra investment in automated tests on the other browser really worth it? The cross browser issues I tend to run into are (minor) visualisation differences (scroll bars showing differently, etc) that my automated tests would not catch, because they are more functional than visual. So I end up testing automatically with chrome and have some additional, manual, testing covering other browsers (and some manual testing is always good because for some reason my automated checks sometimes overlook stuff that humans catch right away). That works quite well for me, and costs far less (in time and money) that running all test on all browsers.

Just my 2cts.

I'll close this issue. If other issues pop up just create a new one.

[aquddos123]

Thanks again for detailed reply. I learned a new thing that Selenium's standalone browsers are non-headless. Since I didn't use them yet I was having impression that everything in Linux based containers is headless. So, if that is the case, then I still have issue running tests in a non-headless chrome using your container hsac/fitnesse-fixtures-test-jre8-chrome. It only runs fine when I use the chrome headless profile like below. So as you mentioned it might be the memory issue so I'll work on docker configuration and try again later.

Yes our primary focus is on Chrome based regressions but since we are supporting other browsers as well we need to occasionally run regressions in Edge, Firefox, and Safari.

Once again thank you so much for the help and keep up the good work!!