search

serenity-bdd / serenity-core

Serenity BDD is a test automation library designed to make writing automated acceptance tests easier, and more fun.
http://serenity-bdd.info
Other
720 stars 519 forks source link

Executes tests in parallel failing on Jenkins but passing locally #2907

Closed vayaszsolt closed 1 year ago

vayaszsolt commented 2 years ago

Hi all.

I am writing here maybe I can get some ideas what can be the issue.

I am using serenity with cucumber and spring. The following packages are used by serenity 3.3.2: serenity-core serenity-screenplay serenity-screenplay-webdriver serenity-screenplay-rest serenity-ensure serenity-spring serenity-junit serenity-cucumber

Additional libraries (required for spring): spring-boot-starter-test spring-beans spring-rabbit I am using other other libraries too, but are used for heling during test development.

I configured the tests to be executed in parallel using the maven-failsafe plugin and the documentation from here I use the 3.0.0-M3 failsafe plugin version (otherwise the tests are not triggered to be executed in parallel).

The tests are running in parallel in case I am executing them with maven locally. I tried in two different OS machines: Windows and Unix. The tests were executed without any problem, all of them passed.

I have issue in case I am executing the tests on Jenkins. First of all the tests are triggered in a parallel manner on Jenkins as well (the thread information are showed in the logs: pool-1-thread-2; pool-1-thread-1; pool-1-thread-3). Some of the tests are failing on Jenkins. The tests failed because the element cannot be find on the current state. The screenshot capture is enabled. The elements are displayed accordingly on the screenshot. Also I checked the assertion where the test fails is performed by the same thread that had performed other steps above in the same test.

I am using the Xvfb Jenkins plugin in order to be able to perform UI interaction with the tests: image

The agent has configured with 5 executors. These executors are not used because the the tests are running in parallel on the same machine (not multiple machines). If I am wrong please correct me.

I don't have any idea what can be the problem. Somebody has any experience with this kind of configuration? I am welcome any ideas.

vayaszsolt commented 2 years ago

@wakaleo Do you have any idea where should I post my issue? I am very sure I am not the only guy who uncounted this type of issue.... I posted at stackoverflow as well but with no success...

wakaleo commented 2 years ago

If it's passing locally it's probably not a Serenity or Selenium library issue as such - could be wait conditions, driver crashing or some other issue. The Gitter channel would be a good place to ask.

vayaszsolt commented 2 years ago

@wakaleo The issue was focus related. The tests lose focus in the browser window in case multiple tests are running on the same machine. I moved the tests to run in headless mode and this it fixed the issue.