With the new version long running suites are getting stuck in Jenkins and coming out with results having 1 scenario

[neo81-83]

I have a suite of 1500 scenarios which i am running with 40 threads, normally with version 3.3.0 it has been running for approximately 1 hrs. 45 mins BUT with the upgrade to 5.8.0 the suite runs fine for a while and then after sometime execution slows down and after 4 hrs. it produces results with just 1 scenario.

FYI, everything is same only Courgette version has changed from 3.3 to 5.8. What could be possible issues here? What is default heap settings for Java and Maven?

[neo81-83]

This is the error: Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 57,163.202 sec <<< FAILURE! 2021-01-12 10:39:48.567 parallelRun(com.onetrust.cucumber.runner.TestScenarioRunner) Time elapsed: 57,153.845 sec <<< FAILURE! 2021-01-12 10:39:48.567 java.lang.OutOfMemoryError: Java heap space 2021-01-12 10:39:48.567 2021-01-12 10:39:48.567 2021-01-12 10:39:48.567 Results : 2021-01-12 10:39:48.567 2021-01-12 10:39:48.567 Failed tests: parallelRun(com.onetrust.cucumber.runner.TestScenarioRunner): Java heap space

[prashant-ramcharan]

Courgette version 3.3.0 uses Cucumber 4 and Courgette version 5.8.0 uses Cucumber 6.

There has been significant changes to Cucumber between these versions. One of which is html reporting which produces large reports compared to earlier versions (i.e. Cucumber 4).

The report messages are stored in memory within Courgette so I'd imagine running 1500 tests using a single Java heap running for 1 hour would cause the OOME.

• How much memory does your Jenkins agent have to run 40 threads?
• Have you considered splitting your tests into multiple jobs in Jenkins?
• You can try allocating more memory to the process that starts the Courgette runner.

You can check your default heap sizes using the following command:

java -XX:+PrintFlagsFinal -version | grep HeapSize

[neo81-83]

Thanks for response..the jenkins node has around 120 GB memory..various heap limits are as follows:

size_t ErgoHeapSizeLimit                        = 0                                         {product} {default}
   size_t HeapSizePerGCThread                      = 43620760                                  {product} {default}
   size_t InitialHeapSize                          = 2113929216                                {product} {ergonomic}
   size_t LargePageHeapSizeThreshold               = 134217728                                 {product} {default}
   size_t MaxHeapSize                              = 32178700288                               {product} {ergonomic}
    uintx NonNMethodCodeHeapSize                   = 8182140                                {pd product} {ergonomic}
    uintx NonProfiledCodeHeapSize                  = 121738050                              {pd product} {ergonomic}
    uintx ProfiledCodeHeapSize                     = 121738050                              {pd product} {ergonomic}
openjdk version "11.0.5" 2019-10-15
OpenJDK Runtime Environment (build 11.0.5+10-post-Ubuntu-2ubuntu116.04)
OpenJDK 64-Bit Server VM (build 11.0.5+10-post-Ubuntu-2ubuntu116.04, mixed mode, sharing)

How can I allocate more memory to the process that starts Courgette runner? I have following config. in failsafe plugin:

<argLine>-Xms1024m -Xmx1536m -XX:MaxPermSize=512m</argLine>
<reuseForks>true</reuseForks>

[prashant-ramcharan]

As you have sufficient memory on your build agent you could perhaps try updating the heap sizes to the following:

<reuseForks>false</reuseForks>
<argLine>-Xms8g -Xmx16g -XX:MaxPermSize=4g</argLine>

[neo81-83]

ok, can I use -Dcourgette.vmoptions='-Xms8g -Xmx16g -XX:MaxPermSize=4g' instead to override from cmd?

[prashant-ramcharan]

using -Dcourgette.vmoptions would only set the heap size for each test / thread.

You need to tweak the heap size of the starting process (i.e. Maven). In your case, it's in the maven failsafe plugin.

[neo81-83]

aah! got it thanks...I will make the changes and let you know..

[pagmoro]

"One of which is html reporting which produces large reports compared to earlier versions"

Can the courgette reporting be turned off?

[prashant-ramcharan]

Can the courgette reporting be turned off?

Yes, I could add a feature to disable specific html reporting.

[prashant-ramcharan]

@neo81-83 Please use version 5.9.0.

I've added a new Courgette option to disable Cucumber and Courgette html reporting.

My suggestion would be to disable the Cucumber report as this would save a lot of memory as Courgette wouldn't have to store large reports in memory which would eventually lead to an out of memory exception when having 1000+ tests.

Update your runner to include the disableHtmlReport option.

disableHtmlReport = {HtmlReport.CUCUMBER_HTML}

or disable both reports

disableHtmlReport = {HtmlReport.CUCUMBER_HTML, HtmlReport.COURGETTE_HTML}

[prashant-ramcharan]

Closing this issue - please try suggestions noted above.