search

testng-team / testng

TestNG testing framework
https://testng.org
Apache License 2.0
2k stars 1.02k forks source link

java.lang.OutOfMemoryError: Java heap space - NavigatorPanel.generateMethodList(NavigatorPanel.java:288) #2334

Open kyleshrader opened 4 years ago

kyleshrader commented 4 years ago

TestNG Version

7.1.0

Expected behavior

Expecting the following reports to be generated: image

Actual behavior

Recieved the included exception during report generation and the following was generated: image

Exception

[ERROR] Exception in provider
[ERROR] org.apache.maven.surefire.booter.SurefireExecutionException: Exception in provider
[ERROR] at org.apache.maven.plugin.surefire.InPluginVMSurefireStarter.runSuitesInProcess(InPluginVMSurefireStarter.java:91)
[ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1166)
[ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1011)
[ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:857)
[ERROR] at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
[ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
[ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
[ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
[ERROR] at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
[ERROR] at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
[ERROR] at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
[ERROR] at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
[ERROR] at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
[ERROR] at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
[ERROR] at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
[ERROR] at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
[ERROR] at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
[ERROR] at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
[ERROR] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[ERROR] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR] at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
[ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
[ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
[ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
[ERROR] Caused by: java.lang.OutOfMemoryError: Java heap space
[ERROR] at java.base/java.util.Arrays.copyOf(Arrays.java:3745)
[ERROR] at java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:172)
[ERROR] at java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:538)
[ERROR] at java.base/java.lang.StringBuilder.append(StringBuilder.java:174)
[ERROR] at org.testng.reporters.Files.readFile(Files.java:31)
[ERROR] at org.testng.reporters.Files.readFile(Files.java:22)
[ERROR] at org.testng.reporters.FileStringBuffer.toString(FileStringBuffer.java:135)
[ERROR] at org.testng.reporters.XMLStringBuffer.toXML(XMLStringBuffer.java:339)
[ERROR] at org.testng.reporters.jq.NavigatorPanel.generateMethodList(NavigatorPanel.java:288)
[ERROR] at org.testng.reporters.jq.NavigatorPanel.generateResult(NavigatorPanel.java:125)
[ERROR] at org.testng.reporters.jq.NavigatorPanel.generate(NavigatorPanel.java:71)
[ERROR] at org.testng.reporters.jq.Main.generateReport(Main.java:62)
[ERROR] at org.testng.TestNG.generateReports(TestNG.java:1069)
[ERROR] at org.testng.TestNG.run(TestNG.java:1012)
[ERROR] at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:135)
[ERROR] at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeMulti(TestNGDirectoryTestSuite.java:193)
[ERROR] at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:94)
[ERROR] at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:146)
[ERROR] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[ERROR] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR] at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[ERROR] at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray2(ReflectionUtils.java:206)
[ERROR] at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:161)
[ERROR] at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:84)
[ERROR] at org.apache.maven.plugin.surefire.InPluginVMSurefireStarter.runSuitesInProcess(InPluginVMSurefireStarter.java:87)
[ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1166)
[ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1011)
[ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:857)
[ERROR] at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
[ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
[ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)

Is the issue reproductible on runner?

Test case sample

None available. Running 500,000+ tests using data provider.

krmahadevan commented 4 years ago

@kyleshrader - Are you seeing this even after bumping up your JVM memory settings via the sure fire plugin? I think you should try running this with a higher JVM memory settings. I say this because the number of tests is huge.

@juherr @cbeust Your thoughts?

juherr commented 4 years ago

@kyleshrader The default reporter is not very well optimized. For specific usages like yours, you should consider having your own reporter (or find a better one in 3rd party lib).

kyleshrader commented 4 years ago

Thank you for the recommendations.

  1. I will first try various surefire configurations to see if I can provide the memory needed to surefire.
  2. Then I will look into 3rd party reporting libraries, or write a simple reporting library to handle the large amounts of tests.
  3. Due to the nature of my tests, I will also look at breaking them down into smaller test runs, but this might not be a suitable solution for everyone that runs into this scenario.

I will report back what I find.

@juherr - do you have any recommended 3rd party reporter for large numbers of tests, that I could start off looking into?