JetBrains-Research / TestSpark

TestSpark - a plugin for generating unit tests. TestSpark natively integrates different AI-based test generation tools and techniques in the IDE. Started by SERG TU Delft. Currently under implementation by JetBrains Research (ICTL) for research purposes.
MIT License
35 stars 9 forks source link

Repeated test generation during execution of already generated tests causes `NullPointerException` in `TestsExecutionResultService` #243

Open Vladislav0Art opened 1 month ago

Vladislav0Art commented 1 month ago

Describe the bug I generated tests and started their execution via "Run all" button. Then BEFORE the execution has finished, I requested the 2nd test generation. It resulted in the java.lang.NullPointerException error. The trace of the error is as follows:

Screenshot 2024-06-05 at 16 51 05
2024-06-05 16:46:45,641 [ 112968] SEVERE - #c.i.o.p.Task - null
java.lang.NullPointerException
    at org.jetbrains.research.testspark.services.TestsExecutionResultService.addPassedTest(TestsExecutionResultService.kt:35)
    at org.jetbrains.research.testspark.tools.TestProcessor.processNewTestCase(TestProcessor.kt:199)
    at org.jetbrains.research.testspark.display.TestCasePanelFactory.runTest(TestCasePanelFactory.kt:519)
    at org.jetbrains.research.testspark.display.TestCasePanelFactory.access$runTest(TestCasePanelFactory.kt:59)
    at org.jetbrains.research.testspark.display.TestCasePanelFactory$addTask$1.invoke(TestCasePanelFactory.kt:511)
    at org.jetbrains.research.testspark.display.TestCasePanelFactory$addTask$1.invoke(TestCasePanelFactory.kt:510)
    at org.jetbrains.research.testspark.display.TopButtonsPanelFactory$executeTasks$1$1.run(TopButtonsPanelFactory.kt:170)
    at com.intellij.openapi.progress.impl.CoreProgressManager.startTask(CoreProgressManager.java:477)
    at com.intellij.openapi.progress.impl.ProgressManagerImpl.startTask(ProgressManagerImpl.java:133)
    at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcessWithProgressAsynchronously$6(CoreProgressManager.java:528)
    at com.intellij.openapi.progress.impl.ProgressRunner.lambda$submit$4(ProgressRunner.java:250)
    at com.intellij.openapi.progress.ProgressManager.lambda$runProcess$0(ProgressManager.java:100)
    at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcess$1(CoreProgressManager.java:221)
    at com.intellij.platform.diagnostic.telemetry.helpers.TraceKt.use(trace.kt:46)
    at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcess$2(CoreProgressManager.java:220)
    at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$executeProcessUnderProgress$13(CoreProgressManager.java:660)
    at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:735)
    at com.intellij.openapi.progress.impl.CoreProgressManager.computeUnderProgress(CoreProgressManager.java:691)
    at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:659)
    at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:79)
    at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:202)
    at com.intellij.openapi.progress.ProgressManager.runProcess(ProgressManager.java:100)
    at com.intellij.openapi.progress.impl.ProgressRunner.lambda$submit$5(ProgressRunner.java:250)
    at com.intellij.openapi.progress.impl.ProgressRunner$ProgressRunnable.run(ProgressRunner.java:500)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:702)
    at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:699)
    at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
    at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1.run(Executors.java:699)
    at java.base/java.lang.Thread.run(Thread.java:840)
2024-06-05 16:46:45,652 [ 112979] SEVERE - #c.i.o.p.Task - IntelliJ IDEA 2024.1  Build #IC-241.14494.240
2024-06-05 16:46:45,652 [ 112979] SEVERE - #c.i.o.p.Task - JDK: 17.0.10; VM: OpenJDK 64-Bit Server VM; Vendor: JetBrains s.r.o.
2024-06-05 16:46:45,652 [ 112979] SEVERE - #c.i.o.p.Task - OS: Mac OS X
2024-06-05 16:46:45,652 [ 112979] SEVERE - #c.i.o.p.Task - Plugin to blame: TestSpark version: 0.2.1
2024-06-05 16:46:45,652 [ 112979] SEVERE - #c.i.o.p.Task - Last Action: TestSpark.TestSparkActions

To Reproduce Steps to reproduce the behavior (potential scenario):

  1. Execute test generation for class via EvoSuite (LLM and other code types are also sufficient, but in the mentioned configuration, the number of tests generated is usually greater).
  2. Once tests are generated, run them via "Run all" button, and then BEFORE the execution is finished, request the test generation again (e.g., for a class via EvoSuite)
  3. See error.

Expected behavior The application must not contain a NullPointerException, and the repeated test generation request must execute without errors.