JetBrains-Research / TestSpark

TestSpark - a plugin for generating unit tests. TestSpark natively integrates different AI-based test generation tools and techniques in the IDE. Started by SERG TU Delft. Currently under implementation by JetBrains Research (Software Testing Research) for research purposes.
MIT License
53 stars 23 forks source link

[Kotlin] Package directive does not match file location & test cases do not see modifications in the file under test #296

Open Frosendroska opened 4 months ago

Frosendroska commented 4 months ago

Describe the bug

How it works for Java

The following java class

package org.tests;

<some imports>

public class GeneratedMethodPublicInvokesPrivateTest {

    @Test
    public void methodPublicInvokesPrivateTest() {
       <some code>
    }
}

is located in

/private/var/folders/pm/<>/T/testSparkResults/test_gen_result_<>/org/tests/GeneratedMethodPublicInvokesPrivateTest.java

And the IDEA see that the package name and the location (the suffix of the path) matches. So we can have the proper access to the package elements.

How it works for Kotlin

The following kotlin class

package org.example

<some imports>

class GeneratedConstructorPrimaryNameOnlyTest {

    @Test
    fun constructorPrimaryNameOnlyTest() {
        <some code>
    }
}

is located in

/private/var/folders/pm/<>/T/testSparkResults/test_gen_result_<>/org/example/GeneratedConstructorPrimaryNameOnlyTest.kt

But IDEA shows me the warning: Package directive does not match the file location. So we have the problem with access.

For example, we can not access, so the compilation for all tests of this class will fail.

internal class MyClass(private val name: String) {}
Vladislav0Art commented 1 month ago

What works

Currently, the described scenario works fine: the test case compiles and runs successfully. Yet, the warning persists, it does not affect the execution.

Screenshot 2024-10-08 at 14 05 10

What does not work

But there will be an error if I generate a test suite, then add any class/method/function in my file under test and try to use the newly created construct in any test case. I will get the following error:

Screenshot 2024-10-08 at 14 09 28

Steps to reproduce:

  1. Generate a test suite for a class CalcKotlin in the following file:
    
    internal class A {
    val a = "123"
    }

class CalcKotlin { val a = 10;

fun add(a: Int, b: Int): Int {
    return a + b
}

}

2. Suppose the following test case is generated:
```kotlin
class GeneratedAddNegativeNumbersTest {
    private val calc = CalcKotlin()
    @Test
    fun addNegativeNumbersTest() {
        val result = calc.add(-5, -10)
        assertEquals(-15, result)
    }
}
  1. Now, add a creation of A and an assert for its value, it should execute successfully:

    class GeneratedAddNegativeNumbersTest {
    private val calc = CalcKotlin()
    @Test
    fun addNegativeNumbersTest() {
        val result = calc.add(-5, -10)
        assertEquals(-15, result)
    
        val a = A()
        assertEquals("123", a.a)
    }
    }
  2. Now, add the class B into the file under test:

internal class A { val a = "123" }

class B { // added this class; notice that it is public! val b = "1" }

class CalcKotlin { val a = 10;

fun add(a: Int, b: Int): Int {
    return a + b
}

}

5. Now, instantiate this class in the test case:
```kotlin
class GeneratedAddNegativeNumbersTest {
    private val calc = CalcKotlin()
    @Test
    fun addNegativeNumbersTest() {
        val result = calc.add(-5, -10)
        assertEquals(-15, result)

        val a = A()
        assertEquals("123", a.a)

       val b = B()
       // optional:
       assertEquals("1", b.b)
    }
}
  1. Execute the test case and see the error as in the image above.