UnitTestBot / UTBotJava

Automated unit test generation and precise code analysis for Java
Apache License 2.0
136 stars 43 forks source link

Wrong constructor arguments are generated for `enum` constructors #673

Open dtim opened 2 years ago

dtim commented 2 years ago

Description

Suppose we have an enum class with a constructor that sets a field using its argument, and a method is defined for this enum that reads the field (e.g., a getter).

As enum constructors are implicitly private, and it is forbidden to create additional instances of the enum class besides the automatically created values, the set of arguments that may be passed to the constructor, and the set of possible values assigned to the field, is closed.

When concrete execution is turned off, the symbolic engine generates executions that assume that the constructor has been called with some arbitrary arguments that do not belong to the "real" value set (e.g., null).

When concrete executor is enabled, it fixes the problem by providing a real argument value.

This bug results in generation of failing tests if concrete execution is turned off.

Found by @volivan239

To Reproduce

Set useConcreteExecution = false in .utbot/settings.properties in your home directory.

Generate a test suite for the getConstant method in the following class:

public enum EnumWithConstant {
    ENUM_VALUE("constant_1");

    EnumWithConstant(String constant) {
        this.constant = constant;
    }

    public String getConstant() {
        return constant;
    }

    private final String constant;
}

Expected behavior

The generated test should check that getConstant() returns "constant_1".

Actual behavior

The generated test compares the return value of getConstant() with an arbitrary string, e.g., the empty string. The test fails when executed.

Visual proofs (screenshots, logs, images)

The test that is generated without concrete execution:

public class EnumWithConstantTest {
    ///region Test suites for executable enums.EnumWithConstant.getConstant

    ///region SUCCESSFUL EXECUTIONS for method getConstant()

    /**
     * <pre>
     * Test returns from: {@code return constant; }
     * </pre>
     */
    @Test
    public void testGetConstant_ReturnConstant() {
        EnumWithConstant enumWithConstant = EnumWithConstant.ENUM_VALUE;

        String actual = enumWithConstant.getConstant();

        String expected = "";
        assertEquals(expected, actual);
    }
    ///endregion

    ///endregion

}

Environment

Concrete execution is turned off in UtSettings.

Additional context

This bug causes unit tests in org.utbot.examples.annotations.LombokAnnotationTest test suite to fail when concrete execution is off.

dtim commented 2 years ago

Maybe the right way to fix this issue is to change the UtModel hierarchy one more time and make UtEnumConstantModel a subclass of UtCompositeModel.

Rationale: enums can do all that regular classes do (declare fields and methods, overload methods, mutate internal state, have constructors) except creating new instances (one cannot create a new enum instance neither directly nor via reflection). By representing enums as composite models, we can model their internal structure, and by using a distinct class we can prevent forbidden operations.