Problem with the jupiter @Public Annotation

[hornflip]

Describe the bug We were testing with the IOTester and stumbled accross an odd error. Minimal working example Main Method:

import java.util.Scanner;

public class InOutTester {
    public static void main(String[] args) {
        Scanner scanner = new Scanner(System.in);
        System.out.println("Bitte geben Sie Ihren Namen ein: ");
        String input = scanner.nextLine();
        scanner.close();
        System.out.println(input);
    }
}

Test Class:

import de.tum.in.test.api.io.IOTester;
import org.junit.jupiter.api.*;

import de.tum.in.test.api.BlacklistPath;
import de.tum.in.test.api.StrictTimeout;
import de.tum.in.test.api.WhitelistPath;
import de.tum.in.test.api.jqwik.Public;

/**
 * @author Stephan Krusche (krusche@in.tum.de)
 * @version 5.1 (11.06.2021)
 */
@Public
@WhitelistPath("target") // mainly for Artemis
@BlacklistPath("target/test-classes") // prevent access to test-related classes and resources
class InOutTesterTest {

    private IOTester tester;

    @BeforeEach
    void init() {
        tester = IOTester.installNew(false, 100000);
    }

    @AfterEach
    void cleanUp() {
        tester.uninstall();
    }

    @Test
    @StrictTimeout(1)
    void testMain() {
        tester.provideInputLines("Volker");

        InOutTester.main(null);

        tester.err().assertThat().isEmpty();
        tester.out().assertThat().isEqualTo("""
                Bitte geben Sie Ihren Namen ein:\s
                Volker""");
    }
}

This is how we run the IOTester. When we try to run it like shown in the Docs (with the parameter) we get an ParameterResolutionException:

import de.tum.in.test.api.BlacklistPath;
import de.tum.in.test.api.StrictTimeout;
import de.tum.in.test.api.WhitelistPath;
import de.tum.in.test.api.io.IOTester;
import de.tum.in.test.api.jqwik.Public;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;

/**
 * @author Stephan Krusche (krusche@in.tum.de)
 * @version 5.1 (11.06.2021)
 */
@Public
@WhitelistPath("target") // mainly for Artemis
@BlacklistPath("target/test-classes") // prevent access to test-related classes and resources
class InOutTester2Test {

    @Test
    @StrictTimeout(1)
    void testMain(IOTester tester) {
        tester.provideInputLines("Volker");

        InOutTester.main(null);

        tester.err().assertThat().isEmpty();
        tester.out().assertThat().isEqualTo("""
                Bitte geben Sie Ihren Namen ein:\s
                Volker""");
    }
}

So the Doc example does not work and to get it running via @ BeforeEach and @ AfterEach we have to adjust the import library for the @ Public Annotation from de.tum.in.test.api.jupiter.Public to de.tum.in.test.api.jqwik.Public. Now it works but this is kinda weird to understand for people who have to write test cases and don't know about it.

So what is the Issue here? Why do we get a java.lang.ExceptionInInitializerError thrown, when we use the jupiter package instead of jqwik?

[MaisiKoleni]

This is due to the two different test engines (JUnit Jupiter and jqwik) and their lifecycle extension mechanisms. Normally, nothing would happen, but due to the parameter without extension registered (as Jupiter disregards the jqwik extension), you get this exception.

So everything is working as intended.

But of course, this does not solve the problem that the API is confusing here. Unfortunately, this cannot be solved without breaking the API, I think. We could try to merge both extensions and basically make them equivalent (confusing as well, but no unexpected or dangerous behavior). Alternatively, we could deprecate the jqwik support, this would be easy to notice in an IDE. I won't do that because I no longer actively maintain Ares due to a number of issues, maybe someone else is willing to take care of that?

But meanwhile, I simply recommend being careful.