search

adoptium / STF

The System Test Framework for executing https://github.com/adoptium/aqa-systemtest
Apache License 2.0
8 stars 35 forks source link

Revisit Java invocation build sequence logic in STF #101

Open Mesbah-Alam opened 3 years ago

Mesbah-Alam commented 3 years ago

STF enforces a particular order in which we can define a LoadTestProcessDefinition instance. This enforcement can cause STF test plugins to become complicated (or less simple than what is desirable). For example: 

LoadTestProcessDefinition loadTestInvocation = test.createLoadTestSpecification()
                .addJvmOption("-Djava.classloading.dir=" + notonclasspathDirRoot)  // Expose the bin_notonclasspath root directory to the deadlock test
                .addJvmOption("-Djava.version.number=" + javaVersion)
                .addModules(modulesAdd)
                .addPrereqJarToClasspath(JavaProcessDefinition.JarId.JUNIT)
                .addPrereqJarToClasspath(JavaProcessDefinition.JarId.HAMCREST)
                .addProjectToClasspath("openjdk.test.classloading"); 

        if (isTimeBasedLoadTest) { 
            loadTestInvocation = loadTestInvocation.setTimeLimit(timeLimit); // If it's a time based test, stop execution after given time duration
        }

        loadTestInvocation = loadTestInvocation.setAbortIfOutOfMemory(false)
                .addSuite("classloading")
                .setSuiteThreadCount(cpuCount - 1, 10)
                .setSuiteInventory(inventoryFile);

        if (!isTimeBasedLoadTest) { 
            loadTestInvocation = loadTestInvocation.setSuiteNumTests(totalTests * testCountMultiplier);
        }

        loadTestInvocation = loadTestInvocation.setSuiteRandomSelection();

The above definition could be simpler, if STF allowed the following: 

LoadTestProcessDefinition loadTestInvocation = test.createLoadTestSpecification()
                .addJvmOption("-Djava.classloading.dir=" + notonclasspathDirRoot)  // Expose the bin_notonclasspath root directory to the deadlock test
                .addJvmOption("-Djava.version.number=" + javaVersion)
                .addModules(modulesAdd)
                .addPrereqJarToClasspath(JavaProcessDefinition.JarId.JUNIT)
                .addPrereqJarToClasspath(JavaProcessDefinition.JarId.HAMCREST)
                .addProjectToClasspath("openjdk.test.classloading") 
                .setAbortIfOutOfMemory(false)
                .addSuite("classloading");

        if (isTimeBasedLoadTest) { 
            loadTestInvocation = loadTestInvocation.setTimeLimit(timeLimit); // If it's a time based test, stop execution after given time duration
        } else { 
            loadTestInvocation = loadTestInvocation.setSuiteNumTests(totalTests * testCountMultiplier);
        }

        loadTestInvocation = loadTestInvocation.setSuiteThreadCount(cpuCount - 1, 10)
                .setSuiteInventory(inventoryFile)
                .setSuiteRandomSelection();

This issue is opened to revisit the design principles of these order enforcements and update the logic therein if possible.

FYI @llxia