Add module-info to TCK

[keilw]

Allowing to configure the TCK test suite via ServiceConfiguration in a module-info instead of META-INF:

module com.service.provider {
    requires tech.units.tck;
    provides tech.units.tck.util.ServiceConfiguration with com.service.provider.MyServiceConfig;
}

[keilw]

There are several dependencies not compatible with JPMS, especially MutabilityDetector and JBoss Test Audit. @desruisseaux Any idea, if those could be added easily? I looked at major TCKs of Jakarta EE, including those by JBoss using Test Audit, and none of them seem to bother exposing the TCK itself via JPMS.

Therefore, if we do not find a quick and proper solution for the non-modular dependencies, we may have to abandon the idea.

[desruisseaux]

I'm not sure to understand the meaning of dependency that are not compatible. Are they "classical" dependencies, without module-info files? In that case, it should be possible to use those dependencies anyway. Posting the error message would help.

If those dependencies try to load a JAR file dynamically (after JVM startup) using their own URLClassLoader, indeed the way to do that in JPMS is very different (requires the use of ModuleLayer). But is it the problem encountered in this issue?

[keilw]

Essentially the JBoss libraries break it (and JBoss are notorious JPMS-haters, hence they don't really care about it in places like Jakarta EE, too)

[INFO] 42 errors 
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  8.204 s
[INFO] Finished at: 2023-08-30T11:41:17+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.11.0:compile (default-compile) on project unit-tck: Compilation failure: Compilation failure: 
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/util/TestUtils.java:[33,27] cannot find symbol
[ERROR]   symbol:   class MatcherAssert
[ERROR]   location: package org.hamcrest
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/util/TestUtils.java:[33,1] static import only from classes and interfaces
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/util/TestUtils.java:[34,27] package org.hamcrest.number is not visible
[ERROR]   (package org.hamcrest.number is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/util/TestUtils.java:[34,1] static import only from classes and interfaces
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/spi/ObtainingUnitsTest.java:[43,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/spi/ObtainingUnitsTest.java:[44,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/unit/UnitConversionTest.java:[40,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/unit/UnitConversionTest.java:[41,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/format/QuantityFormatTest.java:[43,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/format/QuantityFormatTest.java:[44,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/quantity/QuantityInterfaceTest.java:[41,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/quantity/QuantityInterfaceTest.java:[42,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/quantity/QuantityTypesTest.java:[41,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/quantity/QuantityTypesTest.java:[42,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/format/UnitFormatTest.java:[43,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/format/UnitFormatTest.java:[44,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/spi/QuantityFactoryTest.java:[47,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/spi/QuantityFactoryTest.java:[48,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/unit/UnitDimensionTest.java:[40,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/unit/UnitDimensionTest.java:[41,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/spi/ServiceProviderTest.java:[32,27] cannot find symbol
[ERROR]   symbol:   class MatcherAssert
[ERROR]   location: package org.hamcrest
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/spi/ServiceProviderTest.java:[32,1] static import only from classes and interfaces
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/spi/ServiceProviderTest.java:[33,27] package org.hamcrest.number is not visible
[ERROR]   (package org.hamcrest.number is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/spi/ServiceProviderTest.java:[33,1] static import only from classes and interfaces
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/spi/ServiceProviderTest.java:[34,27] package org.hamcrest.number is not visible
[ERROR]   (package org.hamcrest.number is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/spi/ServiceProviderTest.java:[34,1] static import only from classes and interfaces
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/spi/ServiceProviderTest.java:[46,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/spi/ServiceProviderTest.java:[47,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/FundamentalTypesTest.java:[35,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/FundamentalTypesTest.java:[36,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/spi/ServicesTest.java:[52,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/spi/ServicesTest.java:[53,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/unit/UnitInterfaceTest.java:[40,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/unit/UnitInterfaceTest.java:[41,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/TCKSetupTest.java:[38,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/TCKSetupTest.java:[39,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/unit/PrefixInterfaceTest.java:[38,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/unit/PrefixInterfaceTest.java:[39,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/spi/ObtainingQuantiesTest.java:[49,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/spi/ObtainingQuantiesTest.java:[50,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/spi/SystemOfUnitsTest.java:[45,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/tech/units/tck/tests/spi/SystemOfUnitsTest.java:[46,28] package org.jboss.test.audit.annotations is not visible
[ERROR]   (package org.jboss.test.audit.annotations is declared in the unnamed module, but module tech.units.tck does not read it)
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

But the missing Hamcrest import makes things even a bit worse, because if org.hamcrest is added, then these errors add to the problem:

[ERROR] the unnamed module reads package org.hamcrest from both MutabilityDetector and org.hamcrest
[ERROR] the unnamed module reads package org.hamcrest.internal from both MutabilityDetector and org.hamcrest
[ERROR] the unnamed module reads package org.hamcrest.core from both MutabilityDetector and org.hamcrest
[ERROR] module org.reflections reads package org.hamcrest from both MutabilityDetector and org.hamcrest
[ERROR] module org.reflections reads package org.hamcrest.internal from both MutabilityDetector and org.hamcrest
[ERROR] module org.reflections reads package org.hamcrest.core from both MutabilityDetector and org.hamcrest
[ERROR] module org.hamcrest reads package org.hamcrest from both MutabilityDetector and org.hamcrest
[ERROR] module org.hamcrest reads package org.hamcrest.internal from both MutabilityDetector and org.hamcrest
[ERROR] module org.hamcrest reads package org.hamcrest.core from both MutabilityDetector and org.hamcrest
[ERROR] module MutabilityDetector reads package org.hamcrest.internal from both org.hamcrest and MutabilityDetector
[ERROR] module MutabilityDetector reads package org.hamcrest from both org.hamcrest and MutabilityDetector
[ERROR] module MutabilityDetector reads package org.hamcrest.core from both org.hamcrest and MutabilityDetector
[ERROR] module org.testng reads package org.hamcrest from both MutabilityDetector and org.hamcrest
[ERROR] module org.testng reads package org.hamcrest.internal from both MutabilityDetector and org.hamcrest
[ERROR] module org.testng reads package org.hamcrest.core from both MutabilityDetector and org.hamcrest
[ERROR] /C:/Users/Werner/git/unit-tck/src/main/java/module-info.java:[30,1] module tech.units.tck reads package org.hamcrest from both MutabilityDetector and org.hamcrest
[ERROR] -> [Help 1]

The JBoss JARs include hyphens, and are not recognized like MutabilityDetector. Maybe they need to be substituted with other strings, do you know something about that?

[keilw]

Hamcrest was just used in a single place and while assertThat() in a matcher looks a bit more elegant, a simple assertTrue() with a number comparison also works. Leaving only the JBoss library issue.

[keilw]

@desruisseaux I think I found the missing declaration for the JBoss modules:

    requires jboss.test.audit.api;
    requires jboss.test.audit.impl;

Which seems a fallback by JPMS if the JAR file is not a proper qualifier (e.g. contains "-" or similar characters)

There is still one usage of Hamcrest in ServiceProviderTest which is currently not possible because MutabilityDetector bundled some Hamcrest classes (if they are used internally, they should have used Sealed Classes) and created package-splitting, but the easiest solution here is to remove the convenience assertions and stick to assertTrue() of TestNG.

[keilw]

@desruisseaux I am out of ideas now. Converted all tests to TestNG, but the build still fails:

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  10.807 s
[INFO] Finished at: 2023-09-24T17:56:49+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.11.0:testCompile (default-testCompile) on project unit-tck: Fatal error compiling: warning: source release 9 requires target release 9 -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

I created an experimental branch with everything I did, feel free to check it out and try to fix it. Maybe the whole Antrun plugin option used in the API might help, I would rather not use the Toolchain plugin, but if you prefer, please do. Otherwise we have to go without JPMS for this release, it's already been postponed because of the TCK only.

[desruisseaux]

I have not yet looked at the experiment, but my first thoughts would be:

• Does the TCK really needs all those dependencies (JBoss, etc.)? For long-term projects, I try to limit dependencies to those that are strictly needed. What values brings JBoss, is it something we could easily do ourselves with a few lines of code?
• What is the error message exactly? Is it a class that the compiler does not found? Which JAR file provides that class? What is the output of jar --describe-module -f that_file.jar?
• The Maven output reported in the comment last month, was it with JUnit 4, JUnit 5 or something else?
• I do not know if TestNG is a JPMS module, but JUnit 5 is JPMS. Did you tried it?

[keilw]

The JBoss plugin is crucial to the TCK, it binds all tests to the spec entries.

Same for TestNG, it would be a tremendous effort if not impossible to change the test harness to JUnit 5 now. The dynamic configuration of test profiles at runtime also is something only TestNG can provide the same way, CDI or Jakarta Validation also use it, as does almost any Jakarta EE TCK.

The error is that Somehow default-testCompile of tests does not understand the default-compile / base-compile separation seemingly working for the main code:

[INFO] 
[INFO] --- resources:3.3.1:testResources (default-testResources) @ unit-tck ---
[INFO] Copying 1 resource from src\test\resources to target\test-classes
[INFO] 
[INFO] --- compiler:3.11.0:testCompile (default-testCompile) @ unit-tck ---
[WARNING] Parameter 'optimize' (user property 'maven.compiler.optimize') is deprecated: This property is a no-op in javac.
[INFO] Changes detected - recompiling the module! :dependency
[INFO] Compiling 2 source files with javac [debug optimize verbose target 1.8] to target\test-classes
[INFO] [1m------------------------------------------------------------------------[m
[INFO] [1;31mBUILD FAILURE[m
[INFO] [1m------------------------------------------------------------------------[m
[INFO] Total time:  12.103 s
[INFO] Finished at: 2023-09-29T20:51:58+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.11.0:testCompile (default-testCompile) on project unit-tck: Fatal error compiling: warning: source release 9 requires target release 9 -> [Help 1]

I don't think there is any more problem with JPMS, even the libraries without explicit module-info are recognized, all I had to do is remove the one that really duplicated the packages from another (

[keilw]

I was now able to enforce the Java version for tests. Now the problem of MutabilityDetector duplicating several packages of Hamcrest remains. And several other libraries use Hamcrest makes it almost impossible to avoid that, so hopefully it can be replaced by something else like PIT.

@desruisseaux Do you know or use another mutability test library?

[keilw]

It seems testImmutable*() methods were added very early in JSR 363 times, but the TCK test suite/runner never actually made use of them. Also in the TCK we should not simply remove a method, but deprecate it and mark it as unused and to be pruned.

It seems, MutabilityDetector is quite powerful (although we never used it in our TCK or Unit Tests, so I cannot say, how well it works) but has not seen releases in over a year now.

We may show in unit-tck-usage how additional tests like PIT (via the Maven plugin), SigTest (also has a Maven plugin through NetBeans) or MutabilityDetector could work. Using a non-JPMS version, I think it's time to revive and repurpose the "java8" branch of TCK usage, where MutabilityDetector could be used, while the Jigsaw-enabled "main" branch probably has to use something else.

[keilw]

@desruisseaux Please review and approve (if you find it OK) https://github.com/unitsofmeasurement/unit-tck/pull/52. It should fix this task. While I found that testImmutable*() is not really used and the API has no mandatory immutability expectations, Quantity implementations must all be Comparable, thus one more test case was added (the method already existed, it had to be improved like testSerializable*()).

Serializable is currently not expected by the API, I do not want to unwrap 2.2 which was available for almost 6 months, but we may revisit that requirement in the future, if it's fine with all compatible implementations.

Please check the Comparable requirement, but the way Seshat implements it in Scalar, I guess it should pass.