Can't use kaitai-struct-runtime on JDK 8: NoSuchMethodError

[archiecobbs]

Can't run kaitai on JDK 8 or you get this exception:

Exception in thread "main" java.lang.NoSuchMethodError: java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer;
    at io.kaitai.struct.ByteBufferKaitaiStream.seek(ByteBufferKaitaiStream.java:153)
    at com.example.MicrosoftCfb.dir(MicrosoftCfb.java:413)
    at com.example.MicrosoftCfb.print(MicrosoftCfb.java:434)
    at com.example.MicrosoftCfb.main(MicrosoftCfb.java:430)

Please build with --release 8 to avoid this problem.

The problem is described in more detail here.

[dgelessus]

The pom.xml already configures maven-compiler-plugin so that the source and target language level are set to Java 6, but this just corresponds to javac's -source and -target options - it doesn't set anything like --release (introduced in Java 9).

https://github.com/kaitai-io/kaitai_struct_java_runtime/blob/ea3abf89ee7745ac56c814202c4ddb26fc668e11/pom.xml#L118-L121

From what I've read, at least on Java 9 and newer, it should be enough to replace the <source>6</source> and <target>6</target> settings with just <release>6</release>, which will set --release 6. However I'm not sure if maven-compiler-plugin's release setting is compatible with Java 8 and older (the linked SO answer indicates that it isn't). I'm not all that familiar with Maven (I think I don't even have a working Maven installation right now, because I use Gradle for everything) - maybe someone who knows a bit more than me can check how well this works or what configuration would be needed to make the runtime compile correctly on both Java <= 8 and Java >= 9.

Also I'm wondering why the language level is still set to Java 6 - the runtime uses java.nio.file APIs in a few places, which were introduced in Java 7, so it's impossible to use the runtime on Java 6. I think javac from JDK 9 and newer will report this as an error if you set --release 6, because then javac will check the code against the actual APIs available in Java 6, instead of blindly using the current API definitions and just compiling with an older language level. Also, JDK 12 and newer no longer support compiling for Java 6 at all (new minimum language level is Java 7), so this setting will prevent the runtime from being built on current JDK versions.

[generalmimon]

Now when I know how -source + -target are different from --release, I can understand why the package io.kaitai:kaitai-struct-runtime:0.9 in the Maven Central doesn't work on Java 8 - @GreyCat said that he ran mvn deploy on a machine with JDK 11 when releasing 0.9.

I'm not sure if maven-compiler-plugin's release setting is compatible with Java 8 and older

Yeah, it really isn't. The --release option of javac (or the corresponding <release> parameter) was added in Java 9. So building and deploying the Maven package would not work in older versions than JDK 9. That's not a big deal, though - I suppose that only @GreyCat, @ams-tschoening or me sometimes need to run mvn package or mvn deploy in the Java runtime repo. It is a purely development thing.

If we want to stay compatible with Java 8 and older (on the side of the KS runtime library development machine), then I think the best thing we can do is this:

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.8.1</version>
        <configuration>
          <!-- For Java 8 and older -->
          <!-- <source>7</source>
          <target>7</target> -->
          <!-- /For Java 8 and older -->

          <!-- For Java 9+ -->
          <release>7</release>
          <!-- /For Java 9+ -->
        </configuration>
      </plugin>

Any users of Java 8 or older will have to uncomment the For Java 8 and older section and comment the For Java 9+, whereas the Java 9+ version will be enabled by default.

I tried to find if there is some --release 8 equivalent in Java 8 and there actually is (--source 8 --target 8 -bootclasspath <path>: you need to specify a path to a folder/ZIP archive with the API version that you want to target). But the necessity of providing -bootclasspath makes it very inconvenient for us - we'd need to probably include a ZIP archive with Java 8 classes in this source repository, and that's not worth at all.

So IMHO this little discomfort of not working on Java 8 out-of-the-box and having to do a small local change in the pom.xml for the package deployment is quite acceptable. We should just mention it in the repo README and/or the Developers memo.

[ams-tschoening]

How about using the Maven concepts of profiles and toolchains instead? Profiles can be used to trigger concrete builds with special aspects and toolchains to provide different JDKs compatible with different versions.

I've tested a bit with my available version of the runtime and a concrete JDK 8 and 11. The following is my toolchain.xml:

<?xml version="1.0" encoding="UTF-8"?>
<toolchains>
  <toolchain>
    <type>jdk</type>
    <provides>
      <version>1.6</version>
      <vendor>OpenJDK</vendor>
    </provides>
    <configuration>
      <jdkHome>C:\Program Files\Java\jdk-8</jdkHome>
    </configuration>
  </toolchain>

  <toolchain>
    <type>jdk</type>
    <provides>
      <version>1.8</version>
      <vendor>OpenJDK</vendor>
    </provides>
    <configuration>
      <jdkHome>C:\Program Files\Java\jdk-8</jdkHome>
    </configuration>
  </toolchain>

  <toolchain>
    <type>jdk</type>
    <provides>
      <version>11</version>
      <vendor>OpenJDK</vendor>
    </provides>
    <configuration>
      <jdkHome>C:\Program Files\Java\jdk-11</jdkHome>
    </configuration>
  </toolchain>
</toolchains>

The following is what I changed in pom.xml:

  <profiles>
    <profile>
      <id>JDK-8</id>

      <build>
        <plugins>
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-toolchains-plugin</artifactId>
            <version>1.1</version>
            <executions>
              <execution>
                <goals>
                  <goal>toolchain</goal>
                </goals>
              </execution>
            </executions>
            <configuration>
              <toolchains>
                <jdk>
                  <version>1.8</version>
                </jdk>
              </toolchains>
            </configuration>
          </plugin>
        </plugins>
      </build>
    </profile>

    <profile>
      <id>JDK-11</id>

      <build>
        <plugins>
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-toolchains-plugin</artifactId>
            <version>1.1</version>
            <executions>
              <execution>
                <goals>
                  <goal>toolchain</goal>
                </goals>
              </execution>
            </executions>
            <configuration>
              <toolchains>
                <jdk>
                  <version>11</version>
                </jdk>
              </toolchains>
            </configuration>
          </plugin>
        </plugins>
      </build>
    </profile>
  </profiles>

The different JDKs/profiles can be targeted on the shell like the following:

mvn package -P JDK-8
mvn package -P JDK-11

By changing the node version above, one can e.g. easily trigger errors because a JDK is not mentioned in toolchains.xml. Which makes clear that really the expected JDK is used. Using the same approach different compiler settings most likely can be used as well, I already saw such things in other projects.

The good thing about the approach above is that all other plugins etc. are simply shared by all profiles, so that one doesn't need to specify things redundantly. Without providing any concrete profile, simply what is available as JAVA_HOME or alike is used with whatever is mentioned in pom.xml without association to any profile.

[2RedSquares]

I have run into this same issue. Would it be possible to get a new push of 0.9 to Maven?

[generalmimon]

@ams-tschoening Your suggestion is probably in the right direction, but it looks quite intimidating and I suppose it only works for versions JDK 8 and 11. But today, I've come across this solution on SO (https://stackoverflow.com/a/65655542). It also uses profiles, but it's simpler and I believe it's exactly what we want:

<profile>
  <id>java-8-api</id>
  <activation>
    <jdk>[9,)</jdk>
  </activation>
  <properties>
    <maven.compiler.release>8</maven.compiler.release>
  </properties>
</profile>