Java 11 Upgrade (no free security fixes for Oracle Java 8 after January 2019)

[matthew-a-dunlap]

At some point we should upgrade to Java 9 to gain the power of new java technology and remain secure.

Outcome of the #4217 spike.

[matthew-a-dunlap]

[Info from #4217]

Java 9: Looks pretty good, I seem to have it mostly running after a few hours today, but we probably don't want to upgrade yet.

Details:

• Tested on Oracle JDK 9.0.1+11 on osx 10.13
• Need Netbeans 9 for Java 9 to compile easily. There is only a developer build of Netbeans 9 and no release date (but active development). May be able to work around this with digging into maven but I did not do this.
• Builds w/o tests after cleaning up a few new ambiguous references (from new classes brought into Java 9)
• Tests have 1 error, the root cause seems to be that MimetypesFileTypeMap() getContentType is not returning right for png. Looking like implementation weirdness from Java 9, for now I'm just leaving it broken.
• Starts with more sql noise than I remember, but its been a while since I deployed from command line (see java 9 sql error.txt).
• Most of the simple dataverse interactions work! Create/delete dataset/dataverse works. Create user works. Uploading data DOES NOT work (see java 9 file upload error.txt).
• TravisCI works fine switching to Java9.
• Glassfish 4.1 itself is unable to detect the JDK version for Java 9. Glassfish needs to be set to explicitly use Java 8.

[pdurbin]

I'm glad we now have a issue dedicated to Java 9. I like working in small chunks. Thanks @matthew-a-dunlap

Here's the comment I left on #4217.

Java 9

I don't believe it makes sense to upgrade to Java 9 at this time. It's only been out for a few weeks so Linux vendors haven't had time to package it as java-1.9.0-openjdk. We want the same easy upgrade that we had from Java 7 to Java 8 where we simply change a single digit in a yum command like we did in 1abbd77 for #2151.

The next question is, "When will we be able to simply run yum install java-1.9.0-openjdk on CentOS and get Java 9? I'm not exactly sure but we can expect java-1.9.0-openjdk to land first in Fedora and then CentOS. According to https://fedoraproject.org//wiki/Changes/Java9TechPreview Java 9 will be part of Fedora 27 and https://bugzilla.redhat.com/show_bug.cgi?id=1447237 appears to be the tracking bug for including it. We've been able to happily run the java-1.8.0-openjdk RPM on CentOS 6 and I'm not sure if that means we'll be able to run java-1.9.0-openjdk on CentOS 6 as well or if we'll need to upgrade to CentOS 7. According to the "Oracle Java SE Support Roadmap" doc at http://www.oracle.com/technetwork/java/eol-135779.html Java 8 will stop receiving security updates in Sep 2018.

[pdurbin]

After listening to http://www.javaoffheap.com/2017/11/episode-28-back-from-javaone-with.html the other day I have even less reason to believe we should spend much time trying to upgrade to Java 9.

There's some good discussion at the end about how Java 9 is a non-LTS release. In the Java world we are used to every major release being supported for quite a while but the game is changing. Here's a chart from http://www.oracle.com/technetwork/java/eol-135779.html

First, note that Java 9 is a non-LTS release. The note how it says, "For releases after Java SE 8, Oracle Java SE product releases not designated as LTS releases will only be maintained until the subsequent release."

Forcing our users to upgrade to Java 8 in #2151 was the right thing to do because we were prodding them get on a supported version of Java if they weren't already. We should not force our users to move to a non-LTS version of Java. It's fine for developers to play around with Java 9, of course, but for production use, we should target the next LTS version of Java, which is 18.9 (the numbering has changed to be more like Ubuntu), due out in September 2018 according to the chart above.

Of course, as I'm saying this, I'm wondering to myself, "How long will Java 8 be supported?" According to this chart from the same page, Java 8 will continue to get public updates (security patches for the non-paid version) until September 2018:

As the chart above indicates, Java 8 will get public updates for a longer time than Java 9 (September 2018 vs. March 2018). To me, this is even more reason not to be in a hurry to switch from Java 8 to Java 9. The date to pay attention to is the one for "End of Public Updates" and to make sure that our community stays on a version of Java that receives security updates.

[pdurbin]

I've been listening to http://enterprisejavanews.com/episode-38-dec-2017 and there's a similar opinion about not bothering to upgrade to Java 9 since it isn't an LTS release. They're discussing https://www.infoworld.com/article/3223690/java/java-9-will-not-receive-long-term-support.html and over at https://youtu.be/Y1NJRGQwN8A?t=1h22m28s you can hear @thesteve0 say, "We as a Java community are in kind of a tricky place right now with supported Java because Java 9 support ends officially in September, I think, of 2018 and support for Java 8 ends offically September 2018." He goes on about this and it's a good listen.

I think we should think hard about what definition of done for this issue means. I have no problem with Dataverse developers playing around with Java 9 features on their laptops, but let's not force our community into upgrading to a non-LTS release of Java by bumping the required version in our pom file.

[pdurbin]

More on Java 9:

• https://www.azul.com/java-stable-secure-free-choose-two-three/
• http://blog.joda.org/2018/02/java-9-has-six-weeks-to-live.html

[pdurbin]

Since I just saw Java 9 mentioned at #4681 I want to reiterate that Java 9 is EOL. It no longer receives security updates. It was a non-LTS release. We should only be recommending LTS releases to our community.

[poikilotherm]

Hey guys,

it just came to my attention that non-commercial support for Oracle JDK 8 will end in January 2019.

There might huge impact for this project lurk in the shadows. As there will be no public updates to Oracle JDK 8 available anymore, people would be forced to use unsafe systems. That most certainly is not what they (should) want...

Updates to OpenJDK 8 seem to be coming in anyway, see here and here.

IMHO there should be some actions taken to be future-proof:

1. Change docs at http://guides.dataverse.org/en/4.9.4/installation/prerequisites.html#java to clearly state not to use Oracle JDK if not willing to spend money for a support license. Point people to other options like Zulu, AdoptOpenJdk, etc.
2. Test things with OpenJDK 8. Change TravisCI config not to use Oracle JDK 8 anymore.
3. Start moving to Java 11, which is a LTS version and GA as of 2018/09/25.
   • Beware: Java EE has been removed from the JDK, but can be builtin via Maven deps
   • Don't panic, stay calm and don't let the dogs bite you.
   • There might be a pitfall while moving to 11 staying with Glassfish 4 dependency. Needs testing, but upgrading should be done anyway, see #4248 .
   • Sidenote: after upgrading to 11, using Oracle JDK or OpenJDK built by Oracle (which is free of charge but unsupported) is interchangeable as stated by Oracle e.g. in their blog
4. Test, test, test and test again. Did I mention tests? :wink:

/me willing to act on this if help needed/wanted.

[pdurbin]

@poikilotherm thanks for you comment. It's good to know that a new LTS release of Java is finally out. I wonder when I'll be able to run yum install java-1.11.0-openjdk (or whatever Java 11 is) on CentOS.

With regard to what the Dataverse Installation Guide recommends in terms of where to download Java from, this is the quote from https://blogs.oracle.com/java-platform-group/oracle-jdk-releases-for-java-11-and-later that caught my eye:

"If you are used to getting Oracle Java SE binaries for free, you can simply continue doing so with Oracle’s OpenJDK builds available at jdk.java.net."

Here's what the guides say right now (permalink for the same section you linked to) at http://guides.dataverse.org/en/4.9.2/installation/prerequisites.html#installing-java

I guess I'm confused about if it's free as in beer to use Oracle JDK 11 or not. OpenJDK still seems to be free. In the Dataverse Installation Guide we don't currently require or recommend software you have to pay for. All the dependencies can be downloaded for free.

[poikilotherm]

I guess I'm confused about if it's free as in beer to use Oracle JDK 11 or not. OpenJDK still seems to be free.

Indeed, OpenJDK is free and will be free forever. This is the reference implementation on that any other JVM I'm aware of (with an exception for Android Java AFAIK) is building upon.

Oracle will provide two downloads:

1. the today commonly used "Oracle JDK" which is fully supported, but requires a license for at least commercial use (which most R&D institutes are for Oracle...).
2. the OpenJDK based binary build, which is not supported in any way - this is just a build and thats it. But it will be free of charge, as you mentioned above from the blog post.

The links in the docs currently point to the future-non-free "Oracle JDK".

(Sidenote: IIRC the "Oracle JDK" will stay free of charge for development and testing, but not for staging and production environments)

[pdurbin]

@poikilotherm ok, if paying for a license is required for Oracle JDK in production for R&D institutions, it sounds like we should switch to OpenJDK for the Installation Guide (and testing leading up to it).

I see your side note about testing but any thoughts on what's written at http://guides.dataverse.org/en/4.9.2/developers/dev-environment.html#install-java ? On Mac I've always used Oracle JDK for development. Here's a screenshot:

[poikilotherm]

@pdurbin wrote:

I wonder when I'll be able to run yum install java-1.11.0-openjdk (or whatever Java 11 is) on CentOS.

Citing https://access.redhat.com/articles/3409141 about that:

Java SE 11 is scheduled for September 2018, and will be the first release with LTS. Red Hat recommends Java SE 11 as a successor to existing Java SE 8 workloads in production.

Red Hat is planning to release a distribution of OpenJDK 11 for Red Hat Enterprise Linux 7 following the availability of the Java SE 11 release, and will provide long term production support for the distribution. [...] Red Hat will also work to certify its Java based products, the Red Hat JBoss portfolio of products in the Full Support phase of the lifecycle in particular, on Java SE 11 as soon as possible.

@pdurbin wrote:

license is required for Oracle JDK in production for R&D institutions

Currently I have no proof in form of a written document somewhere, just the live experience here at FZJ. We had a license audit from Oracle some time ago, when they wanted to know if some enterprise Java analysis tooling is in use. IIRC that where some profiling and heap analysis things that shipped with the JDK but their use required a commercial license. Lesson learned from that: for Oracle R&D is just like other commercial activities. Currently a lot of people here are whirring sirens regarding the Oracle JDKs used all around the campus because of that very lesson.

I see your side note about testing but any thoughts on what's [...]

I am using OpenJDK 8 (1.8.0_181) for development on my laptop and don't see any reason why not to use it for most scenarios.

The beast to tame is Java 11, where a lots of things changed coming from 8. And of course there always may exist some kind of race conditions or other changes, that hurt us now not being present while using Java 8.

The only things that helps with this is proper unit testing, integration tests and end-to-end tests, ideally combined with testing on different sizes of deployments and data inside plus some bigger cannons like Artillery. (Hell yeah, I'm just dreaming... :smile: Most certainly the majority of this will never happen, right?)

[pdurbin]

@poikilotherm ok, is your dev environment Mac or Linux? I just downloaded openjdk-11+28_osx-x64_bin.tar.gz for Mac from http://jdk.java.net/11/ and uncompressed it and it seems to run fine on my Mac running OS X 10.11 (El Capitan):

$ jdk-11.jdk/Contents/Home/bin/java --version
openjdk 11 2018-09-25
OpenJDK Runtime Environment 18.9 (build 11+28)
OpenJDK 64-Bit Server VM 18.9 (build 11+28, mixed mode)

It would be nice if I could install this via brew but I'm not sure if there's a formula for it.

[pdurbin]

As of 5390f28 on the "develop" branch Dataverse does not compile on Java 11 (openjdk 11 2018-09-25 on Mac). I ran this:

JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk-11.jdk/Contents/Home mvn package

And got compilation errors like this:

[ERROR] /Users/pdurbin/NetBeansProjects/dataverse/src/main/java/edu/harvard/iq/dataverse/util/JhoveFileType.java:[103,13] reference to Module is ambiguous

This ambiguity can be resolved by specifying edu.harvard.hul.ois.jhove.Module like this...

-            Module module = jb.getModule(null);
+            edu.harvard.hul.ois.jhove.Module module = jb.getModule(null);

... but now there are other errors:

murphy:dataverse pdurbin$ JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk-11.jdk/Contents/Home mvn package
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building dataverse 4.9.2
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:2.6:copy (default) @ dataverse ---
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.5.201505241946:prepare-agent (jacoco-initialize) @ dataverse ---
[INFO] argLine set to -javaagent:/Users/pdurbin/.m2/repository/org/jacoco/org.jacoco.agent/0.7.5.201505241946/org.jacoco.agent-0.7.5.201505241946-runtime.jar=destfile=/Users/pdurbin/NetBeansProjects/dataverse/target/jacoco.exec
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ dataverse ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 17 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ dataverse ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 715 source files to /Users/pdurbin/NetBeansProjects/dataverse/target/classes
[WARNING] /Users/pdurbin/NetBeansProjects/dataverse/src/main/java/edu/harvard/iq/dataverse/ingest/tabulardata/impl/plugins/xlsx/XLSXFileReader.java: Some input files use or override a deprecated API.
[WARNING] /Users/pdurbin/NetBeansProjects/dataverse/src/main/java/edu/harvard/iq/dataverse/ingest/tabulardata/impl/plugins/xlsx/XLSXFileReader.java: Recompile with -Xlint:deprecation for details.
[WARNING] /Users/pdurbin/NetBeansProjects/dataverse/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/MoveDataverseCommand.java: Some input files use unchecked or unsafe operations.
[WARNING] /Users/pdurbin/NetBeansProjects/dataverse/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/MoveDataverseCommand.java: Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ dataverse ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 16 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ dataverse ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 149 source files to /Users/pdurbin/NetBeansProjects/dataverse/target/test-classes
[WARNING] /Users/pdurbin/NetBeansProjects/dataverse/src/test/java/edu/harvard/iq/dataverse/api/FilesIT.java: Some input files use or override a deprecated API.
[WARNING] /Users/pdurbin/NetBeansProjects/dataverse/src/test/java/edu/harvard/iq/dataverse/api/FilesIT.java: Recompile with -Xlint:deprecation for details.
[WARNING] /Users/pdurbin/NetBeansProjects/dataverse/src/test/java/edu/harvard/iq/dataverse/mocks/MocksFactory.java: Some input files use unchecked or unsafe operations.
[WARNING] /Users/pdurbin/NetBeansProjects/dataverse/src/test/java/edu/harvard/iq/dataverse/mocks/MocksFactory.java: Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-surefire-plugin:2.22.0:test (default-test) @ dataverse ---
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[WARNING] Corrupted STDOUT by directly writing to native stream in forked JVM 1. See FAQ web page and the dump file /Users/pdurbin/NetBeansProjects/dataverse/target/surefire-reports/2018-09-28T10-23-39_304-jvmRun1.dumpstream
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 16.218 s
[INFO] Finished at: 2018-09-28T10:23:40-04:00
[INFO] Final Memory: 90M/310M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.0:test (default-test) on project dataverse: There are test failures.
[ERROR] 
[ERROR] Please refer to /Users/pdurbin/NetBeansProjects/dataverse/target/surefire-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream.
[ERROR] The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
[ERROR] Command was /bin/sh -c cd /Users/pdurbin/NetBeansProjects/dataverse && /Library/Java/JavaVirtualMachines/jdk-11.jdk/Contents/Home/bin/java -javaagent:/Users/pdurbin/.m2/repository/org/jacoco/org.jacoco.agent/0.7.5.201505241946/org.jacoco.agent-0.7.5.201505241946-runtime.jar=destfile=/Users/pdurbin/NetBeansProjects/dataverse/target/jacoco.exec -Duser.timezone=UTC -Dfile.encoding=UTF-8 -Duser.language=en -Duser.region=US -jar /Users/pdurbin/NetBeansProjects/dataverse/target/surefire/surefirebooter14148824480688595114.jar /Users/pdurbin/NetBeansProjects/dataverse/target/surefire 2018-09-28T10-23-39_304-jvmRun1 surefire11290752819963119907tmp surefire_08795585695108816698tmp
[ERROR] Error occurred in starting fork, check output in log
[ERROR] Process Exit Code: 134
[ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
[ERROR] Command was /bin/sh -c cd /Users/pdurbin/NetBeansProjects/dataverse && /Library/Java/JavaVirtualMachines/jdk-11.jdk/Contents/Home/bin/java -javaagent:/Users/pdurbin/.m2/repository/org/jacoco/org.jacoco.agent/0.7.5.201505241946/org.jacoco.agent-0.7.5.201505241946-runtime.jar=destfile=/Users/pdurbin/NetBeansProjects/dataverse/target/jacoco.exec -Duser.timezone=UTC -Dfile.encoding=UTF-8 -Duser.language=en -Duser.region=US -jar /Users/pdurbin/NetBeansProjects/dataverse/target/surefire/surefirebooter14148824480688595114.jar /Users/pdurbin/NetBeansProjects/dataverse/target/surefire 2018-09-28T10-23-39_304-jvmRun1 surefire11290752819963119907tmp surefire_08795585695108816698tmp
[ERROR] Error occurred in starting fork, check output in log
[ERROR] Process Exit Code: 134
[ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:671)
[ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:533)
[ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:278)
[ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:244)
[ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1194)
[ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1022)
[ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:868)
[ERROR] at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
[ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
[ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
[ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
[ERROR] at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
[ERROR] at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
[ERROR] at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
[ERROR] at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
[ERROR] at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
[ERROR] at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
[ERROR] at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
[ERROR] at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
[ERROR] at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
[ERROR] at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
[ERROR] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[ERROR] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR] at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
[ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
[ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
[ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
murphy:dataverse pdurbin$ 

We should get Dataverse compiling on Java 11 since it's the next LTS release after Java 8. According to https://www.theregister.co.uk/2018/09/26/oracle_java_11_binary_code_license/ "After January 2019, Oracle will no longer provide free updates to Java 8, which means shifting to a supported version of Java, relying on OS vendors to provide Java patches, paying a third-party for support, building the OpenJDK on your own, or getting builds from AdoptOpenJDK."

[poikilotherm]

Hi @pdurbin,

I just created #5116. If you want to try it out, go ahead :smile: My results are mixed.

To use my branch:

1. Install Java 11 first. Used dnf install java-11-openjdk-devel on my Fedora 28 Linux Dev Box :wink:
2. Run JAVA_HOME=<your_java_11_home> mvn -Pjava11 clean test

Watch unit tests fail at two tests currently:

[ERROR] Failures: 
[ERROR]   StorageIOTest.testGetDvObject:72 expected:<[class edu.harvard.iq.dataverse.Dataset cannot be cast to class edu.harvard.iq.dataverse.DataFile (edu.harvard.iq.dataverse.Dataset and edu.harvard.iq.dataverse.DataFile are in unnamed module of loader 'app')]> but was:<[edu.harvard.iq.dataverse.Dataset cannot be cast to edu.harvard.iq.dataverse.DataFile]>
[ERROR]   FileUtilTest.testDetermineFileType:194 expected:<[image/png]> but was:<[application/octet-stream]>
[ERROR] Tests run: 549, Failures: 2, Errors: 0, Skipped: 5

This seems to be related to a different message since JDK 8 and a change in handling of media files. Needs more evaluation though.

Valuable sources for the upgrade:

• https://winterbe.com/posts/2018/08/29/migrate-maven-projects-to-java-11-jigsaw/
• https://medium.com/criciumadev/its-time-migrating-to-java-11-5eb3868354f9
• https://www.jacoco.org/jacoco/trunk/doc/maven.html
• and some more on StackOverflow

[poikilotherm]

This ambiguity can be resolved by specifying edu.harvard.hul.ois.jhove.Module like this...

IMHO this is not a very clean approach. See my solution in cd5296d68, which I find to be more inline with good coding practise.

[pdurbin]

@poikilotherm thanks for the pull request and links. Yes, I was just trying to illustrate what "Module" is in our code. Let's go with your approach.

In my mind the next steps are:

• fix those two failing unit tests
• put a FIXME next to every reference to Java 8 in the docs and code to help us think about the scope of work. Vagrant? Docker? Installation Guide? OpenShift?
• actually make the changes to the docs and code

The thing is though, we're going to have to wait for CentOS or EPEL to provide Java 11 RPMs, right? (Are you following https://fedoraproject.org/wiki/Changes/java-11-openjdk-TechPreview#How_To_Test ?)

[poikilotherm]

wait for CentOS or EPEL to provide Java 11 RPMs

I think as soon as RHEL upstream will provide them, they will be in CentOS shortly after.

Both failing tests need debugging... :disappointed: Not easy to tell why this is not working as supposed.

[pdurbin]

@poikilotherm I went ahead and assigned this issue to you and dragged it to "community dev" at https://waffle.io/IQSS/dataverse

Once we decide the scope for this thing (Vagrant? Docker? OpenShift? Installation Guide?) let's rename the title of this issue from "Java 9 Upgrade" to something more reasonable. Or even before we decide the exact scope. Java 9 is EOL.

[poikilotherm]

Hey @pameyer, once the unit tests are all fixed, I will need to tackle the integration tests. Currently, those run in the docker-aio container using Java 8.

What makes more sense to you guys: try to

1. make a second container using Java 11 or
2. instead try to get things moving and use a Travis CI approach with a test matrix including both Java versions (We can even use Docker inside TravisCI jobs) or
3. try to come up with things like Arquillian Container Management plus Travis CI with different Java versions?

(As suggested by @pdurbin I'm pulling @donsizemore into this, too :smile: )

[pameyer]

@poikilotherm With the caveat that I think IQSS jenkins is still the authoritative "source of truth" for integration tests; I think it might make sense to start with another container for Java 11. But part of that is because I'm not too familiar with Travis CI or Arquillian - so if these are a simpler approach, I'm not opposed to it.

My (completely unsupported) suspicion is that the current state of Dataverse dockerization deviates sufficiently from best practices that there might be some impedance mismatch trying to use more sophisticated tooling. But technical evaluations beet unsupported suspicions if you think it's worth investigating.

[poikilotherm]

@pdurbin An update on this: I'm having a hard time getting the MimetypeFilesTypeMap working for the FileUtil class... Despite adding the dependency, this is constantly rejecting to work :disappointed:

(Also trying in a simple example project works flawless).

Will continue on this next week.

[pdurbin]

@poikilotherm thanks, I really appreciate the legwork on this. We don't want to be stuck on Java 8 forever.

[poikilotherm]

Alright, I think I found the reason why the MIME type detection based on javax.activation.MimetypesFileTypeMap is not working...

The abdera-core library used by SWORDv2 is depending on the geronimo-activation_1.1_spec as a compile time dependency, which is included in the final WAR.

As Java 11 does not have a javax.activation implementation onboard (has been removed from 11 as it is a Java EE component), this library is used instead. And as I can confirm from my simple test code, this causes the failing code. It seems like this library is not Java 11 compliant.

Using the upstream javax.activation implementation lets things work correctly again. I am not sure if it is a good idea to replace the jars inside the WAR and what unintended side effects may arise...

This took me about three hours to debug. I'm wondering if there might be other things, where no unit test exist and which will break once using Java 11.

So maybe it would be a good idea to come up with more tests first... :wink:

[pdurbin]

Oracle will stop giving out free security updates for Java 8 this month so I just switched to AdoptOpenJDK 1.8.0_192-b12 on my Mac and it works fine. I downloaded it from https://adoptopenjdk.net and untarred it to /Library/Java/JavaVirtualMachines/jdk8u192-b12 where my Mac and Netbeans picked up the change immediately.

Also, @landreev checked Harvard Dataverse are we are not using Oracle's Java any more. We get Java from yum (CentOS 7) so normal patching should keep Java 8 secure for us.

That said, installations of Dataverse will want to move to Java 11 some day, I assume. Does anyone know when? If so, please leave a comment. 😄

[pdurbin]

The summary from #5512 is that we are blocked on upgrading to Java 11 because it's not supported by Glassfish or Payara. From what I can tell JBoss got support for Java 11 only recently but so far we only use Glassfish.

[pdurbin]

I just noticed "GlassFish 5 is not supported on JDK 9+. There's hundreds of other problems to be solved in addition to this one. I'm closing this bug." at https://github.com/eclipse-ee4j/glassfish/issues/22130#issuecomment-461246868

[pdurbin]

I'm going to go out on a limb here and suggest that Payara will probably have support for Java 11 before Glassfish does. Payara has an (unmerged) branch they are encouraging users to test. Please see https://github.com/payara/Payara/issues/2296#issuecomment-478334106

[pdurbin]

Payara 5.192 just came out and supports Java 11 as a tech preview:

"At long last, Payara Platform is shipping with support for JDK 11. It should be stressed that firstly: this support is in tech preview and secondly: if upgrading from an older version of Payara, you'll need to add some extra JVM options to your domain.xml configuration files." https://github.com/payara/Payara/releases/tag/payara-server-5.192