search

Homebrew / homebrew-core

🍻 Default formulae for the missing package manager for macOS (or Linux)
https://brew.sh
BSD 2-Clause "Simplified" License
13.6k stars 12.35k forks source link

Java 9 Migration #19696

Closed ilovezfs closed 6 years ago

ilovezfs commented 6 years ago

This will serve as a tracking issue for the Java 9 migration.

Broken at run time (brew test fails or other breakage observed): (Check mark indicates it was either fixed or pinned to Java 8)

Broken at build time (brew install --build-from-source fails):

Works at run time (brew test passes if it has a test block) and build time status is unknown or not applicable (bottle unneeded) (NB: may still be broken if the formula has no test block):

Non-bottle-unneeded formulae with non-optional depends_on :java not yet using a hard Java 8 req:

ilovezfs commented 6 years ago

I've added the status of all bottle unneeded ones above. The bottle-needed ones still need to be assessed.

SethTisue commented 6 years ago

scala is already reasonably JDK9 friendly, as of last week's Scala 2.12.4 release.

scala@2.10 will be reasonably JDK9 friendly once Scala 2.10.7 becomes available (within a few weeks, we think). scala@2.11, ditto for Scala 2.11.12.

sbt will be reasonably JDK9 friendly in its upcoming 1.0.3 release and, in the sbt@0.13 line, in its 0.13.17 release. Neither release exists yet.

UPDATE: sbt 1.0.3 is out

ilovezfs commented 6 years ago

Thanks @SethTisue! That's very helpful.

botic commented 6 years ago

The earliest version of ringojs with JDK 9 support is likely 2.x (~ Q1/Q2 2018) since it depends on full support of Java 9 in Jetty and Rhino. So testing RingoJS with Java 8 would solve the issue. The broken test: https://jenkins.brew.sh/job/Homebrew%20Core%20Pull%20Requests/11720/

jrwiegand commented 6 years ago

Works great, thanks for all the work you did on this.

Moisan commented 6 years ago

javarepl issue is known upstream

yetisyny commented 6 years ago

libbluray should be listed, the current stable version 1.0.2 does not yet work with java 9 and homebrew is set to only let it work with java 8, although they are working on making the next stable version of libbluray compatible with java 9 and if you get a HEAD build it probably would work already (almost all of the recent commits have been related to java 9 compatibility but I am not sure if it is 100% compatible yet). You can try to see if it is compatible with the --HEAD option with libbluray in homebrew.

Anyway I would add this to your list which does not list libbluray. I am very sure that when the next version of libbluray comes out it will be java 9 compatible, after looking at their recent commit history and their history of releases as well as their mailing list. It should be added to the first list, with a checkmark next to it for being fixed, since it is working right now by being pinned to java 8. This way you can keep track of it along with all the other java-dependent formulae with issues with java 9.

ilovezfs commented 6 years ago

@yetisyny thanks. added.

yetisyny commented 6 years ago

Java is moving to rapid-release, with major releases every 6 months and long-term support releases every 3 years, a major change from the past. Java SE 8 will stop getting security updates in January 2019. Java SE 9 will stop getting updates even sooner, next month, in March 2018, when Java SE 10 (18.3) comes out and supersedes it, and again, in September 2018, Java SE 10 (18.3) will stop getting updates and Java SE 11 (18.9-LTS) will come out and supersede it. Java SE 11 (18.9-LTS) will be supported for 3 years most likely, probably until September 2021, so really we will want everything working on Java 11 since it will be out a lot longer than Java 9 or 10 are going to last.

For more information see http://www.oracle.com/technetwork/java/javase/eol-135779.html. Anyway my point here is, in just one month from now, Java 9 will be history and Java 10 will be out. And also Java 8, the version that pretty much everything is compatible with, is going to stop getting updates in January next year and then everyone will be forced to either update to one of these rolling-release versions that comes out every 6 months or a long-term support release like Java 11 which will be supported for 3 years.

So in other words, it’s a much faster release cycle starting next month and everybody will have to get things compatible with new Java versions every 6 months. Which means a lot more work for people keeping formulae that use Java compatible with the latest Java versions, so please get ready for it.

ilovezfs commented 6 years ago

We'll remove whatever doesn't work. There's nothing to get ready for.

apjanke commented 6 years ago

Well, darn. Here's apache-flink:

  depends_on :java => "1.7+"

We don't have a way of expressing "at least Java 1.7, but no greater than 1.8".

Part of me wants to say "just use '1.8'", because Java 1.7 is pretty old and obsolete at this point. (It went EOL for non-enterprise users way back in April 2015.) But that's not the part of me that's running older machines.

ilovezfs commented 6 years ago

@apjanke grep through core. These have all been handled so far with a hard Java 8 dep.

ilovezfs commented 6 years ago

Note that in the case of apache-flink that has proven insufficient, so it will soon be removed if it cannot coexist peacefully on a dual Java 8 / Java 9 system. See https://github.com/Homebrew/homebrew-core/pull/21823 https://github.com/Homebrew/homebrew-core/pull/21970 https://github.com/Homebrew/homebrew-core/pull/21596

apjanke commented 6 years ago

Sounds good.

I agree with a hard Java 8 dep. IMHO, Homebrew should not be supporting users running Java 1.7, since it is no longer receiving security updates.

ilovezfs commented 6 years ago

@apjanke FYI I added a new section above with the catchy title

Non-bottle-unneeded formulae with non-optional depends_on :java not yet using a hard Java 8 req and of unknown build status with Java 9:

apjanke commented 6 years ago

Ooh, more testing to do!

ilovezfs commented 6 years ago

@apjanke OK I ran all of those through test-bot and moved the broken ones up to the broken-at-build-time section, which were

abcl https://gist.github.com/ilovezfs/b5b17887667a596cfa6db0749b934a30 avian https://gist.github.com/ilovezfs/a8e728be2d946270589356ac3280d296 aws-apigateway-importer https://gist.github.com/ilovezfs/2138b6aac69c225ea2cac21957a85cac drip https://gist.github.com/ilovezfs/1903acf1061128a36a62a322779afa9e fastbit https://gist.github.com/ilovezfs/52d7066e66d1c847e7aac492fa6cb399 gssh https://gist.github.com/anonymous/5c1ed619783391b67a49f64ad5670426 henplus https://gist.github.com/ilovezfs/96a4a3ace9830a948456cfba6c0fa218 hexgui https://gist.github.com/ilovezfs/188016cce1af8d876c7e2f677c1b820b joshua (fetch fails for https://cs.jhu.edu/~post/language-packs/language-pack-es-en-phrase-2015-03-06.tgz) lcm https://gist.github.com/ilovezfs/1a193ee2a24f3467775a507cc47b9094 mysql-cluster https://gist.github.com/ilovezfs/7c211da8b2574e51de9fbe511b716e61 opentsdb https://gist.github.com/ilovezfs/5b4c6c9a21c4eb10386606ee439549ad pillar https://gist.github.com/ilovezfs/566c3231c5646c0cb3d5d18332f351e3 upscaledb https://gist.github.com/ilovezfs/9d54c270a47a7108c64c370ecad84922 voldemort https://gist.github.com/ilovezfs/8aedf73017211e085e6c1c79c8a5bb0f wartremover #https://gist.github.com/ilovezfs/70941f8b35ade0935db86c9708f137b2

yetisyny commented 6 years ago

Java SE 10 has been out for almost a week, since March 20th, and Java SE 9 is no longer supported AT ALL, with all Java SE 9 users being automatically updated to Java SE 10, at least for the JRE (maybe not with the JDK, I think JDK still requires manual updates).

Java SE 8 continues to have public security updates, so basically the 2 versions of Java that are out now that it is important to check for compatibility with are 8 and 10. See http://www.oracle.com/technetwork/java/eol-135779.html.

I suggest that it’s probably time to close this tracking bug and open a new one for Java SE 10.

ilovezfs commented 6 years ago

@yetisyny we know, but thanks for the heads up.

Jahhein commented 6 years ago

@yetisyny In a few more weeks it will no longer be called Java. Java EE rebranded to Jakarta EE

The deal with oracle is the same with Java SE. Don't call it Java anymore.

apjanke commented 6 years ago

apache-flink is still broken under apache-flink 1.5.0, but it looks like a simple fix in its startup wrapper scripts. They're just using a raw java call instead of using $JAVA_HOME in one place. Upstream bug report: https://issues.apache.org/jira/browse/FLINK-9445

apjanke commented 6 years ago

It looks like embulk is fixed now, by 4219b20e5b14c653deafe6b75395d92138adb3a6.

apjanke commented 6 years ago

Following up on ecj and #24029: What do you all think about just dropping ecj and its Java support in the earlier GCCs (gcc@5, gcc@6, gcc@4.9)? Vendoring it in wouldn't be very hard, but given the low usage (<1% of users install ecj, per https://github.com/Homebrew/homebrew-core/pull/24029#issuecomment-365229689) it doesn't seem worth the maintenance effort to me. I'd be kinda surprised if any significant number of people were actually using the old GCC Java support, instead of just picking it up with an optimistic --with-all-languages. I'm in favor of dropping it, and will put in a PR to do so if you're amenable.

apjanke commented 6 years ago

Are you sure fastbit is due to a JDK 9 issue? The error in your Gist doesn't look Java-related to me:

checking if compiler supports __sync_add_and_fetch for 64-bit integers... yes
checking if the default FlexLexer.h defines yywrap... configure: error: no, must have FlexLexer.h to continue

... and brew install -s fastbit works fine on my macOS 10.11 machine with JDK 9 and 10 installed.

apjanke commented 6 years ago

joshua's URL has been fixed; brew install -s joshua works for me with JDK 9 & 10 installed; it can be removed from this list.

apjanke commented 6 years ago

Fixes incoming:

abcl: #28312 avian: #28313 drip: #28314 aws-apigateway-importer: #28315 gssh: #28316 henplus: #28317 hexgui: #28318 lcm: #28319 mysql-cluster: #28320

These all address the issue by pinning it to Java 8 specifically, either at build or run time or both. I think that's really the only way to go at this point, until upstreams all make the leap to Java 9/10/11.

apjanke commented 6 years ago

jooby-bootstrap: #28311

apjanke commented 6 years ago

upscaledb works for me now, on a 10.13 system with Java 8 & 9 & 10.

Looks to me like the build error here was not Java related, and it can probably be removed from the list:

In file included from 3changeset/changeset.cc:18:
./2worker/worker.h:83:16: error: use of class template 'boost::asio::strand' requires template arguments
  boost::asio::strand strand;
               ^
/usr/local/include/boost/asio/strand.hpp:29:7: note: template is declared here
class strand
      ^
1 error generated.
make[2]: *** [3changeset/changeset.lo] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [install-recursive] Error 1
make: *** [install-recursive] Error 1
apjanke commented 6 years ago

Fixes incoming:

pillar: #28321 voldemort: #28322 wartremover: #28323

That's all the low-hanging fruit.

ilovezfs commented 6 years ago

I think fastbit was fixed in https://github.com/Homebrew/homebrew-core/pull/25228 and unrelated to Java.

Thanks for the all the awesome work @apjanke!

So I think we have outstanding problems with apache-flink, ecj, and opentsdb?

apjanke commented 6 years ago

You're welcome!

hbase needs to go back on the "to do" list, too. Its own install and test succeed. But it emits warnings indicating it's not really ready for Java 9.


$ start-hbase.sh
Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
starting master, logging to /usr/local/var/log/hbase/hbase-janke-master-angharad.local.out
Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Java HotSpot(TM) 64-Bit Server VM warning: Ignoring option PermSize; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: Ignoring option MaxPermSize; support was removed in 8.0

And when actually exercising it, as is done in the opentsdb post_install, it throws Java 9 related errors.

$ brew install -s opentsdb
==> Downloading https://github.com/OpenTSDB/opentsdb/releases/download/v2.3.0/opentsdb-2.3.0.tar.gz
Already downloaded: /Users/janke/Library/Caches/Homebrew/opentsdb-2.3.0.tar.gz
==> ./configure --disable-silent-rules --prefix=/usr/local/Cellar/opentsdb/2.3.0_1 --mandir=/usr/local/Cellar/opentsdb/2.
==> make
==> make install
==> /usr/local/opt/hbase/bin/start-hbase.sh
==> /usr/local/Cellar/opentsdb/2.3.0_1/share/opentsdb/tools/create_table_with_env.sh
Last 15 lines from /Users/janke/Library/Logs/Homebrew/opentsdb/post_install.02.create_table_with_env.sh:

Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.jruby.java.invokers.RubyToJavaInvoker (file:/usr/local/Cellar/hbase/1.2.6/libexec/lib/jruby-complete-1.6.8.jar) to method java.lang.Object.registerNatives()
WARNING: Please consider reporting this to the maintainers of org.jruby.java.invokers.RubyToJavaInvoker
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
ArgumentError: wrong number of arguments (0 for 1)
  method_added at file:/usr/local/Cellar/hbase/1.2.6/libexec/lib/jruby-complete-1.6.8.jar!/builtin/javasupport/core_ext/object.rb:10
  method_added at file:/usr/local/Cellar/hbase/1.2.6/libexec/lib/jruby-complete-1.6.8.jar!/builtin/javasupport/core_ext/object.rb:129
       Pattern at file:/usr/local/Cellar/hbase/1.2.6/libexec/lib/jruby-complete-1.6.8.jar!/builtin/java/java.util.regex.rb:2
        (root) at file:/usr/local/Cellar/hbase/1.2.6/libexec/lib/jruby-complete-1.6.8.jar!/builtin/java/java.util.regex.rb:1
       require at org/jruby/RubyKernel.java:1062
        (root) at file:/usr/local/Cellar/hbase/1.2.6/libexec/lib/jruby-complete-1.6.8.jar!/builtin/java/java.util.regex.rb:42
        (root) at /usr/local/opt/hbase/libexec/bin/../bin/hirb.rb:38
==> /usr/local/opt/hbase/bin/stop-hbase.sh

Note that those are coming from the hbase Java code, not opentsdb itself.

apjanke commented 6 years ago

hbase: #28346 opentsdb: #28367 apache-flink: in progress: #28342 ecj: I say we delete it.

And that's everything!

ilovezfs commented 6 years ago

ecj deleted!

I'm closing this out since we have open PRs for all the outstanding problems. Thanks for the help everyone, and especially @apjanke, the MVP!

apjanke commented 6 years ago

🎉