foursquare / fsqio

A monorepo that holds all of Foursquare's opensource projects
Apache License 2.0
254 stars 54 forks source link

[error] invalid source release: 1.8 #14

Closed blackmad closed 8 years ago

blackmad commented 8 years ago

running "./pants pom-resolve" from a clean checkout

               [1/8] Compiling 8 zinc sources in 1 target (src/jvm/io/fsq/spindle/common/thrift/base:base).INFO] killing nailgun server pid=12192

08:22:56 00:01 [zinc] ==== stderr ====

                   ==== stdout ====
                   [info] Compiling 8 Java sources to /home/blackmad/Code/fsqio/.pants.d/compile/zinc/252d64521cf9/src.jvm.io.fsq.spindle.common.thrift.base.base/current/classes...
                   [error] invalid source release: 1.8

08:22:56 00:01 [compile] compile(src/jvm/io/fsq/spindle/common/thrift/base:base) failed: Zinc compile failed. FAILURE: Compilation failure: Failed jobs: compile(src/jvm/io/fsq/spindle/common/thrift/base:base)

08:22:57 00:02 [complete] FAILURE

FAILURE

08:22:57 00:02 [complete]

mateor commented 8 years ago

Hi! So, sorry this error is not very helpful - it comes from upstream Pants (and in turn from the compiler). It is indicating that you are not using JDK 8. The requirements for Fsq.io are here: https://github.com/foursquare/fsqio#requirements

I am about to push the Travis CI hook so there will be a working example about how to run this from non-Foursquare machines. It also occurs to me that I could catch the JDK mismatch in the Pants wrapper since it is a hard requirement. I will try and follow up with that if possible.

I am closing the issue but feel free to reopen if this is a false diagnosis.

blackmad commented 8 years ago

failing even with jdk8

[blackmad@ns4005169 fsqio (master)]$ java -version openjdk version "1.8.0_91" OpenJDK Runtime Environment (build 1.8.0_91-8u91-b14-0ubuntu4~14.04-b14) OpenJDK 64-Bit Server VM (build 25.91-b14, mixed mode)

On Sat, Jun 18, 2016 at 6:19 PM Mateo Rodriguez notifications@github.com wrote:

Hi! So, sorry this error is not very helpful - it comes from upstream Pants (and in turn from the compiler). It is indicating that you are not using JDK 8. The requirements for Fsq.io are here: https://github.com/foursquare/fsqio#requirements

I am about to push the Travis CI hook so there will be a working example about how to run this from non-Foursquare machines. It also occurs to me that I could catch the JDK mismatch in the Pants wrapper since it is a hard requirement. I will try and follow up with that if possible.

I am closing the issue but feel free to reopen if this is a false diagnosis.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/foursquare/fsqio/issues/14#issuecomment-226957162, or mute the thread https://github.com/notifications/unsubscribe/AAbMsFmLMAWa36NQIHtR9CL1ki3taLqoks5qNDaegaJpZM4I48AG .

mateor commented 8 years ago

Hmm...that is a bit surprising. But the JDK resolution in upstream Pants haas been changing a lot - it is quite possible that we have not kept up and now allow a tool to inadvertantly remain unpinned. Fsq.io is also mostly being run on OSX.

I am going to spin up a VM and see if I can repro. I am independently pushing some stricter JDK rules as part of the Travis hook so this will be a nice chance to see those in action.

mateor commented 8 years ago

By chance the only Ubuntu image I had around was your exact version of the 14.04 server. So I had a perfect lab to try and repro. I spun up a fresh image and installed openjdk-7-jdk so that I could trigger the repro.

  sudo apt-get install python-dev postgresql mongodb build-essential libpq-dev git-core
  sudo apt-get install openjdk-7-jdk-headless
  git clone https://github.com/foursquare/fsqio
  cd fsqio
  ./pants pom-resolve

This triggers your repro above, as expected. So all well and good. Then I installed the openjdk-8 by installing the OpenJDK ppa, getting the package and setting the update-alternatives.

  sudo add-apt-repository ppa:openjdk-r/ppa
  sudo apt-get install openjdk-8-jdk-headless
  sudo update-alternatives --config java                 # choose JDK 8
  sudo update-alternatives --config javac               # choose JDK8
  ./pants pom-resolve

In this run, everything worked as did the followup ./pants compile. I did not need to clean- we roll the jvm information into the fingerprint system. So I heavily suspect that Pants is just using the wrong java verision - your setup may subtly conflict with the JDK resolution scheme.

As I said, I am also about to push more restrictive JDK resolution rules which the following test suggets should help. I used update-alternatives to reset java and javac to openjdk7 and applied the following patch:

  diff --git a/pants.ini b/pants.ini
  index febf8be..bba7e36 100644
  --- a/pants.ini
  +++ b/pants.ini
  @@ -205,6 +205,9 @@ write_to = []
   [jvm]
   debug_args: %(debug_opts)s

  +[jvm-distributions]
  +minimum_version: 1.8.0
  +maximum_version: 1.8.999

   [jar-tool]
   jvm_options: ['-Xms2g','-Xmx2g', '-Djava.io.tmpdir=%(pants_workdir)s']

This forces Pants to use only JDKs that fit the distribution limits. It requires that you have a JDK8 on disk, which we know you have. That allowed me to have my system set to use JDK7 but Fsq.io itself used the JDK8 I had installed elsewhere on the path.

I intend to land that patch in Fsqio pretty soon here, so you could wait for that or just try it yourself.

Pants is basically just parsing your $PATH so it may just be misidentifying your local version of java for any number ofenv-related reasons. Falling back to using update-alternatives (which just symlinks /usr/bin/java` AFAIK) worked for me.

mateor commented 8 years ago

I pushed the commit that pins to the range (the patch above) so you can consume it by pulling. I am hopeful that it will solve your problem.

c4d9f8d43cd98214ef015a40f0fcc1a8cff3ab86

I also turned on the Travis CI so we have a sanity check there: https://github.com/foursquare/fsqio/blob/master/pants-travis-ci.ini

blackmad commented 8 years ago

thanks! fixed!