Difficulty installing reach through sbt

djinnome commented 3 years ago

Hi folks,

Apologies for the newbie questions.

I followed these directions for installing a local version of reach here:

brew install sbt
git clone https://github.com/clulab/reach.git
cd reach
sbt 'runMain org.clulab.reach.export.server.ApiServer'

But I got this error message:

OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
copying runtime jar...
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[info] [launcher] getting org.scala-sbt sbt 1.4.0  (this may take some time)...
[info] [launcher] getting Scala 2.13.3 (for sbt)...
java.lang.NoClassDefFoundError: scala/Serializable
    at sbt.internal.ClassLoaderWarmup$.warmup(XMainConfiguration.scala:31)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:567)
    at sbt.internal.XMainConfiguration.run(XMainConfiguration.scala:82)
    at sbt.xMain.run(Main.scala:46)
    at xsbt.boot.Launch$.$anonfun$run$1(Launch.scala:149)
    at xsbt.boot.Launch$.withContextLoader(Launch.scala:176)
    at xsbt.boot.Launch$.run(Launch.scala:149)
    at xsbt.boot.Launch$.$anonfun$apply$1(Launch.scala:44)
    at xsbt.boot.Launch$.launch(Launch.scala:159)
    at xsbt.boot.Launch$.apply(Launch.scala:44)
    at xsbt.boot.Launch$.apply(Launch.scala:21)
    at xsbt.boot.Boot$.runImpl(Boot.scala:78)
    at xsbt.boot.Boot$.run(Boot.scala:73)
    at xsbt.boot.Boot$.main(Boot.scala:21)
    at xsbt.boot.Boot.main(Boot.scala)
Caused by: java.lang.ClassNotFoundException: scala.Serializable
    at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:436)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
    ... 18 more
[error] [launcher] error during sbt launcher: java.lang.NoClassDefFoundError: scala/Serializable

I noticed that sbt downloaded Scala 2.13.3 instead of Scala 2.12.8 as specified in reach/build.sbt, but I think the problem is even more basic, because I get the same error even if I check the version of sbt:

$ sbt --version

results in:

OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
java.lang.NoClassDefFoundError: scala/Serializable
    at sbt.internal.ClassLoaderWarmup$.warmup(XMainConfiguration.scala:31)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:567)
    at sbt.internal.XMainConfiguration.run(XMainConfiguration.scala:82)
    at sbt.xMain.run(Main.scala:46)
    at xsbt.boot.Launch$.$anonfun$run$1(Launch.scala:149)
    at xsbt.boot.Launch$.withContextLoader(Launch.scala:176)
    at xsbt.boot.Launch$.run(Launch.scala:149)
    at xsbt.boot.Launch$.$anonfun$apply$1(Launch.scala:44)
    at xsbt.boot.Launch$.launch(Launch.scala:159)
    at xsbt.boot.Launch$.apply(Launch.scala:44)
    at xsbt.boot.Launch$.apply(Launch.scala:21)
    at xsbt.boot.Boot$.runImpl(Boot.scala:78)
    at xsbt.boot.Boot$.run(Boot.scala:73)
    at xsbt.boot.Boot$.main(Boot.scala:21)
    at xsbt.boot.Boot.main(Boot.scala)
Caused by: java.lang.ClassNotFoundException: scala.Serializable
    at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:436)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
    ... 18 more
[error] [launcher] error during sbt launcher: java.lang.NoClassDefFoundError: scala/Serializable
sbt script version: 1.5.4

djinnome commented 3 years ago

Same problem even after I install openjdk 11 through homebrew:

$ brew install openjdk@11
$ sbt

still results in:

OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
java.lang.NoClassDefFoundError: scala/Serializable
    at sbt.internal.ClassLoaderWarmup$.warmup(XMainConfiguration.scala:31)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:567)
    at sbt.internal.XMainConfiguration.run(XMainConfiguration.scala:82)
    at sbt.xMain.run(Main.scala:46)
    at xsbt.boot.Launch$.$anonfun$run$1(Launch.scala:149)
    at xsbt.boot.Launch$.withContextLoader(Launch.scala:176)
    at xsbt.boot.Launch$.run(Launch.scala:149)
    at xsbt.boot.Launch$.$anonfun$apply$1(Launch.scala:44)
    at xsbt.boot.Launch$.launch(Launch.scala:159)
    at xsbt.boot.Launch$.apply(Launch.scala:44)
    at xsbt.boot.Launch$.apply(Launch.scala:21)
    at xsbt.boot.Boot$.runImpl(Boot.scala:78)
    at xsbt.boot.Boot$.run(Boot.scala:73)
    at xsbt.boot.Boot$.main(Boot.scala:21)
    at xsbt.boot.Boot.main(Boot.scala)
Caused by: java.lang.ClassNotFoundException: scala.Serializable
    at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:436)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
    ... 18 more
[error] [launcher] error during sbt launcher: java.lang.NoClassDefFoundError: scala/Serializable

kwalcock commented 3 years ago

Hi. Thanks for the detective work. I don't think that sbt understands --version, but it is supposed to understand about and sbtVersion. Do those give errors?

$ sbt about
$ sbt sbtVersion

I was thinking that the Java version might be the culprit. We usually work with 8 for Scala programs.

djinnome commented 3 years ago

To downgrade to Java 8, I did this:

$ brew install openjdk@8

Still get the same error:

$ sbt about
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
java.lang.NoClassDefFoundError: scala/Serializable
    at sbt.internal.ClassLoaderWarmup$.warmup(XMainConfiguration.scala:31)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:567)
    at sbt.internal.XMainConfiguration.run(XMainConfiguration.scala:82)
    at sbt.xMain.run(Main.scala:46)
    at xsbt.boot.Launch$.$anonfun$run$1(Launch.scala:149)
    at xsbt.boot.Launch$.withContextLoader(Launch.scala:176)
    at xsbt.boot.Launch$.run(Launch.scala:149)
    at xsbt.boot.Launch$.$anonfun$apply$1(Launch.scala:44)
    at xsbt.boot.Launch$.launch(Launch.scala:159)
    at xsbt.boot.Launch$.apply(Launch.scala:44)
    at xsbt.boot.Launch$.apply(Launch.scala:21)
    at xsbt.boot.Boot$.runImpl(Boot.scala:78)
    at xsbt.boot.Boot$.run(Boot.scala:73)
    at xsbt.boot.Boot$.main(Boot.scala:21)
    at xsbt.boot.Boot.main(Boot.scala)
Caused by: java.lang.ClassNotFoundException: scala.Serializable
    at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:436)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
    ... 18 more
[error] [launcher] error during sbt launcher: java.lang.NoClassDefFoundError: scala/Serializable

djinnome commented 3 years ago

I'm suspect I am falling prey to some newbie Java trap. Maybe a CLASSPATH issue?

kwalcock commented 3 years ago

Does java -version at least work OK? A google search for sbt internal.ClassLoaderWarmup$.warmup(XMainConfiguration.scala:31) turns up some hits, so you aren't the first person to get the error. It might not be directly related to reach. Does a simple sbt command (and then "exit" to get out of it) work in the reach directory or an empty directory?

djinnome commented 3 years ago

Apparently (part of) the problem was using brew install sbt

Homebrew maintainers have added a dependency to JDK 13 because they want to use more brew dependencies

(brew#50649). This causes sbt to use JDK 13 even when java available on PATH is JDK 8 or 11.

To prevent sbt from running on JDK 13, install jEnv or switch to using SDKMAN.

I downloaded the sbt tarball, set my PATH, and in every directory but reach I can do this:

$ which sbt
/usr/local/share/sbt/bin/sbt
$ jenv version
1.8 (set by JENV_VERSION environment variable)
$ sbt sbtVersion
[info] welcome to sbt 1.5.4 (Oracle Corporation Java 1.8.0_282)
[info] loading project definition from /Users/zuck016/Projects/CausalInference/MAA/project
[info] set current project to maa (in build file:/Users/zuck016/Projects/CausalInference/MAA/)
[info] 1.5.4

But as soon as I enter the reach directory, the same problem occurs:

$ git clone https://github.com/clulab/reach.git
$ cd reach
$ which sbt
/usr/local/share/sbt/bin/sbt
$ jenv version
1.8 (set by JENV_VERSION environment variable)
$ sbt sbtVersion
java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError: scala/Serializable
    at sbt.internal.ClassLoaderWarmup$.warmup(XMainConfiguration.scala:31)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at sbt.internal.XMainConfiguration.run(XMainConfiguration.scala:82)
    at sbt.xMain.run(Main.scala:46)
    at xsbt.boot.Launch$.$anonfun$run$1(Launch.scala:149)
    at xsbt.boot.Launch$.withContextLoader(Launch.scala:176)
    at xsbt.boot.Launch$.run(Launch.scala:149)
    at xsbt.boot.Launch$.$anonfun$apply$1(Launch.scala:44)
    at xsbt.boot.Launch$.launch(Launch.scala:159)
    at xsbt.boot.Launch$.apply(Launch.scala:44)
    at xsbt.boot.Launch$.apply(Launch.scala:21)
    at xsbt.boot.Boot$.runImpl(Boot.scala:78)
    at xsbt.boot.Boot$.run(Boot.scala:73)
    at xsbt.boot.Boot$.main(Boot.scala:21)
    at xsbt.boot.Boot.main(Boot.scala)
Caused by: java.lang.NoClassDefFoundError: scala/Serializable
    ... 18 more
Caused by: java.lang.ClassNotFoundException: scala.Serializable
    at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
    ... 18 more
[error] [launcher] error during sbt launcher: java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError: scala/Serializable

Any clue what could be going on?

kwalcock commented 3 years ago

It seems close. I wonder if sbt has cached some things that it now has trouble reading because in that directory there is a real project. sbt clean probably doesn't work. Are there any target directories that can be erased or can a clean clone be made from github? I hope a real Mac user can chime in. We have plenty of them. I see in project/build.properties that reach wants sbt 1.4.0. If some other sbt is installed, it gets reconfigured somehow to act like or use this version. It may be that 1.5.4 as used in other directories is working but 1.4.0 is not. Installing the lower version would save some translating which could be going awry.

djinnome commented 3 years ago

OK, check this out:

$ cd /usr/local/share
$ rm -rf sbt
$ wget -c https://github.com/sbt/sbt/releases/download/v1.4.0/sbt-1.4.0.tgz
$ tar xzvf sbt-1.4.0.tgz
$ /usr/local/share/sbt/bin/sbt sbtVersion
[info] welcome to sbt 1.5.4 (Oracle Corporation Java 1.8.0_282)
[info] loading project definition from /usr/local/share/project
[info] set current project to share (in build file:/usr/local/share/)
[info] 1.5.4

Why does it think that version 1.4.0 is actually 1.5.4?

And it still fails in the reach directory even when I perform a clean clone:

$ rm -rf reach
$ git clone https://github.com/clulab/reach.git
$ cd reach
$ /usr/local/share/sbt/bin/sbt sbtVersion
java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError: scala/Serializable
    at sbt.internal.ClassLoaderWarmup$.warmup(XMainConfiguration.scala:31)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at sbt.internal.XMainConfiguration.run(XMainConfiguration.scala:82)
    at sbt.xMain.run(Main.scala:46)
    at xsbt.boot.Launch$$anonfun$run$1.apply(Launch.scala:111)
    at xsbt.boot.Launch$.withContextLoader(Launch.scala:131)
    at xsbt.boot.Launch$.run(Launch.scala:111)
    at xsbt.boot.Launch$$anonfun$apply$1.apply(Launch.scala:37)
    at xsbt.boot.Launch$.launch(Launch.scala:120)
    at xsbt.boot.Launch$.apply(Launch.scala:20)
    at xsbt.boot.Boot$.runImpl(Boot.scala:56)
    at xsbt.boot.Boot$.main(Boot.scala:18)
    at xsbt.boot.Boot.main(Boot.scala)
Caused by: java.lang.NoClassDefFoundError: scala/Serializable
    ... 16 more
Caused by: java.lang.ClassNotFoundException: scala.Serializable
    at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
    ... 16 more
[error] [launcher] error during sbt launcher: java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError: scala/Serializable

Interestingly, the same phenotype occurs when I run sbt in the /usr/local/share/sbt directory

$ cd /usr/local/share
$ /usr/local/share/sbt/bin/sbt sbtVersion
 sbt/bin/sbt sbtVersion
[info] welcome to sbt 1.5.4 (Oracle Corporation Java 1.8.0_282)
[info] loading project definition from /usr/local/share/project
[info] set current project to share (in build file:/usr/local/share/)
[info] 1.5.4
$ cd /usr/local/share/sbt
$ /usr/local/share/sbt/bin/sbt sbtVersion
java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError: scala/Serializable
    at sbt.internal.ClassLoaderWarmup$.warmup(XMainConfiguration.scala:31)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at sbt.internal.XMainConfiguration.run(XMainConfiguration.scala:82)
    at sbt.xMain.run(Main.scala:46)
    at xsbt.boot.Launch$$anonfun$run$1.apply(Launch.scala:111)
    at xsbt.boot.Launch$.withContextLoader(Launch.scala:131)
    at xsbt.boot.Launch$.run(Launch.scala:111)
    at xsbt.boot.Launch$$anonfun$apply$1.apply(Launch.scala:37)
    at xsbt.boot.Launch$.launch(Launch.scala:120)
    at xsbt.boot.Launch$.apply(Launch.scala:20)
    at xsbt.boot.Boot$.runImpl(Boot.scala:56)
    at xsbt.boot.Boot$.main(Boot.scala:18)
    at xsbt.boot.Boot.main(Boot.scala)
Caused by: java.lang.NoClassDefFoundError: scala/Serializable
    ... 16 more
Caused by: java.lang.ClassNotFoundException: scala.Serializable
    at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
    ... 16 more
[error] [launcher] error during sbt launcher: java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError: scala/Serializable

bgyori commented 3 years ago

Hi @djinnome, a while ago I added a Dockerfile for Reach that might be easier to get working: if you have the reach repo cloned, you can build it as

cd docker
docker build --tag reach:latest .

and then run it as

docker run -d -it -p 8080:8080 reach:latest

You can then interact with it the same way as a natively running local Reach web service. Does that work for you?

kwalcock commented 3 years ago

Thanks, Ben! djinnome, the only explanation I can think of is that either the path or some brew thing is interfering. My impression is that sbt is only a shell program that calls up the real sbt (i.e., runs java on some jar file) and it may be be getting the wrong version number and compatibility issues from the wrong jar file or wrong java.

djinnome commented 3 years ago

Hi @kwalcock

I think sbt 1.4.0 is the issue here, according to https://github.com/sbt/sbt/issues/6587#issuecomment-877714299 Any chance you can upgrade to a later version of sbt?

djinnome commented 3 years ago

Hi @bgyori

I tried the instructions you provided, but I got some kind of authentication error.

$ cd docker
docker build --tag reach:latest .

<snip>

------
executor failed running [/bin/sh -c add-apt-repository -y ppa:openjdk-r/ppa &&     echo "deb https://dl.bintray.com/sbt/debian /" | tee -a /etc/apt/sources.list.d/sbt.list &&     apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 2EE0EA64E40A89B84B2DF73499E82A75642AC823 &&     apt-get update]: exit code: 100
(anaconda3-2020.11) bash-3.2$ docker run -d -it -p 8080:8080 reach:latest
Unable to find image 'reach:latest' locally
docker: Error response from daemon: pull access denied for reach, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
See 'docker run --help'.

Full logfile attached: docker.log

djinnome commented 3 years ago

Good news! I changed project/build.properties to use sbt 1.5.4 and it is now running the server locally.

Thanks everyone for the helpful hints!

kwalcock commented 3 years ago

Congratulations! Thanks or sticking it out. We'll see what it takes to update everyone to that version.

djinnome commented 3 years ago

Everyone else may not need to upgrade all the way to sbt 1.5.4. @eed3si9n seemed to think the issue was resolved by sbt 1.4.3

kwalcock commented 3 years ago

Thanks for the tip.

clulab / reach

Difficulty installing reach through sbt #758