java.lang.UnsatisfiedLinkError: Can't load library: /usr/lib/jvm/java-11-openjdk-amd64/lib/libawt_xawt.so

[niloc132]

2021-12-28T17:00:05.701Z | main                 |  INFO | .u.GroovyDeephavenSession | Adding persistent import (static/wildcard): "import static io.deephaven.plot.PlottingConvenience.*;" from original string: "import static io.deephaven.plot.PlottingConvenience.*;"
Initiating shutdown due to: Uncaught exception in thread main
java.lang.UnsatisfiedLinkError: Can't load library: /usr/lib/jvm/java-11-openjdk-amd64/lib/libawt_xawt.so
        at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2630)
        at java.base/java.lang.Runtime.load0(Runtime.java:768)
        at java.base/java.lang.System.load(System.java:1837)
        at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method)
        at java.base/java.lang.ClassLoader$NativeLibrary.load(ClassLoader.java:2442)
        at java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(ClassLoader.java:2498)
        at java.base/java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2694)
        at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2648)
        at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:830)
        at java.base/java.lang.System.loadLibrary(System.java:1873)
        at java.desktop/java.awt.Toolkit$3.run(Toolkit.java:1399)
        at java.desktop/java.awt.Toolkit$3.run(Toolkit.java:1397)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at java.desktop/java.awt.Toolkit.loadLibraries(Toolkit.java:1396)
        at java.desktop/java.awt.Toolkit.<clinit>(Toolkit.java:1429)
        at java.desktop/java.awt.Color.<clinit>(Color.java:275)
        at io.deephaven.gui.color.Color.<init>(Color.java:45)
        at io.deephaven.gui.color.Color.color(Color.java:162)
        at io.deephaven.gui.color.Color.<clinit>(Color.java:402)
        at java.base/java.lang.Class.forName0(Native Method)
        at java.base/java.lang.Class.forName(Class.java:398)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.lambda$createCallStaticSite$2(CallSiteArray.java:65)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.createCallStaticSite(CallSiteArray.java:63)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.createCallSite(CallSiteArray.java:156)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:130)
        at io.deephaven.dynamic.groovy_3_plotting_5.run(groovy_3_plotting_5.groovy:48)
        at groovy.lang.GroovyShell.evaluate(GroovyShell.java:427)
        at groovy.lang.GroovyShell.evaluate(GroovyShell.java:461)
        at groovy.lang.GroovyShell.evaluate(GroovyShell.java:436)
        at io.deephaven.engine.util.GroovyDeephavenSession.evaluateCommand(GroovyDeephavenSession.java:222)
        at io.deephaven.engine.util.GroovyDeephavenSession.lambda$evaluate$2(GroovyDeephavenSession.java:245)
        at io.deephaven.util.locks.FunctionalLock.doLockedInterruptibly(FunctionalLock.java:46)
        at io.deephaven.engine.util.GroovyDeephavenSession.evaluate(GroovyDeephavenSession.java:245)
        at io.deephaven.engine.util.AbstractScriptSession.evaluateScript(AbstractScriptSession.java:111)
        at io.deephaven.engine.util.GroovyDeephavenSession.runScript(GroovyDeephavenSession.java:161)
        at io.deephaven.engine.util.GroovyDeephavenSession.<init>(GroovyDeephavenSession.java:142)
        at io.deephaven.server.console.groovy.GroovyConsoleSessionModule.bindGroovySession(GroovyConsoleSessionModule.java:26)
        at io.deephaven.server.console.groovy.GroovyConsoleSessionModule_BindGroovySessionFactory.bindGroovySession(GroovyConsoleSessionModule_BindGroovySessionFactory.java:46)
        at io.deephaven.server.console.groovy.GroovyConsoleSessionModule_BindGroovySessionFactory.get(GroovyConsoleSessionModule_BindGroovySessionFactory.java:35)
        at io.deephaven.server.console.groovy.GroovyConsoleSessionModule_BindGroovySessionFactory.get(GroovyConsoleSessionModule_BindGroovySessionFactory.java:10)
        at io.deephaven.server.console.groovy.GroovyConsoleSessionModule_BindScriptSessionFactory.get(GroovyConsoleSessionModule_BindScriptSessionFactory.java:31)
        at io.deephaven.server.console.groovy.GroovyConsoleSessionModule_BindScriptSessionFactory.get(GroovyConsoleSessionModule_BindScriptSessionFactory.java:10)
        at io.deephaven.server.console.ConsoleServiceGrpcImpl.initializeGlobalScriptSession(ConsoleServiceGrpcImpl.java:119)
        at io.deephaven.server.runner.DeephavenApiServer.run(DeephavenApiServer.java:101)
        at io.deephaven.server.jetty.JettyMain.main(JettyMain.java:29)

[niloc132]

cc @mofojed

[niloc132]

This is caused by the dependency on our Plotting libraries to java.awt.Color and friends - some JVM installations are "headless" and do not provide awt, and server startup fails with plotting enabled.

This might only happen right away with groovy, more testing would be a good idea.

As a simple workaround, installing a non-headless JVM will get past this issue.

[devinrsmith]

It surprises me that we haven't hit this before. Ostensibly, we are installed the headless jvm in our docker images:

https://github.com/deephaven/deephaven-base-images/blob/84859e93d85440caddc4fae6773d021e40f28f48/server/01-java/install.sh#L22-L23

[niloc132]

Our docker container indeed doesn't hit this, but it has these:

/usr/lib/jvm/java-11-openjdk-amd64/lib/libawt.so
/usr/lib/jvm/java-11-openjdk-amd64/lib/libawt_headless.so

My read is that libawt.so is what the jvm loads as part of this, and Bender's jvm for some reason was loading libawt_xawt.so, perhaps because he was already running X, unlike our docker container.

I confirmed this by connecting the docker container to a running X instance and trying to run the same thing, roughly:

$ docker run -it --rm -e DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix --user="$(id --user):$(id --group)" --entrypoint jshell deephaven/server:local-build
Dec 28, 2021 7:16:39 PM java.util.prefs.FileSystemPreferences$1 run
WARNING: Couldn't create user preferences directory. User preferences are unusable.
Dec 28, 2021 7:16:39 PM java.util.prefs.FileSystemPreferences$1 run
WARNING: java.io.IOException: No such file or directory
|  Welcome to JShell -- Version 11.0.12
|  For an introduction type: /help intro

jshell> java.awt.Color.decode("#000000")
|  Exception java.lang.UnsatisfiedLinkError: Can't load library: /usr/lib/jvm/java-11-openjdk-amd64/lib/libawt_xawt.so
|        at ClassLoader.loadLibrary (ClassLoader.java:2630)
|        at Runtime.load0 (Runtime.java:768)
|        at System.load (System.java:1837)
|        at ClassLoader$NativeLibrary.load0 (Native Method)
|        at ClassLoader$NativeLibrary.load (ClassLoader.java:2442)
|        at ClassLoader$NativeLibrary.loadLibrary (ClassLoader.java:2498)
|        at ClassLoader.loadLibrary0 (ClassLoader.java:2694)
|        at ClassLoader.loadLibrary (ClassLoader.java:2648)
|        at Runtime.loadLibrary0 (Runtime.java:830)
|        at System.loadLibrary (System.java:1873)
|        at Toolkit$3.run (Toolkit.java:1399)
|        at Toolkit$3.run (Toolkit.java:1397)
|        at AccessController.doPrivileged (Native Method)
|        at Toolkit.loadLibraries (Toolkit.java:1396)
|        at Toolkit.<clinit> (Toolkit.java:1429)
|        at Color.<clinit> (Color.java:275)
|        at (#1:1) 

So this is pretty niche, a headless JVM on a machine running an X client.

[kosak]

Sorry, I forgot to follow up on this. I was going to "make it happen" on a fresh Ubuntu VM if you needed me to.

Regarding this comment.

So this is pretty niche, a headless JVM on a machine running an X client.

It's not niche because it's what our documentation tells you to install. https://deephaven.io/core/docs/how-to-guides/launch-build/ (expand Installing Java)