search

deephaven / deephaven-server-docker

Deephaven Server Docker
4 stars 10 forks source link

Ensure deephaven-core PyPi package does not become out-of-sync with server version #63

Open devinrsmith opened 1 year ago

devinrsmith commented 1 year ago

A user on slack reported an error starting up a deephaven docker server:

deephaven_docker-deephaven-1  | Type: <class 'ValueError'>
deephaven_docker-deephaven-1  | Value: Java class 'io.deephaven.server.plugin.python.LivePyObjectWrapper' not found
deephaven_docker-deephaven-1  | Line: 24
deephaven_docker-deephaven-1  | Namespace: <module>
deephaven_docker-deephaven-1  | File: /opt/deephaven/venv/lib/python3.10/site-packages/deephaven/_wrapper.py
deephaven_docker-deephaven-1  | Traceback (most recent call last):
deephaven_docker-deephaven-1  |   File "/opt/deephaven/venv/lib/python3.10/site-packages/deephaven/__init__.py", line 20, in <module>
deephaven_docker-deephaven-1  |   File "/opt/deephaven/venv/lib/python3.10/site-packages/deephaven/table.py", line 23, in <module>
deephaven_docker-deephaven-1  |   File "/opt/deephaven/venv/lib/python3.10/site-packages/deephaven/_wrapper.py", line 24, in <module>
deephaven_docker-deephaven-1  | 
deephaven_docker-deephaven-1  |         at org.jpy.PyLib.importModule(Native Method)
deephaven_docker-deephaven-1  |         at org.jpy.PyModule.importModule(PyModule.java:94)
deephaven_docker-deephaven-1  |         at io.deephaven.integrations.python.PythonDeephavenSession.<init>(PythonDeephavenSession.java:92)
deephaven_docker-deephaven-1  |         at io.deephaven.server.console.python.PythonConsoleSessionModule.bindPythonSession(PythonConsoleSessionModule.java:37)    
deephaven_docker-deephaven-1  |         at io.deephaven.server.console.python.PythonConsoleSessionModule_BindPythonSessionFactory.bindPythonSession(PythonConsoleSessionModule_BindPythonSessionFactory.java:64)
deephaven_docker-deephaven-1  |         at io.deephaven.server.console.python.PythonConsoleSessionModule_BindPythonSessionFactory.get(PythonConsoleSessionModule_BindPythonSessionFactory.java:51)
deephaven_docker-deephaven-1  |         at io.deephaven.server.console.python.PythonConsoleSessionModule_BindPythonSessionFactory.get(PythonConsoleSessionModule_BindPythonSessionFactory.java:16)
deephaven_docker-deephaven-1  |         at io.deephaven.server.console.python.PythonConsoleSessionModule_BindScriptSessionFactory.get(PythonConsoleSessionModule_BindScriptSessionFactory.java:37)
deephaven_docker-deephaven-1  |         at io.deephaven.server.console.python.PythonConsoleSessionModule_BindScriptSessionFactory.get(PythonConsoleSessionModule_BindScriptSessionFactory.java:13)
deephaven_docker-deephaven-1  |         at io.deephaven.server.runner.DeephavenApiServerModule.provideScriptSession(DeephavenApiServerModule.java:96)
deephaven_docker-deephaven-1  |         at io.deephaven.server.runner.DeephavenApiServerModule_ProvideScriptSessionFactory.provideScriptSession(DeephavenApiServerModule_ProvideScriptSessionFactory.java:48)
deephaven_docker-deephaven-1  |         at io.deephaven.server.runner.DeephavenApiServerModule_ProvideScriptSessionFactory.get(DeephavenApiServerModule_ProvideScriptSessionFactory.java:37)
deephaven_docker-deephaven-1  |         at io.deephaven.server.runner.DeephavenApiServerModule_ProvideScriptSessionFactory.get(DeephavenApiServerModule_ProvideScriptSessionFactory.java:13)
deephaven_docker-deephaven-1  |         at dagger.internal.DoubleCheck.get(DoubleCheck.java:47)
deephaven_docker-deephaven-1  |         at io.deephaven.server.runner.DeephavenApiServer.run(DeephavenApiServer.java:130)
deephaven_docker-deephaven-1  |         at io.deephaven.server.jetty.JettyMain.main(JettyMain.java:24)

I suspect this is due to a python plugin being installed that forced a newer version of deephaven-core to be installed; namely, deephaven-plugin-matplotlib is declaring a dependency on deephaven-core>=0.27.0 (due to recent plugin interface changes), but the underlying image was < 0.27.0.

We should see if there is a way to lock the version of deephaven-core in the python virtual environment; alternatively, maybe we set that part of the filesystem as read-only?

devinrsmith commented 1 year ago

Trying to solve it at the filesystem level by removing the write bit from

chmod -w /opt/deephaven/venv/lib/python3.10/site-packages/deephaven/__init__.py

doesn't work b/c the docker image is currently running as root; this would likely work if we had a different user running inside the image.

Another alternative would be chattr +i, which can still be worked around by root, but is an explicit operation:

$ docker run --cap-add CAP_LINUX_IMMUTABLE -it --rm --name deephaven -p 10000:10000 --entrypoint bash ghcr.io/deephaven/server:0.28.0                                                                                  
root@71cff31ed482:/# chattr +i /opt/deephaven/venv/lib/python3.10/site-packages/deephaven/__init__.py                                                                                                                          

root@71cff31ed482:/# pip install deephaven-core==0.28.1                                                                                                                                                                        
...                                                                                                                                                
Installing collected packages: deephaven-core                                                                                                                                                                                  
  Attempting uninstall: deephaven-core                                                                                                                                                                                         
    Found existing installation: deephaven-core 0.28.0                                                                                                                                                                         
    Uninstalling deephaven-core-0.28.0:                                                                                                                                                                                        
ERROR: Could not install packages due to an OSError: [Errno 1] Operation not permitted: '/opt/deephaven/venv/lib/python3.10/site-packages/deephaven/__init__.py'

Unfortunately, I've been unable to figure out how / if it's possible to give these privileges at docker build time; currently running into:

 > [2/3] RUN chattr +i "/opt/deephaven/venv/lib/python3.10/site-packages/deephaven/__init__.py":
0.072 chattr: Operation not permitted while setting flags on /opt/deephaven/venv/lib/python3.10/site-packages/deephaven/__init__.py