MordicusEtCubitus commented 11 months ago

Hi,

First thanks for creating PyVista and trame, that's so huge !

I've never been able to make jupyter work with trame when intalled with TLJH: https://tljh.jupyter.org/

I've got a 404 error

404 : Not Found

You are requesting a page that does not exist!

URL generating error is: https://<server name>/user/manager/proxy/36379/index.html?ui=P_0x7fea90333110_3&reconnect=auto

It works quite well on jupyter or jupyter lab on a single pc without proxy, but I have a lot of issues on remote server. Mine has been installed using the OVH configuration: https://tljh.jupyter.org/en/latest/install/ovh.html

I'm working using a miniforge virtual environment with my notebook

First Question: Do I need to install trame extension on both TLJH python and my notebook python ?

Here are the jupyter versions for

Notebook python

jupyter --version
Selected Jupyter core packages...
IPython          : 8.17.2
ipykernel        : 6.26.0
ipywidgets       : 8.1.1
jupyter_client   : 8.6.0
jupyter_core     : 5.5.0
jupyter_server   : 2.10.0
jupyterlab       : 4.0.8
nbclient         : 0.8.0
nbconvert        : 7.11.0
nbformat         : 5.9.2
notebook         : not installed
qtconsole        : not installed
traitlets        : 5.13.0

TLJH Python

/opt/tljh/user/bin/jupyter --version
Selected Jupyter core packages...
IPython          : 8.17.2
ipykernel        : 6.26.0
ipywidgets       : 8.1.1
jupyter_client   : 8.6.0
jupyter_core     : 5.5.0
jupyter_server   : 2.10.0
jupyterlab       : 4.0.8
nbclient         : 0.7.4
nbconvert        : 7.11.0
nbformat         : 5.9.2
notebook         : 7.0.6
qtconsole        : not installed
traitlets        : 5.13.0

Thanks for helping

jourdain commented 11 months ago

I would think the extension only needs to be installed on your "TLJH python".

It is possible that the PyVista integration is not complete with that extension. But the following should work.

pip install trame trame-vtk trame-vuetify trame-jupyter-extension

Then in a cell you should be able to do the following

from trame.app.demo import Cone

c = Cone()
await a.ui.ready
c.ui

Also make sure you have the proper version match

JupyterLab v4 => trame-jupyter-extension>=2
JupyterLab v3 => trame-jupyter-extension<2

MordicusEtCubitus commented 11 months ago

Thanks for your help. I confirm that your example works fine after having installed the libraries with pip and using await c.ui.ready (not a.ui.ready)

Thanks a lot.

But these two examples do not:

import pyvista as pv
sphere = pv.Sphere()

# short example
sphere.plot(jupyter_backend='trame')

# long example
plotter = pv.Plotter(notebook=True)
plotter.add_mesh(sphere)
plotter.show(jupyter_backend='trame')

They first ask to start a virtual frame buffer (no X on the server, it is a web server) using pyvista.start_xvfb() then, once done I still have the 404 error

MordicusEtCubitus commented 11 months ago

So, I've tried to create my own application, cloning Cone App and merging with pyvista trame example from here: https://tutorial.pyvista.org/tutorial/09_trame/index.html

And it works quite well. Thanks for the helping hand to let run it finally !

import pyvista as pv
from trame.app import get_server
from trame.ui.vuetify import SinglePageLayout
from trame.widgets import vuetify, vtk as vtk_widgets
from pyvista.trame.ui import plotter_ui

pv.start_xvfb()

class MyCone:
    def __init__(self, server_or_name=None):
        self.server = get_server(server_or_name, client_type="vue2")
        self.ui = self._generate_ui()

    @property
    def ctrl(self):
        return self.server.controller

    @property
    def state(self):
        return self.server.state

    @property
    def resolution(self):
        return self.state.resolution

    @resolution.setter
    def resolution(self, v):
        with self.state:
            self.state.resolution = v

    def reset_resolution(self):
        self.resolution = 6

    def _generate_ui(self):
        with SinglePageLayout(self.server) as layout:
            layout.title.set_text("Trame demo")

            with layout.content:

                pl = pv.Plotter()
                mesh1 = pv.Sphere()
                pl.add_mesh(mesh1)

                view = plotter_ui(pl)

            return layout

pv.OFF_SCREEN = True

mc = MyCone()
await mc.ui.ready
mc.ui

MordicusEtCubitus commented 11 months ago

But if you know how this example can work using TLJH it will be easier:

import pyvista as pv
sphere = pv.Sphere()

# short example
sphere.plot(jupyter_backend='trame')

MordicusEtCubitus commented 11 months ago

I've realized that jupyter-server-proxy was only installed in my virtual environment used in notebook and not on the python running jupyter :( Sorry. So after installing it, the 404 error has disappeared and I've got a 500 error with this url:

https:///user/manager/proxy/0/index.html?ui=P_0x7fd07c0a72d0_1&reconnect=auto

Still looking at it.

Haven't you a project to remove jupyter-server-proxy when used with jupyter ?

MordicusEtCubitus commented 11 months ago

Here is the error log that I do not understand :

Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:     HTTPServerRequest(protocol='https', host='<server>.com', method='GET', uri='/user/manager/proxy/0/index.html?ui=P_0x7fead6fa6c50_3&reconnect=auto', version='HTTP/1.1', remote_ip='<remoteip>')
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:     Traceback (most recent call last):
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:       File "/opt/tljh/user/lib/python3.10/site-packages/tornado/web.py", line 1786, in _execute
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:         result = await result
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:       File "/opt/tljh/user/lib/python3.10/site-packages/jupyter_server_proxy/websocket.py", line 111, in get
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:         return await self.http_get(*args, **kwargs)
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:       File "/opt/tljh/user/lib/python3.10/site-packages/jupyter_server_proxy/handlers.py", line 554, in http_get
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:         return await self.proxy(port, proxied_path)
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:       File "/opt/tljh/user/lib/python3.10/site-packages/jupyter_server_proxy/handlers.py", line 354, in proxy
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:         response = await client.fetch(req, raise_error=False)
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:     ConnectionRefusedError: [Errno 111] Connection refused
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]: [E 2023-11-13 21:31:27.213 ServerApp] {
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:       "Host": "<server>.com",
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:       "User-Agent": "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/119.0",
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:       "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8",
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:       "Accept-Encoding": "gzip, deflate, br",
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:       "Accept-Language": "fr,en;q=0.5",
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:       "Cookie": "jupyterhub-user-manager=[secret]; _xsrf=[secret]; jupyterhub-session-id=[secret]",
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:       "Dnt": "1",
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:       "Referer": "https://<server>.com/user/manager/lab/workspaces/auto-6/tree/Gael/PyVistaTest.ipynb",
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:       "Sec-Fetch-Dest": "iframe",
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:       "Sec-Fetch-Mode": "navigate",
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:       "Sec-Fetch-Site": "same-origin",
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:       "Upgrade-Insecure-Requests": "1",
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:       "X-Forwarded-For": "<remoteip>",
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:       "X-Forwarded-Host": "<server>.com",
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:       "X-Forwarded-Port": "443",
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:       "X-Forwarded-Proto": "https",
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:       "X-Forwarded-Server": "sys7",
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:       "X-Real-Ip": "<remoteip>"
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]:     }
Nov 13 21:31:27 sys7 jupyterhub-singleuser[63162]: [E 2023-11-13 21:31:27.213 ServerApp] 500 GET /user/manager/proxy/0/index.html?ui=P_0x7fead6fa6c50_3&reconnect=auto (manager@<remoteip>) 3.73ms

jourdain commented 11 months ago

For the X error, you should uninstall vtk and install vtk-osmesa instead.

You do not need jupyter-server-proxy if you have trame-jupyter-extension but like I said, we need to do a small fix in PyVista to have proper support when trame-jupyter-extension is installed.

@alesgenova

alesgenova commented 10 months ago

@MordicusEtCubitus The trame jupyter extension support just landed in pyvista. Try using PyVista 0.43 (released today) and install trame-jupyter-extension. Let us know if it works now

MordicusEtCubitus commented 10 months ago

Ho... Sure... I'm going to do it right now on my server...

MordicusEtCubitus commented 10 months ago

So, Here is the result of my first test: I've installed TLJH on an OVH server as described here: https://tljh.jupyter.org/en/latest/install/ovh.html Then created miniforge environment:

mamba create --name PyVistaTest pyvista ipykernel
mamba activate PyVistaTest
(PyVistaTest) pip install trame-jupyter-extension

Then run the hello world example: https://docs.pyvista.org/version/stable/examples/00-load/create-geometric-objects.html#sphx-glr-examples-00-load-create-geometric-objects-py

Still see my kernel dying on the p.show() command

So, I've installed pyvista and trame-jupyter-extension into /opt/tljh this way

sudo /opt/tljh/user/bin/mamba install pyvista  # 0.43 ins installed
sudo /opt/tljh/user/bin/pip install  trame-jupyter-extension

Then restart tljh

Kernel is still dying If looking at the logs , it is required to install xvfb: So adding:

import pyvista as pv
pv.start_xvfb()

And it works in static mode. But not with: p.show(jupyter_backend='trame')| In that case I've got the error cannot import name 'vtk' from 'trame.widgets'

VTK is installed in pyvistatest environment.

I've added both in pyvista and tljh environment:

(PyVistaTest) pip install trame trame-vtk trame-vuetify
sudo /opt/tljh/user/bin/pip install trame trame-vtk trame-vuetify

Not better.

If xfvb is not initialized I've got a died kernel, if running I've got a 404 error

I cannot continue further right now, it is sport time. I can give you access to my TLJH server on OVH if you can give me a personal email, here is mine:

MordicusEtCubitus commented 10 months ago

I've done new test:

Format and install a fresh TLJH (https://tljh.jupyter.org/en/latest/install/ovh.html) server on ovh

Install pyvista using pip :

sudo  /opt/tljh/user/bin/pip install  pyvista trame-jupyter-extension

Then running this code in Jupyter notebook


import pyvista as pv
from pyvista import examples

dataset = examples.download_lucy() dataset.plot(smooth_shading=True, color='white')


I've got an error about missing libEGL1.so

So installed it:

sudo apt install libegl1*


Then, when running I had a 404 error

So, I've added these librairies:

sudo /opt/tljh/user/bin/pip install trame trame-vtk trame-vuetify


Still have the same error, looking at logs I got:

Dec 09 21:34:50 sys7 jupyterhub-singleuser[25910]: [W 2023-12-09 21:34:50.276 ServerApp] 404 GET /user/manager/proxy/39025/index.html?ui=P_0x7fafb2419ed0_2&reconnect=auto (manager@) 2.56ms



`MyCone`  example still working

MordicusEtCubitus commented 10 months ago

I can give you access to my server as sudo user if you want to try by yourself, just give me your email to get the credentials and url

jourdain commented 10 months ago

It is still trying to use the server proxy. Try updating pyvista and make sure the trame extension is selected. @alesgenova

MordicusEtCubitus commented 9 months ago

I've tried a new installation like this

mamba create --name pyvista python=3.10 xarray dask datashader jupyterlab ipywidgets
mamba activate pyvista
pip install pyvista trame-jupyter-extension
pip install pyvista-xarray

Then start jupyter lab:

jupyter lab

And in a new notebook, run pyvista-xarray example:

import pvxarray
import xarray as xr

ds = xr.tutorial.load_dataset("air_temperature")
da = ds.air[dict(time=0)]  # Select DataArray for a timestep

# Or grab the mesh object for use with PyVista
mesh = da.pyvista.mesh(x="lon", y="lat")

until there, it is ok

Then, run

da.pyvista.plot(x="lon", y="lat", show_edges=True, cpos='xy')

Got error

/home/user/miniforge3/envs/pyvista/lib/python3.10/site-packages/pyvista/jupyter/notebook.py:34: UserWarning: Failed to use notebook backend: 

cannot import name 'vtk' from 'trame.widgets' (/home/user/miniforge3/envs/pyvista/lib/python3.10/site-packages/trame/widgets/__init__.py)

Falling back to a static output.
  warnings.warn(

MordicusEtCubitus commented 9 months ago

Here is the created environnement description:

$ mamba env export 
name: pyvista
channels:
  - conda-forge
dependencies:
  - _libgcc_mutex=0.1=conda_forge
  - _openmp_mutex=4.5=2_gnu
  - anyio=4.2.0=pyhd8ed1ab_0
  - argon2-cffi=23.1.0=pyhd8ed1ab_0
  - argon2-cffi-bindings=21.2.0=py310h2372a71_4
  - arrow=1.3.0=pyhd8ed1ab_0
  - asttokens=2.4.1=pyhd8ed1ab_0
  - async-lru=2.0.4=pyhd8ed1ab_0
  - attrs=23.2.0=pyh71513ae_0
  - aws-c-auth=0.7.8=hcf8cf63_3
  - aws-c-cal=0.6.9=h5d48c4d_2
  - aws-c-common=0.9.10=hd590300_0
  - aws-c-compression=0.2.17=h7f92143_7
  - aws-c-event-stream=0.3.2=h0bcb0bb_8
  - aws-c-http=0.7.15=hd268abd_0
  - aws-c-io=0.13.36=hb3b01f7_3
  - aws-c-mqtt=0.10.0=hbafccad_1
  - aws-c-s3=0.4.6=h47b1690_0
  - aws-c-sdkutils=0.1.13=h7f92143_0
  - aws-checksums=0.1.17=h7f92143_6
  - aws-crt-cpp=0.25.0=hfa7cc67_4
  - aws-sdk-cpp=1.11.210=h0853bfa_5
  - babel=2.14.0=pyhd8ed1ab_0
  - beautifulsoup4=4.12.2=pyha770c72_0
  - bleach=6.1.0=pyhd8ed1ab_0
  - bokeh=3.3.2=pyhd8ed1ab_0
  - brotli-python=1.1.0=py310hc6cd4ac_1
  - bzip2=1.0.8=hd590300_5
  - c-ares=1.24.0=hd590300_0
  - ca-certificates=2023.11.17=hbcca054_0
  - cached-property=1.5.2=hd8ed1ab_1
  - cached_property=1.5.2=pyha770c72_1
  - certifi=2023.11.17=pyhd8ed1ab_0
  - cffi=1.16.0=py310h2fee648_0
  - charset-normalizer=3.3.2=pyhd8ed1ab_0
  - click=8.1.7=unix_pyh707e725_0
  - cloudpickle=3.0.0=pyhd8ed1ab_0
  - colorcet=3.0.1=pyhd8ed1ab_0
  - comm=0.1.4=pyhd8ed1ab_0
  - contourpy=1.2.0=py310hd41b1e2_0
  - cytoolz=0.12.2=py310h2372a71_1
  - dask=2023.12.1=pyhd8ed1ab_0
  - dask-core=2023.12.1=pyhd8ed1ab_0
  - datashader=0.16.0=pyhd8ed1ab_0
  - debugpy=1.8.0=py310hc6cd4ac_1
  - decorator=5.1.1=pyhd8ed1ab_0
  - defusedxml=0.7.1=pyhd8ed1ab_0
  - distributed=2023.12.1=pyhd8ed1ab_0
  - entrypoints=0.4=pyhd8ed1ab_0
  - exceptiongroup=1.2.0=pyhd8ed1ab_0
  - executing=2.0.1=pyhd8ed1ab_0
  - fqdn=1.5.1=pyhd8ed1ab_0
  - freetype=2.12.1=h267a509_2
  - fsspec=2023.12.2=pyhca7485f_0
  - gflags=2.2.2=he1b5a44_1004
  - glog=0.6.0=h6f12383_0
  - icu=73.2=h59595ed_0
  - idna=3.6=pyhd8ed1ab_0
  - importlib-metadata=7.0.1=pyha770c72_0
  - importlib_metadata=7.0.1=hd8ed1ab_0
  - importlib_resources=6.1.1=pyhd8ed1ab_0
  - ipykernel=6.28.0=pyhd33586a_0
  - ipython=8.19.0=pyh707e725_0
  - ipywidgets=8.1.1=pyhd8ed1ab_0
  - isoduration=20.11.0=pyhd8ed1ab_0
  - jedi=0.19.1=pyhd8ed1ab_0
  - jinja2=3.1.2=pyhd8ed1ab_1
  - json5=0.9.14=pyhd8ed1ab_0
  - jsonpointer=2.4=py310hff52083_3
  - jsonschema=4.20.0=pyhd8ed1ab_0
  - jsonschema-specifications=2023.12.1=pyhd8ed1ab_0
  - jsonschema-with-format-nongpl=4.20.0=pyhd8ed1ab_0
  - jupyter-lsp=2.2.1=pyhd8ed1ab_0
  - jupyter_client=8.6.0=pyhd8ed1ab_0
  - jupyter_core=5.6.0=py310hff52083_0
  - jupyter_events=0.9.0=pyhd8ed1ab_0
  - jupyter_server=2.12.1=pyhd8ed1ab_0
  - jupyter_server_terminals=0.5.1=pyhd8ed1ab_0
  - jupyterlab=4.0.10=pyhd8ed1ab_0
  - jupyterlab_pygments=0.3.0=pyhd8ed1ab_0
  - jupyterlab_server=2.25.2=pyhd8ed1ab_0
  - jupyterlab_widgets=3.0.9=pyhd8ed1ab_0
  - keyutils=1.6.1=h166bdaf_0
  - krb5=1.21.2=h659d440_0
  - lcms2=2.16=hb7c19ff_0
  - ld_impl_linux-64=2.40=h41732ed_0
  - lerc=4.0.0=h27087fc_0
  - libabseil=20230802.1=cxx17_h59595ed_0
  - libarrow=14.0.2=hfb4d3a9_0_cpu
  - libarrow-acero=14.0.2=h59595ed_0_cpu
  - libarrow-dataset=14.0.2=h59595ed_0_cpu
  - libarrow-flight=14.0.2=h120cb0d_0_cpu
  - libarrow-flight-sql=14.0.2=h61ff412_0_cpu
  - libarrow-gandiva=14.0.2=hacb8726_0_cpu
  - libarrow-substrait=14.0.2=h61ff412_0_cpu
  - libblas=3.9.0=20_linux64_openblas
  - libbrotlicommon=1.1.0=hd590300_1
  - libbrotlidec=1.1.0=hd590300_1
  - libbrotlienc=1.1.0=hd590300_1
  - libcblas=3.9.0=20_linux64_openblas
  - libcrc32c=1.1.2=h9c3ff4c_0
  - libcurl=8.5.0=hca28451_0
  - libdeflate=1.19=hd590300_0
  - libedit=3.1.20191231=he28a2e2_2
  - libev=4.33=hd590300_2
  - libevent=2.1.12=hf998b51_1
  - libffi=3.4.2=h7f98852_5
  - libgcc-ng=13.2.0=h807b86a_3
  - libgfortran-ng=13.2.0=h69a702a_3
  - libgfortran5=13.2.0=ha4646dd_3
  - libgomp=13.2.0=h807b86a_3
  - libgoogle-cloud=2.12.0=h5206363_4
  - libgrpc=1.59.3=hd6c4280_0
  - libiconv=1.17=hd590300_2
  - libjpeg-turbo=3.0.0=hd590300_1
  - liblapack=3.9.0=20_linux64_openblas
  - libllvm14=14.0.6=hcd5def8_4
  - libllvm15=15.0.7=hb3ce162_4
  - libnghttp2=1.58.0=h47da74e_1
  - libnl=3.9.0=hd590300_0
  - libnsl=2.0.1=hd590300_0
  - libnuma=2.0.16=h0b41bf4_1
  - libopenblas=0.3.25=pthreads_h413a1c8_0
  - libparquet=14.0.2=h352af49_0_cpu
  - libpng=1.6.39=h753d276_0
  - libprotobuf=4.24.4=hf27288f_0
  - libre2-11=2023.06.02=h7a70373_0
  - libsodium=1.0.18=h36c2ea0_1
  - libsqlite=3.44.2=h2797004_0
  - libssh2=1.11.0=h0841786_0
  - libstdcxx-ng=13.2.0=h7e041cc_3
  - libthrift=0.19.0=hb90f79a_1
  - libtiff=4.6.0=ha9c0a0a_2
  - libutf8proc=2.8.0=h166bdaf_0
  - libuuid=2.38.1=h0b41bf4_0
  - libwebp-base=1.3.2=hd590300_0
  - libxcb=1.15=h0b41bf4_0
  - libxcrypt=4.4.36=hd590300_1
  - libxml2=2.12.3=h232c23b_0
  - libzlib=1.2.13=hd590300_5
  - llvmlite=0.41.1=py310h1b8f574_0
  - locket=1.0.0=pyhd8ed1ab_0
  - lz4=4.3.2=py310h350c4a5_1
  - lz4-c=1.9.4=hcb278e6_0
  - markupsafe=2.1.3=py310h2372a71_1
  - matplotlib-inline=0.1.6=pyhd8ed1ab_0
  - mistune=3.0.2=pyhd8ed1ab_0
  - msgpack-python=1.0.7=py310hd41b1e2_0
  - multipledispatch=0.6.0=py_0
  - nbclient=0.8.0=pyhd8ed1ab_0
  - nbconvert-core=7.13.1=pyhd8ed1ab_0
  - nbformat=5.9.2=pyhd8ed1ab_0
  - ncurses=6.4=h59595ed_2
  - nest-asyncio=1.5.8=pyhd8ed1ab_0
  - notebook-shim=0.2.3=pyhd8ed1ab_0
  - numba=0.58.1=py310h7dc5dd1_0
  - numpy=1.26.2=py310hb13e2d6_0
  - openjpeg=2.5.0=h488ebb8_3
  - openssl=3.2.0=hd590300_1
  - orc=1.9.2=h4b38347_0
  - overrides=7.4.0=pyhd8ed1ab_0
  - packaging=23.2=pyhd8ed1ab_0
  - pandas=2.1.4=py310hcc13569_0
  - pandocfilters=1.5.0=pyhd8ed1ab_0
  - param=2.0.1=pyhca7485f_0
  - parso=0.8.3=pyhd8ed1ab_0
  - partd=1.4.1=pyhd8ed1ab_0
  - pexpect=4.8.0=pyh1a96a4e_2
  - pickleshare=0.7.5=py_1003
  - pillow=10.1.0=py310h01dd4db_0
  - pip=23.3.2=pyhd8ed1ab_0
  - pkgutil-resolve-name=1.3.10=pyhd8ed1ab_1
  - platformdirs=4.1.0=pyhd8ed1ab_0
  - prometheus_client=0.19.0=pyhd8ed1ab_0
  - prompt-toolkit=3.0.42=pyha770c72_0
  - psutil=5.9.7=py310h2372a71_0
  - pthread-stubs=0.4=h36c2ea0_1001
  - ptyprocess=0.7.0=pyhd3deb0d_0
  - pure_eval=0.2.2=pyhd8ed1ab_0
  - pyarrow=14.0.2=py310hf9e7431_0_cpu
  - pyarrow-hotfix=0.6=pyhd8ed1ab_0
  - pycparser=2.21=pyhd8ed1ab_0
  - pyct=0.5.0=pyhd8ed1ab_0
  - pygments=2.17.2=pyhd8ed1ab_0
  - pysocks=1.7.1=pyha2e5f31_6
  - python=3.10.13=hd12c33a_1_cpython
  - python-dateutil=2.8.2=pyhd8ed1ab_0
  - python-fastjsonschema=2.19.1=pyhd8ed1ab_0
  - python-json-logger=2.0.7=pyhd8ed1ab_0
  - python-tzdata=2023.4=pyhd8ed1ab_0
  - python_abi=3.10=4_cp310
  - pytz=2023.3.post1=pyhd8ed1ab_0
  - pyyaml=6.0.1=py310h2372a71_1
  - pyzmq=25.1.2=py310h795f18f_0
  - rdma-core=49.0=hd3aeb46_2
  - re2=2023.06.02=h2873b5e_0
  - readline=8.2=h8228510_1
  - referencing=0.32.0=pyhd8ed1ab_0
  - requests=2.31.0=pyhd8ed1ab_0
  - rfc3339-validator=0.1.4=pyhd8ed1ab_0
  - rfc3986-validator=0.1.1=pyh9f0ad1d_0
  - rpds-py=0.16.2=py310hcb5633a_0
  - s2n=1.4.1=h06160fa_0
  - scipy=1.11.4=py310hb13e2d6_0
  - send2trash=1.8.2=pyh41d4057_0
  - setuptools=68.2.2=pyhd8ed1ab_0
  - six=1.16.0=pyh6c4a22f_0
  - snappy=1.1.10=h9fff704_0
  - sniffio=1.3.0=pyhd8ed1ab_0
  - sortedcontainers=2.4.0=pyhd8ed1ab_0
  - soupsieve=2.5=pyhd8ed1ab_1
  - stack_data=0.6.2=pyhd8ed1ab_0
  - tblib=3.0.0=pyhd8ed1ab_0
  - terminado=0.18.0=pyh0d859eb_0
  - tinycss2=1.2.1=pyhd8ed1ab_0
  - tk=8.6.13=noxft_h4845f30_101
  - tomli=2.0.1=pyhd8ed1ab_0
  - toolz=0.12.0=pyhd8ed1ab_0
  - tornado=6.3.3=py310h2372a71_1
  - traitlets=5.14.0=pyhd8ed1ab_0
  - types-python-dateutil=2.8.19.14=pyhd8ed1ab_0
  - typing-extensions=4.9.0=hd8ed1ab_0
  - typing_extensions=4.9.0=pyha770c72_0
  - typing_utils=0.1.0=pyhd8ed1ab_0
  - tzdata=2023d=h0c530f3_0
  - ucx=1.15.0=h75e419f_2
  - uri-template=1.3.0=pyhd8ed1ab_0
  - urllib3=2.1.0=pyhd8ed1ab_0
  - wcwidth=0.2.12=pyhd8ed1ab_0
  - webcolors=1.13=pyhd8ed1ab_0
  - webencodings=0.5.1=pyhd8ed1ab_2
  - websocket-client=1.7.0=pyhd8ed1ab_0
  - wheel=0.42.0=pyhd8ed1ab_0
  - widgetsnbextension=4.0.9=pyhd8ed1ab_0
  - xarray=2023.12.0=pyhd8ed1ab_0
  - xorg-libxau=1.0.11=hd590300_0
  - xorg-libxdmcp=1.1.3=h7f98852_0
  - xyzservices=2023.10.1=pyhd8ed1ab_0
  - xz=5.2.6=h166bdaf_0
  - yaml=0.2.5=h7f98852_2
  - zeromq=4.3.5=h59595ed_0
  - zict=3.0.0=pyhd8ed1ab_0
  - zipp=3.17.0=pyhd8ed1ab_0
  - zstd=1.5.5=hfc55251_0
  - pip:
      - aiohttp==3.9.1
      - aiosignal==1.3.1
      - async-timeout==4.0.3
      - cycler==0.12.1
      - fonttools==4.47.0
      - frozenlist==1.4.1
      - kiwisolver==1.4.5
      - matplotlib==3.8.2
      - more-itertools==10.1.0
      - multidict==6.0.4
      - pooch==1.8.0
      - pyparsing==3.1.1
      - pyvista==0.43.1
      - pyvista-xarray==0.1.5
      - scooby==0.9.2
      - trame==3.4.0
      - trame-client==2.14.2
      - trame-jupyter-extension==2.0.1
      - trame-server==2.13.1
      - vtk==9.3.0
      - wslink==1.12.4
      - yarl==1.9.4
prefix: /home/aliquis/miniforge3/envs/pyvista

MordicusEtCubitus commented 9 months ago

Installing pip install trame-vtk trame-vuetify trame-components did it as found there: https://github.com/Kitware/trame/discussions/366

Question: why these packages are not a dependency of trame-jupyter-extension ?

MordicusEtCubitus commented 9 months ago

Now, still have to try with TLJH, this was tested on my laptop

jourdain commented 9 months ago

Because trame-jupyter-extension is just the network part and trame can work by itself if you don't use any components other than standard HTML. PyVista is using those trame components (vtk and vuetify) and is listing them as optional dependency here for the jupyter option (pyvista[jupyter]).

MordicusEtCubitus commented 9 months ago

This still do not work on a freshly installed TLJH serveur on OVH:

sudo /opt/tljh/user/bin/mamba install -c conda-forge xarray dask datashader jupyterlab ipywidgets
sudo /opt/tljh/user/bin/pip install pyvista trame-jupyter-extension
sudo /opt/tljh/user/bin/pip install pyvista-xarray
sudo /opt/tljh/user/bin/pip install trame-vtk trame-vuetify trame-components
sudo tljh-config reload proxy

Then creating a notebook and running:

import pvxarray
import xarray as xr

ds = xr.tutorial.load_dataset("air_temperature")
da = ds.air[dict(time=0)]  # Select DataArray for a timestep

# Plot in 3D
da.pyvista.plot(x="lon", y="lat", show_edges=True, cpos='xy')

And got this error:

/opt/tljh/user/lib/python3.10/site-packages/pyvista/plotting/plotter.py:151: UserWarning: 
This system does not appear to be running an xserver.
PyVista will likely segfault when rendering.

Try starting a virtual frame buffer with xvfb, or using
  ``pyvista.start_xvfb()``

  warnings.warn(
ERROR:root:bad X server connection. DISPLAY=
2023-12-31 22:58:45.831 (   7.354s) [    7F49343AB740]vtkXOpenGLRenderWindow.:456    ERR| vtkXOpenGLRenderWindow (0x559eb849a070): bad X server connection. DISPLAY=

Thought xvfb was no more required with trame-jupyter-extension

jourdain commented 9 months ago

Everything described is expected with your setup.

First of all trame-jupyter-extension is only for the network part and won't affect VTK rendering (bad X server connection).

While I don't know what sudo tljh-config reload proxy is doing. You should be able to get your environment working if you do the following.

sudo /opt/tljh/user/bin/mamba install -c conda-forge xarray dask datashader jupyterlab
sudo /opt/tljh/user/bin/pip install "pyvista[jupyter]" pyvista-xarray
sudo /opt/tljh/user/bin/pip uninstall vtk                                                # remove VTK that leverage X for its rendering
sudo /opt/tljh/user/bin/pip install --extra-index-url https://wheels.vtk.org vtk-osmesa  # install an offscreen CPU only version
sudo tljh-config reload proxy

jourdain commented 9 months ago

A simple test to see if trame is working by itself within Jupyter, you can run the following cell.

from trame.app.demo import Cone

app = Cone()
await app.ui.ready
app.ui

Since that example relies on vuetify and vtk.js, you will need (pip install trame trame-vtk trame-vuetify trame-jupyter-extension), which you already have in your setup but I wanted to mention it for anyone else.

MordicusEtCubitus commented 9 months ago

Thanks a lot, and happy new year by the way !

The cone application is working fine using the installation you have just suggested, and YES, adding the 'pyvista[jupyter]' option prevent from installing trame-vtk trame-vuetify

sudo tljh-config reload proxy is for restarting the whole Jupyter server, to be sure all librairies are reloaded

But even with you solution the pyvista-xarray example is not running and is dying with a 404 error

import pvxarray
import xarray as xr

ds = xr.tutorial.load_dataset("air_temperature")
da = ds.air[dict(time=0)]  # Select DataArray for a timestep
da.pyvista.plot(x="lon", y="lat", show_edges=True, cpos='xy')

While you were responding I was remembering that I needed to install vtk-osmesa (you already said it before) for my server without X and I recompile it using this tutorial from pyvista documentation: https://docs.pyvista.org/version/stable/extras/building_vtk.html

that I reproduce here:

sudo apt-get install libosmesa6-dev cmake ninja-build

git clone https://github.com/Kitware/VTK.git
cd VTK
git checkout v9.1.0
mkdir build
cd build

PYBIN=/usr/bin/python
cmake -GNinja \
      -DCMAKE_BUILD_TYPE=Release \
      -DVTK_BUILD_TESTING=OFF \
      -DVTK_BUILD_DOCUMENTATION=OFF \
      -DVTK_BUILD_EXAMPLES=OFF \
      -DVTK_DATA_EXCLUDE_FROM_ALL:BOOL=ON \
      -DVTK_MODULE_ENABLE_VTK_PythonInterpreter:STRING=NO \
      -DVTK_MODULE_ENABLE_VTK_WebCore:STRING=YES \
      -DVTK_MODULE_ENABLE_VTK_WebGLExporter:STRING=YES \
      -DVTK_MODULE_ENABLE_VTK_WebPython:STRING=YES \
      -DVTK_WHEEL_BUILD=ON \
      -DVTK_PYTHON_VERSION=3 \
      -DVTK_WRAP_PYTHON=ON \
      -DVTK_OPENGL_HAS_EGL=False \
      -DVTK_OPENGL_HAS_OSMESA=True \
      -DVTK_USE_COCOA=FALSE \
      -DVTK_USE_X=FALSE \
      -DVTK_DEFAULT_RENDER_WINDOW_HEADLESS=True \
      -DPython3_EXECUTABLE=$PYBIN ../
ninja
$PYBIN setup.py bdist_wheel

It compiles fine but the built wheel is missing some libraries like libvtkWrappingPythonCore3. Indeed, when unzipping the wheel they are not included:

unzip -l dist/vtk-9.1.0.dev0-cp310-cp310-linux_x86_64.whl |grep Wrapping
<nothing>

So I had to copy it manually

sudo cp -r ./build/lib.linux-x86_64-3.10/vtkmodules/libvtk* /opt/tljh/user/lib/python3.10/site-packages/vtkmodules/

Do you know if this is another bug, this tutorial is outdated or I have missed another key point elsewhere ?

Anyway, thanks a lot for your help.

MordicusEtCubitus commented 9 months ago

So if you can help for pyvista-xarray example on TLJH it will be great !

MordicusEtCubitus commented 9 months ago

I also confirm that the PyVista Hello World example still not working with the installation you have suggested:

import pyvista as pv

cyl = pv.Cylinder()
arrow = pv.Arrow()
sphere = pv.Sphere()
plane = pv.Plane()
line = pv.Line()
box = pv.Box()
cone = pv.Cone()
poly = pv.Polygon()
disc = pv.Disc()

p = pv.Plotter(shape=(3, 3))
# Top row
p.subplot(0, 0)
p.add_mesh(cyl, color="tan", show_edges=True)
p.subplot(0, 1)
p.add_mesh(arrow, color="tan", show_edges=True)
p.subplot(0, 2)
p.add_mesh(sphere, color="tan", show_edges=True)
# Middle row
p.subplot(1, 0)
p.add_mesh(plane, color="tan", show_edges=True)
p.subplot(1, 1)
p.add_mesh(line, color="tan", line_width=3)
p.subplot(1, 2)
p.add_mesh(box, color="tan", show_edges=True)
# Bottom row
p.subplot(2, 0)
p.add_mesh(cone, color="tan", show_edges=True)
p.subplot(2, 1)
p.add_mesh(poly, color="tan", show_edges=True)
p.subplot(2, 2)
p.add_mesh(disc, color="tan", show_edges=True)
# Render all of them
p.show()

Still have the 404 error:

Jan 01 08:51:57 sys7 jupyterhub-singleuser[11103]: [W 2024-01-01 08:51:57.166 ServerApp] 404 GET /user/manager/proxy/40789/index.html?ui=P_0x7fa2d4083760_4&reconnect=auto (manager@<ip>) 2.72ms

jourdain commented 9 months ago

You still use the jupyter server proxy for trame rather than its extension. There is a global setting on PyVista that you need to do to leverage/activate the trame extension. (Don't know how to activate it though @alesgenova )

For the vtk, there is no build to do, you need to follow the uninstall / install step that I mentioned earlier.

MordicusEtCubitus commented 9 months ago

Thanks for the reply.

I've found this page: https://docs.pyvista.org/version/stable/user-guide/jupyter/trame.html

To configure PyVista and Trame to work with jupyter-server-proxy in a remote environment, you will need to set some options on the global PyVista theme:

* pyvista.global_theme.trame.server_proxy_enabled
* pyvista.global_theme.trame.server_proxy_prefix

But if I had in front of helloWorld example code above this line

pyvista.global_theme.trame.server_proxy_enabled = False

I got a connection error in place of 404 error:

La connexion a échoué/Connection Failed

Firefox ne peut établir de connexion avec le serveur à l’adresse localhost:43361.
Firefox cannot etablish connecttion with server   localhost:43361

I think PyVista should be informed to use trame extension ? No ?

Finally I got it, it was below in the page:

pv.global_theme.trame.jupyter_extension_enabled = True

Then it works perfectly !

MordicusEtCubitus commented 9 months ago

So now I've got pyvista-xarray example working this way:

import pvxarray
import xarray as xr
import pyvista as pv
pv.global_theme.trame.server_proxy_enabled = False
pv.global_theme.trame.jupyter_extension_enabled = True

ds = xr.tutorial.load_dataset("air_temperature")
da = ds.air[dict(time=0)]  # Select DataArray for a timestep

# Plot in 3D
da.pyvista.plot(x="lon", y="lat", show_edges=True, cpos='xy')

Thanks a lot for your help and patience !

jourdain commented 9 months ago

Yes that is what I meant... I just didn't know the exact API calls.

MordicusEtCubitus commented 9 months ago

Hi,

I'm coming back. I've reinstalled my TLJH server and followed the procedure above.

But when enabling trame jupyter extension I got error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[1], line 5
      3 import pyvista as pv
      4 pv.global_theme.trame.server_proxy_enabled = False
----> 5 pv.global_theme.trame.jupyter_extension_enabled = True
      7 ds = xr.tutorial.load_dataset("air_temperature")
      8 da = ds.air[dict(time=0)]  # Select DataArray for a timestep

File /opt/tljh/user/lib/python3.10/site-packages/pyvista/plotting/themes.py:1501, in _TrameConfig.jupyter_extension_enabled(self, enabled)
   1498 @jupyter_extension_enabled.setter
   1499 def jupyter_extension_enabled(self, enabled: bool):  # numpydoc ignore=GL08
   1500     if enabled and not self.jupyter_extension_available:
-> 1501         raise ValueError("The trame_jupyter_extension is not available")
   1503     if enabled and self.server_proxy_enabled:
   1504         warnings.warn("Enabling jupyter_extension will disable server_proxy")

ValueError: The trame_jupyter_extension is not available

Running commands jupyter server extension list and jupyter labextension list shows that the extension is enabled and running

Looking at the code is appears that KEY TRAME_JUPYTER_WWW should be in os.environ

 self._jupyter_extension_available = 'TRAME_JUPYTER_WWW' in os.environ
        self._jupyter_extension_enabled = (
            self._jupyter_extension_available and not self._server_proxy_enabled
        )
        self._default_mode = 'trame'

I've added it with value '1' but does not works ! tryied my server url, not better. What should it contains ?

Thank for helping

MordicusEtCubitus commented 9 months ago

Also the cone application is no more working, getting:

from trame.app.demo import Cone

c = Cone()
await c.ui.ready
c.ui

La connexion a échoué

Firefox ne peut établir de connexion avec le serveur à l’adresse localhost:38255.

jourdain commented 9 months ago

No you should not set that ENV variable yourself. It is the role of the extension to provide it on the kernel side. If it is not there, that is probably because the extension is not enabled or the version you installed is not compatible with the jupyterlab version you have.

MordicusEtCubitus commented 9 months ago

That what I suspected first but my versions seems to be ok:

Here version installed in TLJH server

$ /opt/tljh/user/bin/jupyter-lab --version
4.0.11
$ /opt/tljh/user/bin/python -c "import trame_jupyter_extension as tje; print(tje.__version__)"
2.0.1

And the one in used environment

$ mamba activate pyvista
(pyvista) $ jupyter-lab --version
4.0.11
(pyvista) $ python -c "import trame_jupyter_extension as tje; print(tje.__version__)"
2.0.1
(pyvista) $ which python
/home/<user>/miniforge3/envs/pyvista/bin/python
(pyvista) $ which jupyter-lab
/home/<user>/miniforge3/envs/pyvista/bin/jupyter-lab

The doc says trame jupyter extension 2 for jupyter lab 4 So it seems to be ok

Here the installed versions of trame in both environments:

trame-3.5.1.dist-info
trame_client-2.15.0.dist-info
trame_jupyter_extension-2.0.1.dist-info
trame_server-2.15.0.dist-info
trame_vtk-2.7.0.dist-info  # vtk-osmesa
trame_vuetify-2.4.2.dist-info

Also, I'm not able to find where TRAME_JUPYTER_WWW is set in python code:

/opt/tljh/user/lib/python3.10/site-packages$ fgrep -r TRAME_JUPYTER_WWW *
pyvista/plotting/themes.py:        self._jupyter_extension_available = 'TRAME_JUPYTER_WWW' in os.environ
grep: pyvista/plotting/__pycache__/themes.cpython-310.pyc: binary file matches
grep: trame_client/ui/__pycache__/core.cpython-310.pyc: binary file matches
trame_client/ui/core.py:    www_path = os.environ.get("TRAME_JUPYTER_WWW")

anything else to check ?

Thanks in advance

MordicusEtCubitus commented 9 months ago

Well, I'm back again...

After rebooting it works:

The cone app work and pyvista-xarray example too


import pvxarray
import xarray as xr
import pyvista as pv

ds = xr.tutorial.load_dataset("air_temperature") da = ds.air[dict(time=0)] # Select DataArray for a timestep

Plot in 3D

da.pyvista.plot(x="lon", y="lat", show_edges=True, cpos='xy')


BUT only if I do not write

pv.global_theme.trame.server_proxy_enabled = False pv.global_theme.trame.jupyter_extension_enabled = True



Else it fails telling that `ValueError: The trame_jupyter_extension is not available` 

Any idea ?

alesgenova commented 9 months ago

Normally when a new notebook tab is opened, code in the client-side extension triggers the execution of some code in the python kernel, including the setting of the TRAME_JUPYTER_WWW env variable.

Since as you mentioned above the extension is properly activated based on jupyter server extension list and jupyter labextension list, I wonder if for TLJH another mechanism is needed to detect when a new notebook is opened, compared to plain Jupyter Lab.

Any thoughs, Sebastien?

jourdain commented 9 months ago

I'm not sure why the reboot was needed. Probably because the jupyter-lab server needed to start with the extension activated somehow.

For the pyvista True/Flase need, that is a bummer that you need to set both but I guess it could happen depending in which order those get checked/used.

Thanks for reporting back and providing the solution. Ideally the pv.global_theme.trame.jupyter_extension_enabled = True should super-seed the other option.

jourdain commented 9 months ago

Oups I missed the "BUT only if I do not write"

jourdain commented 9 months ago

That means it use the server proxy instead of the extension. Not sure why the extension is not "running" within such environment.

Do you see in the browser console if you have a global object named trameJupyter?

MordicusEtCubitus commented 9 months ago

Ooh ! Thanks for your reply and time.

I'm completely disappointed.

I've changed nothing since my last writing and it is working fine now !!

And yes, actually, when working I have the trameJupyter variable in the javascript console

MordicusEtCubitus commented 9 months ago

Ok, I will format and reinstall it all next week, so I will be able to check again the full installation process and can tell you more. Forget it until monday.

By the way I am actually playing with xarray-pyvista and geovista. This is so great !

Thanks again for this beautiful work : pyvista with trame jupyter extension is a killer feature !

Have a nice week end and good bear !

MordicusEtCubitus commented 9 months ago

good beer

jourdain commented 9 months ago

You may want to look at https://github.com/Kitware/pan3d/ at some point...

MordicusEtCubitus commented 9 months ago

Looks nice, will try it quickly !

MordicusEtCubitus commented 8 months ago

Hi,

I'm back trying to get Pyvista + trame_jupyter_extension working out of the box on a freshly installed TLJH server.

I still cannot get it working this way BUT I've found a work around.

Here is my procedure:

Reinstall/format my OVH bare server with ubuntu 22.04 server (no gui)

Connect by ssh on server and follow this installation procedure

bash$ curl https://raw.githubusercontent.com/jupyterhub/the-littlest-jupyterhub/master/bootstrap/bootstrap.py > bootstrap.py
bash$ sudo python3 bootstrap.py --admin manager
bash$ sudo tljh-config set https.enabled true
bash$ sudo tljh-config set https.letsencrypt.email <my emaiL>
bash$ sudo tljh-config add-item https.letsencrypt.domains "my.server.name"
bash$ sudo tljh-config reload proxy
bash$ sudo tljh-config set services.cull.enabled False
bash$ sudo tljh-config reload

Once the server is up, create the jupyter user and connect with him.

Then, open a terminal on jupyter lab and run these commands:

bash$ sudo /opt/tljh/user/bin/mamba install -c conda-forge xarray dask datashader jupyterlab
bash$ sudo /opt/tljh/user/bin/pip install "pyvista[jupyter]" pyvista-xarray
bash$ sudo /opt/tljh/user/bin/pip uninstall vtk
bash$ sudo /opt/tljh/user/bin/pip install --extra-index-url https://wheels.vtk.org vtk-osmesa
bash$ sudo tljh-config reload proxy

Then, create a new notebook, running these commands

import pyvista as pv
pv.global_theme.trame.server_proxy_enabled = False
pv.global_theme.trame.jupyter_extension_enabled = True

and got this error

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[1], line 5
      3 import pyvista as pv
      4 pv.global_theme.trame.server_proxy_enabled = False
----> 5 pv.global_theme.trame.jupyter_extension_enabled = True
      7 ds = xr.tutorial.load_dataset("air_temperature")
      8 da = ds.air[dict(time=0)]  # Select DataArray for a timestep

File /opt/tljh/user/lib/python3.10/site-packages/pyvista/plotting/themes.py:1501, in _TrameConfig.jupyter_extension_enabled(self, enabled)
   1498 @jupyter_extension_enabled.setter
   1499 def jupyter_extension_enabled(self, enabled: bool):  # numpydoc ignore=GL08
   1500     if enabled and not self.jupyter_extension_available:
-> 1501         raise ValueError("The trame_jupyter_extension is not available")
   1503     if enabled and self.server_proxy_enabled:
   1504         warnings.warn("Enabling jupyter_extension will disable server_proxy")

ValueError: The trame_jupyter_extension is not available

MordicusEtCubitus commented 8 months ago

Here is the work around: read it, it is funny: I just need to compile vtk-osmesa, not installing it !!

So, as our previous discussions shows that the error was from the missing environment variable TRAME_JUPYTER_WWW that should be set by vtk-osmesa and it is clearly not, I've decided to build myself vtk-osmesa following this way: https://docs.pyvista.org/version/stable/extras/building_vtk.html

Here are the commands:

bash$ sudo apt-get install libosmesa6-dev cmake ninja-build
bash$ git clone https://github.com/Kitware/VTK.git
bash$ cd VTK/
bash$  git checkout v9.3.0
bash$  mkdir build
bash$  cd build/
bash$ export PYBIN=/opt/tljh/user/bin/python
bash$  cmake -GNinja       -DCMAKE_BUILD_TYPE=Release       -DVTK_BUILD_TESTING=OFF       -DVTK_BUILD_DOCUMENTATION=OFF       -DVTK_BUILD_EXAMPLES=OFF       -DVTK_DATA_EXCLUDE_FROM_ALL:BOOL=ON       -DVTK_MODULE_ENABLE_VTK_PythonInterpreter:STRING=NO       -DVTK_MODULE_ENABLE_VTK_WebCore:STRING=YES       -DVTK_MODULE_ENABLE_VTK_WebGLExporter:STRING=YES       -DVTK_MODULE_ENABLE_VTK_WebPython:STRING=YES       -DVTK_WHEEL_BUILD=ON       -DVTK_PYTHON_VERSION=3       -DVTK_WRAP_PYTHON=ON       -DVTK_OPENGL_HAS_EGL=False       -DVTK_OPENGL_HAS_OSMESA=True       -DVTK_USE_COCOA=FALSE       -DVTK_USE_X=FALSE       -DVTK_DEFAULT_RENDER_WINDOW_HEADLESS=True       -DPython3_EXECUTABLE=$PYBIN ../
bash$ ninja
bash$ $PYBIN setup.py bdist_wheel

Then, go back to the notebook and it runs fine without installing the built package!

Clearly the built process solved something. It should not be the apt-get command because just after it I'd tried to run the notebook and it was still failing !

Surprising, if you can find why the build process made it working I will be pleased !

MordicusEtCubitus commented 8 months ago

If I install the built package it still working, of course:

sudo /opt/tljh/user/bin/pip install dist/vtk-9.3.0.dev0-cp310-cp310-linux_x86_64.whl

jourdain commented 8 months ago

TRAME_JUPYTER_WWW comes from the trame-jupyter-extension not vtk-osmesa. Also I don't see where you install that extension (trame-jupyter-extension).

jourdain commented 8 months ago

And also, why the build process make that exception go away is weird.

MordicusEtCubitus commented 8 months ago

TRAME_JUPYTER_WWW comes from the trame-jupyter-extension not vtk-osmesa

Really ? TRAME_JUPYTER_WWW is only read from os.environ and is not set in the python code. Is there a C/Cython-code I've missed with the extension/pyvista code ?

pyvista/plotting/themes.py:        self._jupyter_extension_available = 'TRAME_JUPYTER_WWW' in os.environ

# and the failing code
File /opt/tljh/user/lib/python3.10/site-packages/pyvista/plotting/themes.py:1501, in _TrameConfig.jupyter_extension_enabled(self, enabled)
   1498 @jupyter_extension_enabled.setter
   1499 def jupyter_extension_enabled(self, enabled: bool):  # numpydoc ignore=GL08
   1500     if enabled and not self.jupyter_extension_available:
-> 1501         raise ValueError("The trame_jupyter_extension is not available")

Also I don't see where you install that extension (trame-jupyter-extension)

This is pip install pyvista[jupyter] you have recommended it in a previous post But, looking at the link you've provided with, only jupyter_proxy_server is set as an optional dependency of pyvista, trame-jupyter-extension is not in the pyproject.toml of pyvista.

https://github.com/pyvista/pyvista/blob/main/pyproject.toml#L49

So how can have it been installed ? I'm surprised too !

I've copied/pasted my bash commands in the previous post. Nothing more has been installed, but trame-jupyter-extension is there.

jourdain commented 8 months ago

The TRAME_JUPYTER_WWW is set from this code that get executed on the kernel.
pip install pyvista[jupyter] install what is required for PyVista to work in Jupyter. But PyVista is not expecting or forcing the trame-jupyter-extension to be installed since the default is to rely on the server-proxy. That is up to you to install it if you want to use it, but it is not required for PyVista to work in Jupyter with trame. Since you change the default of PyVista, you switch to a more advanced mode that assume you've installed that extension yourself. And if you forget, the exception will let you know it is missing. If you want to install it, just add a pip install trame-jupyter-extension.

I think you bundle several issues into a single "it does not work" that you lost track of which advice was for what. Here is a split from what I can remember:

vtk-osmesa: VTK XLib rendering issue
pyvista[jupyter]: So trame, trame-vtk, trame-vuetify get installed
trame-jupyter-extension: Skip the need of having the JupyterServerProxy

MordicusEtCubitus commented 8 months ago

Yes, you're right. Thanks for putting all the issues in a nutshell and being so kind with my dummy errors

The TRAME_JUPYTER_WWW is set from manager.ts that get executed on the kernel.

Ok, I was searching it in site-package ! :(

So maybe I've got it: it was probably a cache issue, this could explain:

why last time it was working after rebooting
why it was not working then suddenly working without any change
why by just compiling vtk-osmesa without installing it, it suddenly worked: the cache expired while compiling

how trame-jupyter-extension has been installed as it is not a pyvista nor a pyvista-xarray dependency ?

Finally when looking at my bash history, at a time after several try I've installed it manually with pip, so my procedure need to be checked again

Will redo a full server installation more carefully again, so I should be able to provide a good installation procedure for TLJH...

Thanks a lot

Kitware / trame-jupyter-extension

PyVista + TLJH #4

Plot in 3D