voila-dashboards / voila

Voilà turns Jupyter notebooks into standalone web applications
https://voila.readthedocs.io
Other
5.31k stars 497 forks source link

Browser continues heartbeat even though server has finished #1446

Closed ClaytonAstrom closed 4 months ago

ClaytonAstrom commented 4 months ago

Description

I'm noticing that some notebooks finish executing server side, while the front end hasn't received anything (stays on the spinner and continues sending the heartbeat). Eventually the kernel is culled due to inactivity and the front end believes the kernel died erroneously because of that. I've tried increasing kernel polling/culling timeouts, but it seems that if it's missed it's missed. Server side these logs look the same, except for when next the front end runs the heartbeat:

b'[Voila] Kernel shutdown: 50d767d5-22b1-4bde-9d52-80a2a8679cf7\n'
b'[Voila] ERROR | Kernel died while waiting for execute reply.\n'
b"[Voila] ERROR | Error at server while executing cell: ...

I'm not sure why it would say there's a server side error though despite the execution finishing above and the kernel going idle.

I should also mention this is very inconsistent, even for a specific notebook. Some notebooks never have this error, some sometimes, and others frequently (still not 100% of the time though). Which to me says something is behaving incorrectly with the async part.

Context

Troubleshoot Output
$PATH:
    /opt/conda/bin
    /opt/conda/condabin
    /opt/conda/bin
    /usr/local/sbin
    /usr/local/bin
    /usr/sbin
    /usr/bin
    /sbin
    /bin

sys.path:
    /opt/conda/bin
    /opt/conda/lib/python39.zip
    /opt/conda/lib/python3.9
    /opt/conda/lib/python3.9/lib-dynload
    /opt/conda/lib/python3.9/site-packages

sys.executable:
    /opt/conda/bin/python3.9

sys.version:
    3.9.16 | packaged by conda-forge | (main, Feb  1 2023, 21:39:03)
    [GCC 11.3.0]

platform.platform():
    Linux-6.4.16-linuxkit-x86_64-with-glibc2.31

which -a jupyter:
    /opt/conda/bin/jupyter
    /opt/conda/bin/jupyter

pip list:
    Package                   Version
    ------------------------- ---------------
    aiohttp                   3.9.3
    aiosignal                 1.3.1
    alembic                   1.13.1
    anyio                     4.3.0
    argon2-cffi               23.1.0
    argon2-cffi-bindings      21.2.0
    arrow                     1.3.0
    async-generator           1.10
    async-timeout             4.0.3
    attrs                     23.2.0
    Babel                     2.14.0
    backcall                  0.2.0
    beautifulsoup4            4.12.3
    bleach                    6.1.0
    blinker                   1.7.0
    brotlipy                  0.7.0
    cachetools                5.3.2
    certifi                   2024.2.2
    certipy                   0.1.3
    cffi                      1.15.0
    charset-normalizer        2.0.12
    click                     8.1.7
    colorama                  0.4.4
    conda                     4.12.0
    conda-package-handling    1.8.1
    conda-store               0.4.12rc9
    cryptography              42.0.4
    debugpy                   1.8.1
    decorator                 5.1.1
    defusedxml                0.7.1
    direct-dnalab-widgets     0.1.31
    entrypoints               0.4
    exceptiongroup            1.2.0
    fastjsonschema            2.19.1
    fqdn                      1.5.1
    frozenlist                1.4.1
    greenlet                  3.0.3
    idna                      3.3
    importlib-metadata        7.0.1
    importlib-resources       6.1.1
    ipykernel                 6.10.0
    ipython                   7.25.0
    ipython-genutils          0.2.0
    ipywidgets                8.0.0
    isoduration               20.11.0
    jedi                      0.19.1
    jhsingle-native-proxy     0.8.1
    Jinja2                    3.1.0
    json5                     0.9.17
    jsonpointer               2.4
    jsonschema                4.21.1
    jsonschema-specifications 2023.12.1
    jupyter_client            7.4.9
    jupyter_core              4.12.0
    jupyter-events            0.9.0
    jupyter_server            2.3.0
    jupyter_server_terminals  0.5.2
    jupyter-telemetry         0.1.0
    jupyterhub                4.0.2
    jupyterlab_pygments       0.3.0
    jupyterlab_server         2.25.3
    jupyterlab_widgets        3.0.10
    libmambapy                1.1.0
    Mako                      1.3.2
    mamba                     1.1.0
    markdown-it-py            3.0.0
    MarkupSafe                2.1.5
    matplotlib-inline         0.1.6
    mdurl                     0.1.2
    mistune                   0.8.4
    multidict                 6.0.5
    nb_conda_store_kernels    0.1.5rc27
    nbclient                  0.5.13
    nbconvert                 6.4.5
    nbformat                  5.3.0
    nest_asyncio              1.6.0
    oauthlib                  3.2.2
    packaging                 23.2
    pamela                    1.1.0
    pandocfilters             1.5.1
    parso                     0.8.3
    pexpect                   4.9.0
    pickleshare               0.7.5
    pip                       22.1.2
    pkgutil_resolve_name      1.3.10
    platformdirs              4.2.0
    prometheus_client         0.20.0
    prompt-toolkit            3.0.42
    psutil                    5.9.8
    ptyprocess                0.7.0
    pycosat                   0.6.3
    pycparser                 2.21
    pycurl                    7.45.1
    Pygments                  2.17.2
    PyJWT                     2.8.0
    pyOpenSSL                 22.0.0
    PySocks                   1.7.1
    python-dateutil           2.8.2
    python-json-logger        2.0.7
    PyYAML                    6.0.1
    pyzmq                     25.1.2
    referencing               0.33.0
    requests                  2.31.0
    rfc3339-validator         0.1.4
    rfc3986-validator         0.1.1
    rich                      13.7.0
    rpds-py                   0.18.0
    ruamel.yaml               0.18.6
    ruamel.yaml.clib          0.2.8
    ruamel-yaml-conda         0.15.80
    Send2Trash                1.8.2
    setuptools                62.3.2
    simpervisor               0.4
    six                       1.16.0
    sniffio                   1.3.0
    soupsieve                 2.5
    SQLAlchemy                2.0.27
    terminado                 0.18.0
    testpath                  0.6.0
    tornado                   6.2
    tqdm                      4.64.0
    traitlets                 5.6.0
    types-python-dateutil     2.8.19.20240106
    typing_extensions         4.9.0
    uri-template              1.3.0
    urllib3                   1.26.9
    voila                     0.5.5
    wcwidth                   0.2.13
    webcolors                 1.13
    webencodings              0.5.1
    websocket-client          1.7.0
    websockets                12.0
    wheel                     0.37.1
    widgetsnbextension        4.0.10
    yarl                      1.9.4
    zipp                      3.17.0

conda list:
    # packages in environment at /opt/conda:
    #
    # Name                    Version                   Build  Channel
    _libgcc_mutex             0.1                 conda_forge    conda-forge
    _openmp_mutex             4.5                       2_gnu    conda-forge
    aiohttp                   3.9.3            py39hd1e30aa_0    conda-forge
    aiosignal                 1.3.1              pyhd8ed1ab_0    conda-forge
    alembic                   1.13.1             pyhd8ed1ab_1    conda-forge
    anyio                     4.3.0                    pypi_0    pypi
    argon2-cffi               23.1.0                   pypi_0    pypi
    argon2-cffi-bindings      21.2.0                   pypi_0    pypi
    arrow                     1.3.0                    pypi_0    pypi
    async-timeout             4.0.3              pyhd8ed1ab_0    conda-forge
    async_generator           1.10                       py_0    conda-forge
    attrs                     23.2.0             pyh71513ae_0    conda-forge
    babel                     2.14.0                   pypi_0    pypi
    backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
    beautifulsoup4            4.12.3                   pypi_0    pypi
    bleach                    6.1.0                    pypi_0    pypi
    blinker                   1.7.0              pyhd8ed1ab_0    conda-forge
    brotlipy                  0.7.0           py39hb9d737c_1004    conda-forge
    bzip2                     1.0.8                h7f98852_4    conda-forge
    c-ares                    1.18.1               h7f98852_0    conda-forge
    ca-certificates           2024.2.2             hbcca054_0    conda-forge
    cachetools                5.3.2                    pypi_0    pypi
    certifi                   2024.2.2           pyhd8ed1ab_0    conda-forge
    certipy                   0.1.3                      py_0    conda-forge
    cffi                      1.15.0           py39h4bc2ebd_0    conda-forge
    charset-normalizer        2.0.12             pyhd8ed1ab_0    conda-forge
    click                     8.1.7           unix_pyh707e725_0    conda-forge
    colorama                  0.4.4              pyh9f0ad1d_0    conda-forge
    conda                     4.12.0           py39hf3d152e_0    conda-forge
    conda-package-handling    1.8.1            py39hb9d737c_1    conda-forge
    conda-store               0.4.12rc9                pypi_0    pypi
    configurable-http-proxy   4.5.6                h3b247e2_0    conda-forge
    cryptography              42.0.4           py39hd4f0224_0    conda-forge
    debugpy                   1.8.1            py39h3d6467e_0    conda-forge
    decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
    defusedxml                0.7.1                    pypi_0    pypi
    direct-dnalab-widgets     0.1.31                   pypi_0    pypi
    entrypoints               0.4                pyhd8ed1ab_0    conda-forge
    exceptiongroup            1.2.0                    pypi_0    pypi
    fmt                       9.1.0                h924138e_0    conda-forge
    fqdn                      1.5.1                    pypi_0    pypi
    frozenlist                1.4.1            py39hd1e30aa_0    conda-forge
    greenlet                  3.0.3            py39h3d6467e_0    conda-forge
    icu                       70.1                 h27087fc_0    conda-forge
    idna                      3.3                pyhd8ed1ab_0    conda-forge
    importlib-metadata        7.0.1              pyha770c72_0    conda-forge
    importlib_resources       6.1.1              pyhd8ed1ab_0    conda-forge
    ipykernel                 6.10.0           py39hef51801_0    conda-forge
    ipython                   7.25.0           py39hef51801_1    conda-forge
    ipython_genutils          0.2.0                      py_1    conda-forge
    ipywidgets                8.0.0              pyhd8ed1ab_0    conda-forge
    isoduration               20.11.0                  pypi_0    pypi
    jedi                      0.19.1             pyhd8ed1ab_0    conda-forge
    jhsingle-native-proxy     0.8.1              pyhd8ed1ab_0    conda-forge
    jinja2                    3.1.0              pyhd8ed1ab_0    conda-forge
    json5                     0.9.17                   pypi_0    pypi
    jsonpointer               2.4                      pypi_0    pypi
    jsonschema                4.21.1             pyhd8ed1ab_0    conda-forge
    jsonschema-specifications 2023.12.1          pyhd8ed1ab_0    conda-forge
    jupyter-core              4.12.0                   pypi_0    pypi
    jupyter-events            0.9.0                    pypi_0    pypi
    jupyter-server            2.3.0                    pypi_0    pypi
    jupyter-server-terminals  0.5.2                    pypi_0    pypi
    jupyter_client            7.4.9              pyhd8ed1ab_0    conda-forge
    jupyter_telemetry         0.1.0              pyhd8ed1ab_1    conda-forge
    jupyterhub                4.0.2              pyh31011fe_0    conda-forge
    jupyterhub-base           4.0.2              pyh31011fe_0    conda-forge
    jupyterlab-pygments       0.3.0                    pypi_0    pypi
    jupyterlab-server         2.25.3                   pypi_0    pypi
    jupyterlab_widgets        3.0.10             pyhd8ed1ab_0    conda-forge
    keyutils                  1.6.1                h166bdaf_0    conda-forge
    krb5                      1.21.2               h659d440_0    conda-forge
    ld_impl_linux-64          2.36.1               hea4e1c9_2    conda-forge
    libarchive                3.6.2                h3d51595_0    conda-forge
    libcurl                   8.2.1                hca28451_0    conda-forge
    libedit                   3.1.20191231         he28a2e2_2    conda-forge
    libev                     4.33                 h516909a_1    conda-forge
    libffi                    3.4.2                h7f98852_5    conda-forge
    libgcc-ng                 12.1.0              h8d9b700_16    conda-forge
    libgomp                   12.1.0              h8d9b700_16    conda-forge
    libiconv                  1.17                 hd590300_2    conda-forge
    libmamba                  1.1.0                hde2b089_3    conda-forge
    libmambapy                1.1.0            py39hf0aba66_3    conda-forge
    libnghttp2                1.52.0               h61bc06f_0    conda-forge
    libnsl                    2.0.0                h7f98852_0    conda-forge
    libsodium                 1.0.18               h36c2ea0_1    conda-forge
    libsolv                   0.7.28               hfc55251_0    conda-forge
    libsqlite                 3.45.1               h2797004_0    conda-forge
    libssh2                   1.11.0               h0841786_0    conda-forge
    libstdcxx-ng              12.1.0              ha89aaad_16    conda-forge
    libuuid                   2.32.1            h7f98852_1000    conda-forge
    libuv                     1.44.2               hd590300_1    conda-forge
    libxml2                   2.10.3               hca2bb57_4    conda-forge
    libzlib                   1.2.13               hd590300_5    conda-forge
    lz4-c                     1.9.3                h9c3ff4c_1    conda-forge
    lzo                       2.10              h516909a_1000    conda-forge
    mako                      1.3.2              pyhd8ed1ab_0    conda-forge
    mamba                     1.1.0            py39hc5d2bb1_3    conda-forge
    markdown-it-py            3.0.0                    pypi_0    pypi
    markupsafe                2.1.5            py39hd1e30aa_0    conda-forge
    matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
    mdurl                     0.1.2                    pypi_0    pypi
    mistune                   0.8.4                    pypi_0    pypi
    multidict                 6.0.5            py39hd1e30aa_0    conda-forge
    nb-conda-store-kernels    0.1.5rc27                pypi_0    pypi
    nbclient                  0.5.13                   pypi_0    pypi
    nbconvert                 6.4.5                    pypi_0    pypi
    nbformat                  5.3.0                    pypi_0    pypi
    ncurses                   6.3                  h27087fc_1    conda-forge
    nest-asyncio              1.6.0              pyhd8ed1ab_0    conda-forge
    nodejs                    16.19.0              h8d033a5_0    conda-forge
    oauthlib                  3.2.2              pyhd8ed1ab_0    conda-forge
    openssl                   3.2.1                hd590300_0    conda-forge
    packaging                 23.2               pyhd8ed1ab_0    conda-forge
    pamela                    1.1.0              pyh1a96a4e_0    conda-forge
    pandocfilters             1.5.1                    pypi_0    pypi
    parso                     0.8.3              pyhd8ed1ab_0    conda-forge
    pexpect                   4.9.0              pyhd8ed1ab_0    conda-forge
    pickleshare               0.7.5                   py_1003    conda-forge
    pip                       22.1.2             pyhd8ed1ab_0    conda-forge
    pkgutil-resolve-name      1.3.10             pyhd8ed1ab_1    conda-forge
    platformdirs              4.2.0              pyhd8ed1ab_0    conda-forge
    prometheus_client         0.20.0             pyhd8ed1ab_0    conda-forge
    prompt-toolkit            3.0.42             pyha770c72_0    conda-forge
    psutil                    5.9.8            py39hd1e30aa_0    conda-forge
    ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
    pybind11-abi              4                    hd8ed1ab_3    conda-forge
    pycosat                   0.6.3           py39hb9d737c_1010    conda-forge
    pycparser                 2.21               pyhd8ed1ab_0    conda-forge
    pycurl                    7.45.1           py39h9297c8b_3    conda-forge
    pygments                  2.17.2             pyhd8ed1ab_0    conda-forge
    pyjwt                     2.8.0              pyhd8ed1ab_1    conda-forge
    pyopenssl                 22.0.0                   pypi_0    pypi
    pysocks                   1.7.1            py39hf3d152e_5    conda-forge
    python                    3.9.16          h2782a2a_0_cpython    conda-forge
    python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
    python-fastjsonschema     2.19.1             pyhd8ed1ab_0    conda-forge
    python-json-logger        2.0.7              pyhd8ed1ab_0    conda-forge
    python_abi                3.9                      2_cp39    conda-forge
    pyyaml                    6.0.1                    pypi_0    pypi
    pyzmq                     25.1.2           py39h8c080ef_0    conda-forge
    readline                  8.2                  h8228510_1    conda-forge
    referencing               0.33.0             pyhd8ed1ab_0    conda-forge
    reproc                    14.2.3               h7f98852_0    conda-forge
    reproc-cpp                14.2.3               h9c3ff4c_0    conda-forge
    requests                  2.31.0                   pypi_0    pypi
    rfc3339-validator         0.1.4                    pypi_0    pypi
    rfc3986-validator         0.1.1                    pypi_0    pypi
    rich                      13.7.0                   pypi_0    pypi
    rpds-py                   0.18.0           py39h9fdd4d6_0    conda-forge
    ruamel.yaml               0.18.6           py39hd1e30aa_0    conda-forge
    ruamel.yaml.clib          0.2.8            py39hd1e30aa_0    conda-forge
    ruamel_yaml               0.15.80         py39hb9d737c_1007    conda-forge
    send2trash                1.8.2                    pypi_0    pypi
    setuptools                62.3.2           py39hf3d152e_0    conda-forge
    simpervisor               0.4                pyhd8ed1ab_0    conda-forge
    six                       1.16.0             pyh6c4a22f_0    conda-forge
    sniffio                   1.3.0                    pypi_0    pypi
    soupsieve                 2.5                      pypi_0    pypi
    sqlalchemy                2.0.27           py39hd1e30aa_0    conda-forge
    sqlite                    3.38.5               h4ff8645_0    conda-forge
    terminado                 0.18.0                   pypi_0    pypi
    testpath                  0.6.0                    pypi_0    pypi
    tk                        8.6.12               h27826a3_0    conda-forge
    tornado                   6.2                      pypi_0    pypi
    tqdm                      4.64.0             pyhd8ed1ab_0    conda-forge
    traitlets                 5.6.0                    pypi_0    pypi
    types-python-dateutil     2.8.19.20240106          pypi_0    pypi
    typing-extensions         4.9.0                hd8ed1ab_0    conda-forge
    typing_extensions         4.9.0              pyha770c72_0    conda-forge
    tzdata                    2022a                h191b570_0    conda-forge
    uri-template              1.3.0                    pypi_0    pypi
    urllib3                   1.26.9             pyhd8ed1ab_0    conda-forge
    voila                     0.5.5                    pypi_0    pypi
    wcwidth                   0.2.13             pyhd8ed1ab_0    conda-forge
    webcolors                 1.13                     pypi_0    pypi
    webencodings              0.5.1                    pypi_0    pypi
    websocket-client          1.7.0                    pypi_0    pypi
    websockets                12.0                     pypi_0    pypi
    wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
    widgetsnbextension        4.0.10             pyhd8ed1ab_0    conda-forge
    xz                        5.2.6                h166bdaf_0    conda-forge
    yaml                      0.2.5                h7f98852_2    conda-forge
    yaml-cpp                  0.7.0                h59595ed_3    conda-forge
    yarl                      1.9.4            py39hd1e30aa_0    conda-forge
    zeromq                    4.3.5                h59595ed_0    conda-forge
    zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
    zlib                      1.2.13               hd590300_5    conda-forge
    zstd                      1.5.2                h8a70e8d_1    conda-forge

conda env:
    name: base
    channels:
      - conda-forge
    dependencies:
      - _libgcc_mutex=0.1=conda_forge
      - _openmp_mutex=4.5=2_gnu
      - aiohttp=3.9.3=py39hd1e30aa_0
      - aiosignal=1.3.1=pyhd8ed1ab_0
      - alembic=1.13.1=pyhd8ed1ab_1
      - async-timeout=4.0.3=pyhd8ed1ab_0
      - async_generator=1.10=py_0
      - attrs=23.2.0=pyh71513ae_0
      - backcall=0.2.0=pyh9f0ad1d_0
      - blinker=1.7.0=pyhd8ed1ab_0
      - brotlipy=0.7.0=py39hb9d737c_1004
      - bzip2=1.0.8=h7f98852_4
      - c-ares=1.18.1=h7f98852_0
      - ca-certificates=2024.2.2=hbcca054_0
      - certifi=2024.2.2=pyhd8ed1ab_0
      - certipy=0.1.3=py_0
      - cffi=1.15.0=py39h4bc2ebd_0
      - charset-normalizer=2.0.12=pyhd8ed1ab_0
      - click=8.1.7=unix_pyh707e725_0
      - colorama=0.4.4=pyh9f0ad1d_0
      - conda=4.12.0=py39hf3d152e_0
      - conda-package-handling=1.8.1=py39hb9d737c_1
      - configurable-http-proxy=4.5.6=h3b247e2_0
      - cryptography=42.0.4=py39hd4f0224_0
      - debugpy=1.8.1=py39h3d6467e_0
      - decorator=5.1.1=pyhd8ed1ab_0
      - entrypoints=0.4=pyhd8ed1ab_0
      - fmt=9.1.0=h924138e_0
      - frozenlist=1.4.1=py39hd1e30aa_0
      - greenlet=3.0.3=py39h3d6467e_0
      - icu=70.1=h27087fc_0
      - idna=3.3=pyhd8ed1ab_0
      - importlib-metadata=7.0.1=pyha770c72_0
      - importlib_resources=6.1.1=pyhd8ed1ab_0
      - ipykernel=6.10.0=py39hef51801_0
      - ipython=7.25.0=py39hef51801_1
      - ipython_genutils=0.2.0=py_1
      - ipywidgets=8.0.0=pyhd8ed1ab_0
      - jedi=0.19.1=pyhd8ed1ab_0
      - jhsingle-native-proxy=0.8.1=pyhd8ed1ab_0
      - jinja2=3.1.0=pyhd8ed1ab_0
      - jsonschema=4.21.1=pyhd8ed1ab_0
      - jsonschema-specifications=2023.12.1=pyhd8ed1ab_0
      - jupyter_client=7.4.9=pyhd8ed1ab_0
      - jupyter_telemetry=0.1.0=pyhd8ed1ab_1
      - jupyterhub=4.0.2=pyh31011fe_0
      - jupyterhub-base=4.0.2=pyh31011fe_0
      - jupyterlab_widgets=3.0.10=pyhd8ed1ab_0
      - keyutils=1.6.1=h166bdaf_0
      - krb5=1.21.2=h659d440_0
      - ld_impl_linux-64=2.36.1=hea4e1c9_2
      - libarchive=3.6.2=h3d51595_0
      - libcurl=8.2.1=hca28451_0
      - libedit=3.1.20191231=he28a2e2_2
      - libev=4.33=h516909a_1
      - libffi=3.4.2=h7f98852_5
      - libgcc-ng=12.1.0=h8d9b700_16
      - libgomp=12.1.0=h8d9b700_16
      - libiconv=1.17=hd590300_2
      - libmamba=1.1.0=hde2b089_3
      - libmambapy=1.1.0=py39hf0aba66_3
      - libnghttp2=1.52.0=h61bc06f_0
      - libnsl=2.0.0=h7f98852_0
      - libsodium=1.0.18=h36c2ea0_1
      - libsolv=0.7.28=hfc55251_0
      - libsqlite=3.45.1=h2797004_0
      - libssh2=1.11.0=h0841786_0
      - libstdcxx-ng=12.1.0=ha89aaad_16
      - libuuid=2.32.1=h7f98852_1000
      - libuv=1.44.2=hd590300_1
      - libxml2=2.10.3=hca2bb57_4
      - libzlib=1.2.13=hd590300_5
      - lz4-c=1.9.3=h9c3ff4c_1
      - lzo=2.10=h516909a_1000
      - mako=1.3.2=pyhd8ed1ab_0
      - mamba=1.1.0=py39hc5d2bb1_3
      - markupsafe=2.1.5=py39hd1e30aa_0
      - matplotlib-inline=0.1.6=pyhd8ed1ab_0
      - multidict=6.0.5=py39hd1e30aa_0
      - ncurses=6.3=h27087fc_1
      - nest-asyncio=1.6.0=pyhd8ed1ab_0
      - nodejs=16.19.0=h8d033a5_0
      - oauthlib=3.2.2=pyhd8ed1ab_0
      - openssl=3.2.1=hd590300_0
      - packaging=23.2=pyhd8ed1ab_0
      - pamela=1.1.0=pyh1a96a4e_0
      - parso=0.8.3=pyhd8ed1ab_0
      - pexpect=4.9.0=pyhd8ed1ab_0
      - pickleshare=0.7.5=py_1003
      - pip=22.1.2=pyhd8ed1ab_0
      - pkgutil-resolve-name=1.3.10=pyhd8ed1ab_1
      - platformdirs=4.2.0=pyhd8ed1ab_0
      - prometheus_client=0.20.0=pyhd8ed1ab_0
      - prompt-toolkit=3.0.42=pyha770c72_0
      - psutil=5.9.8=py39hd1e30aa_0
      - ptyprocess=0.7.0=pyhd3deb0d_0
      - pybind11-abi=4=hd8ed1ab_3
      - pycosat=0.6.3=py39hb9d737c_1010
      - pycparser=2.21=pyhd8ed1ab_0
      - pycurl=7.45.1=py39h9297c8b_3
      - pygments=2.17.2=pyhd8ed1ab_0
      - pyjwt=2.8.0=pyhd8ed1ab_1
      - pysocks=1.7.1=py39hf3d152e_5
      - python=3.9.16=h2782a2a_0_cpython
      - python-dateutil=2.8.2=pyhd8ed1ab_0
      - python-fastjsonschema=2.19.1=pyhd8ed1ab_0
      - python-json-logger=2.0.7=pyhd8ed1ab_0
      - python_abi=3.9=2_cp39
      - pyzmq=25.1.2=py39h8c080ef_0
      - readline=8.2=h8228510_1
      - referencing=0.33.0=pyhd8ed1ab_0
      - reproc=14.2.3=h7f98852_0
      - reproc-cpp=14.2.3=h9c3ff4c_0
      - rpds-py=0.18.0=py39h9fdd4d6_0
      - ruamel.yaml=0.18.6=py39hd1e30aa_0
      - ruamel.yaml.clib=0.2.8=py39hd1e30aa_0
      - ruamel_yaml=0.15.80=py39hb9d737c_1007
      - setuptools=62.3.2=py39hf3d152e_0
      - simpervisor=0.4=pyhd8ed1ab_0
      - six=1.16.0=pyh6c4a22f_0
      - sqlalchemy=2.0.27=py39hd1e30aa_0
      - sqlite=3.38.5=h4ff8645_0
      - tk=8.6.12=h27826a3_0
      - tqdm=4.64.0=pyhd8ed1ab_0
      - typing-extensions=4.9.0=hd8ed1ab_0
      - typing_extensions=4.9.0=pyha770c72_0
      - tzdata=2022a=h191b570_0
      - urllib3=1.26.9=pyhd8ed1ab_0
      - wcwidth=0.2.13=pyhd8ed1ab_0
      - wheel=0.37.1=pyhd8ed1ab_0
      - widgetsnbextension=4.0.10=pyhd8ed1ab_0
      - xz=5.2.6=h166bdaf_0
      - yaml=0.2.5=h7f98852_2
      - yaml-cpp=0.7.0=h59595ed_3
      - yarl=1.9.4=py39hd1e30aa_0
      - zeromq=4.3.5=h59595ed_0
      - zipp=3.17.0=pyhd8ed1ab_0
      - zlib=1.2.13=hd590300_5
      - zstd=1.5.2=h8a70e8d_1
      - pip:
        - anyio==4.3.0
        - argon2-cffi==23.1.0
        - argon2-cffi-bindings==21.2.0
        - arrow==1.3.0
        - babel==2.14.0
        - beautifulsoup4==4.12.3
        - bleach==6.1.0
        - cachetools==5.3.2
        - conda-store==0.4.12rc9
        - defusedxml==0.7.1
        - direct-dnalab-widgets==0.1.31
        - exceptiongroup==1.2.0
        - fqdn==1.5.1
        - isoduration==20.11.0
        - json5==0.9.17
        - jsonpointer==2.4
        - jupyter-core==4.12.0
        - jupyter-events==0.9.0
        - jupyter-server==2.3.0
        - jupyter-server-terminals==0.5.2
        - jupyterlab-pygments==0.3.0
        - jupyterlab-server==2.25.3
        - markdown-it-py==3.0.0
        - mdurl==0.1.2
        - mistune==0.8.4
        - nb-conda-store-kernels==0.1.5rc27
        - nbclient==0.5.13
        - nbconvert==6.4.5
        - nbformat==5.3.0
        - pandocfilters==1.5.1
        - pyopenssl==22.0.0
        - pyyaml==6.0.1
        - requests==2.31.0
        - rfc3339-validator==0.1.4
        - rfc3986-validator==0.1.1
        - rich==13.7.0
        - send2trash==1.8.2
        - sniffio==1.3.0
        - soupsieve==2.5
        - terminado==0.18.0
        - testpath==0.6.0
        - tornado==6.2
        - traitlets==5.6.0
        - types-python-dateutil==2.8.19.20240106
        - uri-template==1.3.0
        - voila==0.5.5
        - webcolors==1.13
        - webencodings==0.5.1
        - websocket-client==1.7.0
        - websockets==12.0
    prefix: /opt/conda
Command Line Output
[Voila] Looking for voila in /etc/jupyter
[Voila] Looking for voila in /usr/local/etc/jupyter
[Voila] Looking for voila in /opt/conda/etc/jupyter
[Voila] Looking for voila in /home/voila-runner/.local/etc/jupyter
[Voila] Looking for voila in /home/voila-runner/.jupyter
[Voila] Looking for voila in /home/voila-runner
[Voila] using template: lab
[Voila] template paths:
    /opt/conda/share/jupyter/voila/templates/lab
    /opt/conda/share/jupyter/nbconvert/templates/lab
    /opt/conda/share/jupyter/voila/templates/base
    /opt/conda/share/jupyter/nbconvert/templates/base
    /home/voila-runner/.local/share/jupyter
    /home/voila-runner/.local/share/jupyter/voila/templates
    /home/voila-runner/.local/share/jupyter/nbconvert/templates
    /opt/conda/share/jupyter
    /opt/conda/share/jupyter/voila/templates
    /opt/conda/share/jupyter/nbconvert/templates
    /usr/local/share/jupyter
    /usr/local/share/jupyter/voila/templates
    /usr/local/share/jupyter/nbconvert/templates
    /usr/share/jupyter
    /usr/share/jupyter/voila/templates
    /usr/share/jupyter/nbconvert/templates
[Voila] static paths:
    /home/voila-runner/.local/share/jupyter/voila/templates/lab/static
    /home/voila-runner/.local/share/jupyter/nbconvert/templates/lab/static
    /opt/conda/share/jupyter/voila/templates/lab/static
    /opt/conda/share/jupyter/nbconvert/templates/lab/static
    /usr/local/share/jupyter/voila/templates/lab/static
    /usr/local/share/jupyter/nbconvert/templates/lab/static
    /usr/share/jupyter/voila/templates/lab/static
    /usr/share/jupyter/nbconvert/templates/lab/static
    /home/voila-runner/.local/share/jupyter/voila/templates/base/static
    /home/voila-runner/.local/share/jupyter/nbconvert/templates/base/static
    /opt/conda/share/jupyter/voila/templates/base/static
    /opt/conda/share/jupyter/nbconvert/templates/base/static
    /usr/local/share/jupyter/voila/templates/base/static
    /usr/local/share/jupyter/nbconvert/templates/base/static
    /usr/share/jupyter/voila/templates/base/static
    /usr/share/jupyter/nbconvert/templates/base/static
    /opt/conda/lib/python3.9/site-packages/jupyter_server/static
[Voila] Using /tmp to store connection files
[Voila] Storing connection files in /tmp/voila_x3bk1n9n.
[Voila] Serving static files from /opt/conda/lib/python3.9/site-packages/voila/static.
[Voila] serving directory: '/home/voila-runner'
[Voila] Voilà is running at:
http://localhost:8866/
[Voila] WARNING | No web browser found: could not locate runnable browser.
Browser Output
Ok, voila is still executing... (x10)
GET /api/kernels/ 404
Plugin '@jupyter-widgets/jupyterlab-manager:base-2.0.0' failed to activate.
Error: The model for kernel id 50d767d5-22b1-4bde-9d52-80a2a8679cf7 does not exist
Plugin '@jupyter-widgets/jupyterlab-manager:controls-2.0.0' failed to activate.
Error: The model for kernel id 50d767d5-22b1-4bde-9d52-80a2a8679cf7 does not exist
...
ClaytonAstrom commented 4 months ago

Maybe there's something I've missed, this is a part of our upgrade from 0.4 to 0.5. The notebooks run fine in 0.4.

Something I've noticed is that server side in 0.4 says there's a websocket log - Initializing websocket connection /api/kernels/...

However, even on a successful run in 0.5, it skips this and prints the immediate next log - Requesting kernel info from...

trungleduc commented 4 months ago

Hi, can you post a simplified notebook to reproduce the issue? Might be related to https://github.com/voila-dashboards/voila/issues/1428#issuecomment-1927025620 ?

ClaytonAstrom commented 4 months ago

Thanks @trungleduc - I took the simplified notebook you posted in that issue and I can see it seems to be the same issue. It's not consistent 100% of the time, that's also in line with the notebooks I'm testing as well. But the logs and error seem to match the notebooks I'm struggling with, so I'm willing to bet it's the same issue.

I'll close this one out then and keep an eye on the issue you linked.