search

jupyter / jupyter-sphinx

Sphinx extension for rendering of Jupyter interactive widgets.
https://jupyter-sphinx.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
186 stars 65 forks source link

importing a specific libs prevent the output display #184

Closed 12rambau closed 3 years ago

12rambau commented 3 years ago

Descripsion

As you can see on the image provided a normal jupyter-execute prints work but when I import my lib (sepal_ui), I get this strange display ending by "\n "}}}, "version_major": 2, "version_minor": 0}.

It seems related to https://github.com/jupyter-widgets/ipywidgets/issues/1305 but I don't get how it is link to this repository. Did someone already experiment this error and can tell me what's wrong in my lib ?

screenshot

Capture d’écran 2021-09-10 à 15 25 13

steps to reproduce

create a new project:

mkdir docs
cd docs 
sphinx-quickstart

then update your environment with the sepal_uilib:

pip install sepal_ui

update conf.py: 

# conf.py
# [...]

extensions = [
    'jupyter_sphinx',
]

# [...]

update index.rst:

.. jupyter-execute::

    print('hello ')

.. jupyter-execute::

    from sepal_ui import sepalwidgets as sw

build it and watch the produced single html page

akhmerov commented 3 years ago

As a quick check, can you please create a notebook that contains a cell from sepal_ui import sepalwidgets, execute it, and attach here?

Jupyter-sphinx extract the outputs from executing notebooks, and therefore whatever causes the problems should be in that notebook.

12rambau commented 3 years ago

sure: 

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "9ab504b3-fe94-46c6-943b-1c068dba18c6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "abc62f3369e249dfa319912be862f8c1",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Styles()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sepal_ui import sepalwidgets as sw "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
akhmerov commented 3 years ago

Thanks, it seems that the problem is due to ipywidgets. Let's investigate this further.

Firstly, can you confirm that jupyter-sphinx produces the same broken output with

.. jupyter-execute::

   import ipywidgets; ipywidgets.Button()

Then, what version of ipywidgets and sphinx do you have?

Finally, please search in the generated index.html of your file for code embedding the widget. For me it looks like this:

<script type="application/vnd.jupyter.widget-view+json">
{"version_major": 2, "version_minor": 0, "model_id": "17ba71bde1f8459990f9f385b58e36a4"}
</script>

Searching for mimetype should yield the result.

12rambau commented 3 years ago

Firstly, can you confirm that jupyter-sphinx produces the same broken output with

Nope, the result is perfectly normal when I import ipywidgets.

Then, what version of ipywidgets and sphinx do you have?

ipywidgets==7.6.3
Sphinx==4.0.3

I have in my page a first one: 

<div class="cell_output docutils container">
<script type="application/vnd.jupyter.widget-view+json">
{"version_major": 2, "version_minor": 0, "model_id": "3ca7430425ae45d8902065b442db8558"}
</script></div>

and then a huge one: 

<script type="application/vnd.jupyter.widget-state+json">
{"state": {"bb43da8e097447e3bc5bf776472e64c9": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "46022dd5827b44a69311f246c3ee778a": {"model_name": "ForceLoadModel", "model_module": "jupyter-vue", "model_module_version": "^1.5.0", "state": {"_dom_classes": [], "_model_module": "jupyter-vue", "_model_module_version": "^1.5.0", "_model_name": "ForceLoadModel", "_view_count": null, "_view_module": null, "_view_module_version": "", "_view_name": null, "layout": "IPY_MODEL_bb43da8e097447e3bc5bf776472e64c9"}}, "15742b8121314f77b1f6d0b1bb935e21": {"model_name": "ThemeModel", "model_module": "jupyter-vuetify", "model_module_version": "^1.8.1", "state": {"_model_module": "jupyter-vuetify", "_model_module_version": "^1.8.1", "_model_name": "ThemeModel", "_view_count": null, "_view_module": null, "_view_module_version": "^1.8.1", "_view_name": null, "dark": true}}, "92047bdefbb4480b84bd1ce3a30cc246": {"model_name": "ThemeColorsModel", "model_module": "jupyter-vuetify", "model_module_version": "^1.8.1", "state": {"_model_module": "jupyter-vuetify", "_model_module_version": "^1.8.1", "_model_name": "ThemeColorsModel", "_theme_name": "light", "_view_count": null, "_view_module": null, "_view_module_version": "^1.8.1", "_view_name": null, "accent": "#82B1FF", "anchor": null, "error": "#FF5252", "info": "#2196F3", "primary": "#1976D2", "secondary": "#424242", "success": "#4CAF50", "warning": "#FB8C00"}}, "e95ab358f9aa4bd9801152793470271b": {"model_name": "ThemeColorsModel", "model_module": "jupyter-vuetify", "model_module_version": "^1.8.1", "state": {"_model_module": "jupyter-vuetify", "_model_module_version": "^1.8.1", "_model_name": "ThemeColorsModel", "_theme_name": "dark", "_view_count": null, "_view_module": null, "_view_module_version": "^1.8.1", "_view_name": null, "accent": "#a1458e", "anchor": null, "error": "#A63228", "info": "#79B1C9", "primary": "#B3842E", "secondary": "#324a88", "success": "#3F802A", "warning": "#b8721d"}}, "4e48a32be045476fb82c3684149a80e3": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "3ca7430425ae45d8902065b442db8558": {"model_name": "VuetifyTemplateModel", "model_module": "jupyter-vuetify", "model_module_version": "^1.8.1", "state": {"_component_instances": [], "_dom_classes": [], "_jupyter_vue": "IPY_MODEL_46022dd5827b44a69311f246c3ee778a", "_model_module": "jupyter-vuetify", "_model_module_version": "^1.8.1", "_model_name": "VuetifyTemplateModel", "_view_count": null, "_view_module": "jupyter-vuetify", "_view_module_version": "^1.8.1", "_view_name": "VuetifyView", "components": null, "css": null, "data": null, "events": [], "layout": "IPY_MODEL_4e48a32be045476fb82c3684149a80e3", "methods": null, "template": "\n    <style>\n        .leaflet-pane {\n            z-index : 2 !important;\n        }\n        .leaflet-top, .leaflet-bottom {\n            z-index : 2 !important;\n        }\n        main.v-content {\n            padding-top: 0px !important;\n        }\n    </style>\n    "}}, "1d488d83c95b418bb28a0376159b4d41": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "58f377c006094988b871b3e742de4052": {"model_name": "VuetifyTemplateModel", "model_module": "jupyter-vuetify", "model_module_version": "^1.8.1", "state": {"_component_instances": [], "_dom_classes": [], "_jupyter_vue": "IPY_MODEL_46022dd5827b44a69311f246c3ee778a", "_model_module": "jupyter-vuetify", "_model_module_version": "^1.8.1", "_model_name": "VuetifyTemplateModel", "_view_count": null, "_view_module": "jupyter-vuetify", "_view_module_version": "^1.8.1", "_view_name": "VuetifyView", "components": null, "css": null, "data": null, "events": [], "layout": "IPY_MODEL_1d488d83c95b418bb28a0376159b4d41", "methods": null, "resize": 0, "template": "\n    <script>\n        modules.export = {\n            watch: {\n                resize() {\n                    window.dispatchEvent(new Event('resize'));\n                }\n            }\n        }\n        </script>\n    "}}}, "version_major": 2, "version_minor": 0}
</script>
12rambau commented 3 years ago

And I think I get what's the issue, the <scripts> is closed prematurely by the window.dispatchEvent(...) at the very end. (it's coming from here: https://github.com/12rambau/sepal_ui/blob/master/sepal_ui/frontend/js.py)

confirmed by this test: Capture d’écran 2021-09-10 à 17 19 26

12rambau commented 3 years ago

and now I have a minimal reproductible example: 


.. jupyter-execute::

    import ipyvuetify as v
    from traitlets import Unicode

    class ResizeTrigger(v.VuetifyTemplate):

        template = Unicode("<script></script>").tag(sync=True)

    rt = ResizeTrigger()

that leads to:

Capture d’écran 2021-09-10 à 18 25 55

akhmerov commented 3 years ago

Thanks! Now it's clear what is happening. We dump JSON over here:

https://github.com/jupyter/jupyter-sphinx/blob/master/jupyter_sphinx/ast.py#L394-L396

Since this is embedded in html, and not saved as a separate asset, we'd need to escape the JSON for HTML first. Not immediately sure how to do it though.

12rambau commented 3 years ago

I would be happy to contribute !

A first and easy fix is the replace </script> by &lt/scrip>because it's in the end the only one that can cause trouble but i don't know if it can have any other consequences in the lib

akhmerov commented 3 years ago

That part is self-contained and won't influence anything but the widget inclusion, I'm only unsure if that's the only symbol we need to escape, and if it would work correctly after escaping. Special-casing only backslash seems fragile. Having some source on this would help a lot.

12rambau commented 3 years ago

https://stackoverflow.com/questions/22488830/script-within-a-javascript-string-in-a-script-tag

https://stackoverflow.com/questions/1061697/whats-the-easiest-way-to-escape-html-in-python

from what I understand the only one that can interact is </script> as everything is included in a <script> tag. I experimented also with escape and it also work I'm just not sure it's necessary.

For my small usecase it works and everything is displayed as expected on screen.

edit

If the output cell include "<" or ">" I don't think we want them to be escaped so a dedicated replace seems to be the safest to me.