Figure rendering (Plotly)

[phockett]

• [x] Fix Plotly rendering issues in HTML output, probably Sphinx theme as per Read The Docs similar issues, see https://github.com/phockett/ePSproc/issues/27#issuecomment-1023389063 for solutions. FIXED IN https://jupyterbook.org/en/stable/interactive/interactive.html#plotly
• [x] Plotly issues with Figure captions and glue, see https://github.com/executablebooks/jupyter-book/issues/1815. Seems to work with Holoviews wrapper OK however. For more tests, see doc-source/part1/theory_101122-Render-debug.html (local tests only so far).

May also expect to see Plotly doesn't render after maths bug, although haven't had an issue with this as yet in HTML or PDF outputs.

[phockett]

Fixed Plotly display in HTML output in 0e7e7fb4bba9d751fb3c75157b98b59866c501bf, needs Plotly js in config file, as per https://jupyterbook.org/en/stable/interactive/interactive.html#plotly.

For rendering tests, see https://phockett.github.io/Quantum-Metrology-with-Photoelectrons-Vol3/tests/plotly_surface_render_test.html

[phockett]

Plotly status

• For HTML, set in _config.py as per above (0e7e7fb4bba9d751fb3c75157b98b59866c501bf)

• For Glue, now working with Panel wrapper (method borrowed from Holoviews), see https://github.com/executablebooks/jupyter-book/issues/1815#issuecomment-1317637875. This is now implemented in setup_notebook.py (see 6bc38bb715feab2d93bfe6b246b607d2ad8b0bec), but note this needs to be at top of import chain (not sure why, possibly HV or other plotter imports/defaults?).

• For PDF not rendering currently, likely need solution to force image render with type as per https://github.com/executablebooks/jupyter-book/issues/1410 (may be fixed with in-notebook setting, or possibly on back-end if paths issue).

  • Example solution above uses explicit Plotly render settings with

    
    import plotly.io as pio
    pio.renderers.default = "png"```

• Note old test notebooks, from ipynb with no execution, and no glue, were OK for Plotly fig display in PDF, but new notebooks with execute and no glue don't render - notebook settings to check.

  • UPDATE 18/11/22: clean build with no execution for ePSproc_class_demo_161020.ipynb gives Plotly figs in PDF OK, but now missing in HTML build? Plotly lib version issue, or execute vs. non-execute paths? Figs in this case are in _build/latex/ePSproc_class_demo_161020_23_2.png and ePSproc_class_demo_161020_23_4.png
  • Issue with cached vs. uncached/non-executed notebooks? Still not clear why some old notebooks do include Plotly output for PDF build case, but must be to do with this. May need to clean and rebuild for PDF case only? NOPE, pure PDF build chain doesn't fix.
  • Related to MIME type data present in notebooks? See https://nbformat.readthedocs.io/en/5.1.3/format_description.html#display-data and MyST-NB docs https://myst-nb.readthedocs.io/en/latest/configuration.html#rendering

• May need to do solution with env var to define builder? Could add to setup script if always executing notebooks at build...? Alternatively local execution then always build from ipynb might work?

  • 18/11/22 Testing env passing option in setup_notebook.py script... failing due to lack of env passing to build chain. See #1 .

[phockett]

Possible solutions for build-type dependence:

• Plotly offline mode, https://stackoverflow.com/a/73546992
• Sphinx render priority, https://jupyterbook.org/en/stable/content/code-outputs.html?highlight=glue%20render#render-priority
• MyST-NB render priority, https://myst-nb.readthedocs.io/en/latest/render/format_code_cells.html?highlight=nb_mime_priority_overrides#outputs-mime-priority (this looks OK, suggesting issue is with forcing Plotly output...?)
• Per-notebook build-env config for plotly as above, see Plotly docs for more options and static image output https://plotly.com/python/static-image-export/ and render source code https://github.com/plotly/plotly.py/blob/master/packages/python/plotly/plotly/io/_base_renderers.py
• Panel options/save for use with Plotly + Glue solution above? E.g. https://panel.holoviz.org/user_guide/Display_and_Export.html#saving and Plotly object options https://panel.holoviz.org/reference/panes/Plotly.html
• nbsphinx config and env, see https://nbsphinx.readthedocs.io/en/latest/configuring-kernels.html#Kernel-Arguments

[phockett]

Basic solution using setup_notebook.py bodge now in place, see 33e23be7c6d797a6d0406a86ad568cbf98067d3d.

Tested OK for PDF build for non-glue wrapped Plotly figs, provided force notebook execution.

Needs work, also need to fix for gluePlotly() case

• might be import ordering issue here? No - importing first doesn't fix.
• Or additional args to set for Panel...? See https://panel.holoviz.org/reference/panes/Plotly.html

[phockett]

Plotly rendering in PDF now fixed in fcd8653ca2f5ae2676d9ff0ba3c396e046457d16, using static figure additions to gluePlotly()

For details see https://github.com/executablebooks/jupyter-book/issues/1815#issuecomment-1322435036

Note this needs env settings for control over builds, defaults to HTML case. Will also only work generally for clean builds (needs to run per-notebook to set fig options).

For dev stuff, see tests/plotly_pdf_export_test_181122.ipynb. Note that there may be better solutions here (e.g. with Panel wrapper options...?), but this is now working.

[phockett]

Added Holoviews/Bokeh interactive 2D plot in 77f5cf1734bd9e53f3d804539ea151435ede8d9d (theory/density matrix demo, raw cell output, didn't try glue) - this seems to break all other figures in chpt (both HTML and PDF)? Bug with maths, or issues with build chain/dependencies/fig options? May need to specify static output options for PDF?

Test notes in QM3/doc-source/tests/holoviews_render_tests_310123.ipynb - this is working OK for raw and glued Holoviews outputs in HTML format, but nothing in PDF - need to set some static options there it seems.

UPDATE 2: Now rendering also broken in tensor intro section? Issues with cached pages?

UPDATE 3: Rendering fixed after build with forced execution, PDF output now gives text describing HV object. Probably need to implement image write solution as per Plotly case here? Or pass subset of HV object to glue() (possibly display(HVobj))? For options see https://holoviews.org/user_guide/Plots_and_Renderers.html (see also MF recon notes and hvSave functionality, now in PEMtk.fit._plotters).

[phockett]

Now have glueDecorator() in place (see 8d9da059bdcc8bf7a3d3bf90be7b678c8b4c3bd0) for general patches. This is working for Plotly and Holoviews, PDF and HTML, per builds 09/02/23 14-03-21 version.

May still be some issues/weirdness with Plotly surfaces in some cases - TBC, but seems OK with Panel IF run early in build chain only.

TODO: further testing and tidy code.

[phockett]

Plotly PAD plots and formatting

Plot styles now updated in ePSproc (https://github.com/phockett/ePSproc/issues/27#issuecomment-1468435262). Need to fix HTML and static output for book. Currently appearing squashed or chopped in final outputs, so seems to be aspect and/or layout issue?

• Default Plotly PNG renders at 700x500, too small.
• HTML version also uses this effective size? Doesn't fit to page as per original plots in notebook, might be Panel wrapper issue?

Fixes:

• Manual layout fix to objects, e.g. fig.update_layout(height=1400, width=1400) works for all outputs.
• Layouts with <4 columns not too bad by default (although a bit small), also quite a lot of padding in static output?
• Difference between full and reduced Plotly HTML forms? Set at export via fig.write_html(f'{fName}.html', include_plotlyjs="cdn", full_html=False), but currently not running this explicitly.

UPDATE: As of 017025c5c83572dde75504fde177b3a34c432ed4, allow for passing of IMGHEIGHT and IMGWIDTH to setup_notebook.py, defaults to 800x1200 landscape format. Currently set for plotly exporters in glueDecorator(), and also passed to ep.plot.hvPlotters.setPlotters(width=imgWidth, height=imgHeight).

For Plotly HTML figs are a little wider than default JBook template, but OK. For PDF figs are OK, if a little small, so may still want some tweaking of the defaults.

NOTE that defaults are set per notebook, and can't currently override for glued figures - may want to add this as an option if more control is required.

TODO: way to dynamically change height? At the moment not working well for single row PADs... but OK for double row. Should scale appropriately at build if possible. Should all be in fig['layout']? See current padPlot code, also Panel docs e.g. https://panel.holoviz.org/reference/panes/Plotly.html