Plotting routines

[phockett]

To do:

• [ ] Better line plots - Holoviews with Bokeh back end? See tests in geometric_method_dev_low-level_E-fields_200320.ipynb, plus XeF2 test notebooks & old data processing notebooks.
• [ ] Consolidate improved plotting routines for all Xarray objects.
• [ ] Update modified Seaborn routines used in lmPlot(), this currently needs Seaborn==0.9.0

[phockett]

Improved XC plotting with Holoviews + Bokeh: https://epsproc.readthedocs.io/en/dev/tests/basicPlotting_dev_XC_030720.html

Functionalised version now in plot/hvPlotters.py

[phockett]

Also: fix horrible plotting code now propagated and bodged into class functions. This is fragile depending on data type and dimensions. Needs unifying.

[phockett]

Now added general curve plotting routine with Holoviews, see 6030520d2894940c98e3b49af48c419de31ecc6e

Main routine is hvPlotters.curvePlot(). This needs to be further wrapped for specific common plots and class use.

Update: see https://epsproc.readthedocs.io/en/dev/tests/hvPlotters_fn_tests_150720_v250122.html for some usage examples.

Also: in many cases hvPlot() is easiest/quickest from Xarray, e.g. https://epsproc.readthedocs.io/en/dev/tests/hvPlotters_fn_tests_150720_v250122.html#hvPlot-pipeline

See also OCS dev work for simplified BLM plotter routine, may be better than current methods. https://phockett.github.io/ePSdata/OCS-preliminary/OCS_orbs8-11_AFBLMs_VM-ADMs_140122-JAKE_tidy.html#AFBLMs-for-aligned-case

UPDATE: also plotters in https://phockett.github.io/ePSdata/OCS-preliminary/OCS_orbs8-11_AFBLMs_VM-ADMs_140122-JAKE_tidy-replot-200722_v5.html

[phockett]

Improved basicPlotters.BLMplot() routine and added Holoviews backend, see 6bcc112d694ee17b05559678a9bb92e9f33df873

TODO:

• [x] wrap for classes.
• [ ] consolidate matPlot() routine from calc.density.
• [ ] improve dim handling - see also density routine for this, plus handling of multiindex labels.
• [ ] consolidate plot data setup routines to a separate shared function.
• [ ] style mapping (see XCplot routine for some basic examples).
• [ ] fix issue with Xarray unstack > HV dataset, leading to extra indices, e.g. (l=0,m!=0) values appear in legend.
• [x] fix issue with Plotly figures not rendering on ReadTheDocs. Saved pages render OK locally, so looks like RTD theme issue (not nbsphinx/sphinx or missing scripts), see https://github.com/readthedocs/readthedocs.org/issues/4367 and others (e.g. https://github.com/spatialaudio/nbsphinx/issues/128, https://github.com/spatialaudio/nbsphinx/pull/302)...
  • This should now be fixed, so may need to check RTD theme versions, see e.g. https://github.com/readthedocs/sphinx_rtd_theme/issues/788#issuecomment-625514867, I just tested sphinx_rtd_theme==0.5.0rc1 and the issue is fixed,.
  • Manual fix is to include scripts, e.g. https://github.com/readthedocs/sphinx_rtd_theme/issues/788#issuecomment-585785027
  • https://github.com/harupy/sphinx-plotly-directive may also be useful.
  • UPDATE: see notes below https://github.com/phockett/ePSproc/issues/27#issuecomment-1092261133

[phockett]

Fixed Plotly rendering issue in https://github.com/phockett/ePSproc/commit/bfd3771ec8d74e43c4f917a3b74b8f1bc14b6b68 (updates sphinx_rtd_theme >= 1.0.0 in requirements-RTD.txt).

UPDATE 14/11/22: this is NOT working on dev branch docs, not sure why. HV rendering on latest (master) branch is OK however. May just need to force a clean build?

UPDATE 13/07/23: further testing on 3d-AFPAD-dev branch.

• Added new branch to RTD, broken plots.
• Added various scripts to conf.py, under nbsphinx_prolog per fixes above, didn't fix. (E.g. https://github.com/readthedocs/sphinx_rtd_theme/issues/788#issuecomment-585785027, and some variations on this)
• To test:
  • Pushing file to _static, see https://github.com/readthedocs/sphinx_rtd_theme/issues/788#issuecomment-1158705893
  • Loading Holoviews, see https://github.com/plotly/plotly.py/issues/1033#issuecomment-1081908754
  • Could it be a Plotly version mismatch for RTD vs. save notebooks?

UPDATE 06/03/24:

• Revisiting Plotly in RTD issues, currently still as above - OK on some old builds, but not recent docs.
• Additional code/script injections tested, per bbb0c7eaf3aeb985baa73dc85c289d38aacb5096, and outputs https://epsproc.readthedocs.io/en/3d-afpad-dev/demos/Epol_class_demo_docs_030324.html
• TODO: test with notebooks run in updated envs, suspect this is code version issue. Note in current pages get various script errors with either old or new script injections, and code also seems to look for local plotly.js script in some cases.
• See also QM3 rendering and tests, https://github.com/phockett/Quantum-Metrology-with-Photoelectrons-Vol3/issues/2 and https://github.com/phockett/Quantum-Metrology-with-Photoelectrons-Vol3/blob/main/doc-source/tests/plotly_surface_render_test.ipynb.
• WORKING for tests built with Plotly 5.15, f344f9760577387f8ada476de461546b87702606
  • Sphinx RTD conf.py includes html_js_files = ['https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.4/require.min.js']
  • RTD build log reports sphinx-5.3.0.
  • Demo page: https://epsproc.readthedocs.io/en/3d-afpad-dev/demos/Epol_class_demo_docs_030324-newBuild.html
  • NOTE manually exported HTML pages (directly from Jupyter) currently failing, adding various additional script manually doesn't currently fix (see conf.py for various options).
  • FOR single notebooks, adding extra notebook lines works (tried this previously?).

    # Plotly export fix
    import plotly.io as pio
    pio.renderers.default='notebook'

  • TODO: wrap this to scripts? Should also be automatic in Jupyter, but doesn't seem to be in current Docker builds, maybe missing plugin?

[phockett]

Quick hack fix for missing facetDims case for Plotly plotters in sphPlot.sphPlotPL() when calling from ep.classes.base.padPlot(). Pretty ugly however, and breaks subplots for 1D case - needs more work.

See 216cfd33edb69a2a2cc99d59aa0c518097724f47

[phockett]

For some further dev and notes see recent MF recon manuscript notebooks, https://github.com/phockett/Extracting-Molecular-Frame-Photoionization-Dynamics-from-Experimental-Data/tree/submission050922#readme

[phockett]

Implemented better HV backend for BLMplot() class version, see 5fcb0277a2df6d356db9198ecbc68e2c48f3ea8c.

This needs some work, but improves/fixes previous implementation.

NOTE issue with (l,m) stacking introducing invalid coords, as per note at https://epsproc.readthedocs.io/en/dev/tests/hvPlotters_fn_tests_150720_v250122.html#BLMplot()-with-HV-backend. May have fixed this elsewhere? Combination of XR restack and Holoviews selection routines? Need to define mask or selectors on HV object?

[phockett]

Holoviews surface plotting

• Planned to implement to wrap Plotly & Matplotlib routines.
• But... doesn't support polar coords (i.e. non-linear (X,Y,Z) maps).
• TriSurface DOES support this (see https://holoviews.org/reference/elements/plotly/TriSurface.html), but doesn't produce good output for Ylm case. (Plotly additional options non-functional, with HV interface, see https://plotly.com/python/trisurf/)

[phockett]

Plotly PAD plotting routine updated with axes options in 1c8ea8c4580808516fac79cf84c208a331f729b4

• Implements basic createAxesPL() routine.
• Calls from sphPlotPL().

TODO: currently not allowing for all args to be passed, and now just plotting axes as default case. Need to implement global styling as note above (https://github.com/phockett/ePSproc/issues/27#issuecomment-1023389063) via dict per plotter or similar.

UPDATE 16/03/23: further improvements to Plotly plotting with facet options. Also allow for basic **kwargs passing from calling functions and class wrapper, as of b3ce53071eccaa21702196e791ff1c8570e8d711

UPDATE 12/07/23: modified fix for conditional only, otherwise may break computational-only function runs, see aaa13d4ac614da8e07feae662d96fa82107edff8.

[phockett]

TODO: improve on last fix above by adding also "returnFlag" option to avoid forced plotting!

[phockett]

Matplotlib versions for MolPlot - currently fails for >v3.5. This sometimes gives cascaded errors in notebooks, and sometimes seems to only affect MolPlot output.

Ah, looks like issue is Arrow3D class, needs patching for Matplotlib >v3.5, see https://stackoverflow.com/a/74122407

TODO: add to ePSproc with conditional patching depending on MPL version.

[phockett]

Multijob .plotGetCroComp() working with Holoviews as of 16de8a353f07f7a704ea0af15f04811702b56435.

TODO: fix for Ehv use too, currently only working for Eke coords. For Ehv can't swap in restacked data, and if set and used pre-stacking end up with weird HVplot results due to NaNs/discontinuities in data. May have fixed this issue elsewhere already?

[phockett]

To fix: Holoviews implementation in self.padPlot (for base class) stacking bug - seems to sometimes fail to stack by label? Issue with Xarray ordering, or just random?