Save threejs graphics using a screenshotting tool (playwright, selenium, ...)

[mkoeppe]

https://github.com/puppeteer/puppeteer, which is used by screenshotting tools such as https://github.com/marketplace/actions/screenshots-ci-action

This will allow us to drop the dependency of the docbuild (html and pdf) on Jmol (#31027) and improve the quality of the output.

CC: @kwankyu @egourgoulhon

Component: graphics

Branch/Commit: u/mkoeppe/save_threejs_graphics_using_puppeteer @ c368e46

Issue created by migration from https://trac.sagemath.org/ticket/33513

[kwankyu]

comment:1

It sounds a silly question, but why saving static images from 3d graphics in the documentation is an issue?

[mkoeppe]

comment:2

The issue is that it currently relies on ancient technology - either Jmol (which needs a Java runtime) or alternatively tachyon.

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,2 +1,3 @@
 https://github.com/puppeteer/puppeteer, which is used by screenshotting tools such as https://github.com/marketplace/actions/screenshots-ci-action

+This will allow us to drop the dependency of the docbuild (html and pdf) on Jmol (#31027) and improve the quality of the output. 

[kwankyu]

comment:4

Replying to @mkoeppe:

The issue is that it currently relies on ancient technology - either Jmol (which needs a Java runtime) or alternatively tachyon.

I meant why we should save static images in the first place. In interactive use of the documentation, threejs provides a menu to save png image.

[kwankyu]

comment:5

I meant when we use jupyter-sphinx, which provides the menu.

[mkoeppe]

comment:6

Of course for the HTML doc I do prefer embedding the notebook (#33507). But for PDF we probably don't have a better option.

[kwankyu]

comment:7

Okay. I see.

[kwankyu]

comment:8

I wrongly presumed that jupyter-sphinx rendering of 3d images would produce png images for pdf docs. Now I checked that it doesn't work that way.

[mkoeppe]

Branch: u/mkoeppe/save_threejs_graphics_using_puppeteer

[mkoeppe]

Commit: c368e46

[mkoeppe]

comment:10

With this branch, it can be installed via make node_puppeteer, then in ./sage -sh, the example hn.js shown in https://github.com/puppeteer/puppeteer works

---

New commits:

c368e46

build/pkgs/node_puppeteer: New

[kwankyu]

comment:11

In #33507, I take a simple approach to provide static images for pdf documentation.

I think that seeking for an automatic way to produce static images from live images would not be worth the efforts. One of the reasons is that taking a good-looking static image needs human intervention anyway.

[mkoeppe]

comment:12

Yes, that's a very good point

[mkoeppe]

comment:13

There's also pyppeteer, mentioned in https://nbconvert.readthedocs.io/en/latest/changelog.html

[gutow]

comment:14

As I helped with the original integration of Jmol, I can tell you that once the decision to move to other 3D renders occurred there has been no maintenance of Jmol in Sage. In theory it could work, but I think making it optional or removing it would be a good idea at this point. There is some work towards making Jmol work easily in a Jupyter environment. If people wish to combine the molecular rendering savvy of Jmol with Sage, I think that will be the way to go. I encourage you to remove the dependence.

[mkoeppe]

comment:15

Replying to @mkoeppe:

There's also pyppeteer, mentioned in https://nbconvert.readthedocs.io/en/latest/changelog.html

Opened #33533, #33534 for this

[kcrisman]

@kiwifb @mkoeppe Despite https://github.com/sagemath/sage/pull/37138#issuecomment-2019103007 it looks like https://github.com/pyppeteer/pyppeteer/commits/dev/ is still pretty active, in fact may have reached version 2.0.0.

[kiwifb]

Well the README says they only do minor changes and they are looking for a maintainer. If it is alive the README needs updating.

[mkoeppe]

I think people these days use Playwright or Selenium.

[kcrisman]

As I helped with the original integration of Jmol, I can tell you that once the decision to move to other 3D renders occurred there has been no maintenance of Jmol in Sage. In theory it could work, but I think making it optional or removing it would be a good idea at this point. There is some work towards making Jmol work easily in a Jupyter environment. If people wish to combine the molecular rendering savvy of Jmol with Sage, I think that will be the way to go. I encourage you to remove the dependence.

Thanks for weighing in (I just saw this). I agree that the Java applet no longer is necessary, once we can create proper graphics for saving automatically.

[kcrisman]

Thanks, @kiwifb - I just looked at the commits.

I think people these days use Playwright or Selenium.

https://github.com/microsoft/playwright-python

[kiwifb]

All that stuff require a browser since it is driving one. Not a big issue if you provide binaries. I had to install browser on headless servers to use selenium before. And of course you cannot use a firefox snap either. It has to be a "real" firefox if that's what you want to use.

[kcrisman]

Just for reference, @rbeezer , this and a few other tickets might lead to a change in how Sage saves png for 3d plots (if we can figure out how to do them).