sagemath / sage

Main repository of SageMath. Now open for Issues and Pull Requests.
https://www.sagemath.org
Other
1.22k stars 424 forks source link

Make threejs / three.js the default 3d viewer instead of jmol #22408

Closed egourgoulhon closed 4 years ago

egourgoulhon commented 7 years ago

Now that #22389 (Make Three.js work offline) and #22261 (Three.js: Default lighting) have been merged, it is time to make threejs the default 3d viewer, in replacement of jmol.

A discussion about this started in #22389. In addition, one may note the following:

pros:

cons:

CC: @paulmasson @novoselt @sagetrac-tmonteil @kcrisman @slel @williamstein @embray @infinity0 @kiwifb @egourgoulhon @strogdon @timokau @jplab

Component: graphics

Keywords: 3D, plot3d, threejs, three.js, jmol

Author: Paul Masson, Eric Gourgoulhon

Branch: fcac479

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/22408

53959995-fd20-47f5-85e6-5e769b863d1f commented 7 years ago

Branch: u/paulmasson/22408

53959995-fd20-47f5-85e6-5e769b863d1f commented 7 years ago

Commit: 93b0b0e

53959995-fd20-47f5-85e6-5e769b863d1f commented 7 years ago
comment:2

For future reference, this simple alteration is all that is needed to change the default, but no rush about it. After #22389 the viewer does not work locally in notebooks: must fix!


New commits:

93b0b0eChange default
novoselt commented 7 years ago
comment:3

What do you mean it does not work locally anymore??? It did for me...

As for the default change, I move that this happens in 8.0 together with the change in default notebook (perhaps add this as a dependence?). Users will be so shocked, that they won't even notice ;-)

egourgoulhon commented 7 years ago
comment:4

Replying to @novoselt:

As for the default change, I move that this happens in 8.0 together with the change in default notebook (perhaps add this as a dependence?).

+1

egourgoulhon commented 7 years ago
comment:5

Replying to @novoselt:

What do you mean it does not work locally anymore??? It did for me...

It does not for me in Sage 7.6.beta5, as reported in https://groups.google.com/d/msg/sage-release/jftcaPjaniM/KMiw3ZOXDAAJ

Paul, is it the issue that you mentioned in comment:2 ?

53959995-fd20-47f5-85e6-5e769b863d1f commented 7 years ago
comment:6

Replying to @egourgoulhon:

Replying to @novoselt:

What do you mean it does not work locally anymore??? It did for me...

It does not for me in Sage 7.6.beta5, as reported in https://groups.google.com/d/msg/sage-release/jftcaPjaniM/KMiw3ZOXDAAJ

Paul, is it the issue that you mentioned in comment:2 ?

Yup. I tested #22389 at the command line and neglected to test it in the notebooks. When a browser window is opened from the command line, it has access to the local file system to find the JavaScript files. The notebooks by contrast are sitting on a local web server that does not have the same access to the file system, so the JavaScript cannot be loaded.

The fix is to use the current display manager to determine if the rendered file will be viewed from a local web server or not and modify the script tags accordingly. Then I need to figure out how to add the local Three.js as notebook extensions as was done for JSmol.

egourgoulhon commented 7 years ago
comment:7

Replying to @paulmasson:

Yup. I tested #22389 at the command line and neglected to test it in the notebooks. When a browser window is opened from the command line, it has access to the local file system to find the JavaScript files. The notebooks by contrast are sitting on a local web server that does not have the same access to the file system, so the JavaScript cannot be loaded.

The fix is to use the current display manager to determine if the rendered file will be viewed from a local web server or not and modify the script tags accordingly. Then I need to figure out how to add the local Three.js as notebook extensions as was done for JSmol.

Thanks for these explanations.

c4796b8b-df47-4c78-a415-d934347608f6 commented 7 years ago
comment:8

After applying this patch I get many errors in the dochtml build:

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/matplotlib/sphinxext/plot_directive.py", line 520, in run_code
    six.exec_(code, ns)
  File "/usr/lib/python2.7/dist-packages/six.py", line 699, in exec_
    exec("""exec _code_ in _globs_, _locs_""")
  File "<string>", line 1, in <module>
  File "<string>", line 3, in <module>
  File "<string>", line 8, in sphinx_plot
  File "sage/plot/plot3d/base.pyx", line 1631, in sage.plot.plot3d.base.Graphics3d.save (/build/sagemath-7JFEEo/sagemath-7.5.1/sage/src/build/cythonized/sage/plot/plot3d/base.c:21954)
    self.save_image(filename, **kwds)
  File "sage/plot/plot3d/base.pyx", line 1560, in sage.plot.plot3d.base.Graphics3d.save_image (/build/sagemath-7JFEEo/sagemath-7.5.1/sage/src/build/cythonized/sage/plot/plot3d/base.c:21425)
    self._save_image_png(filename, **kwds)
  File "sage/plot/plot3d/base.pyx", line 1527, in sage.plot.plot3d.base.Graphics3d._save_image_png (/build/sagemath-7JFEEo/sagemath-7.5.1/sage/src/build/cythonized/sage/plot/plot3d/base.c:21074)
    raise ValueError('cannot use viewer={0} to render image'.format(viewer))
ValueError: cannot use viewer=threejs to render image
/build/sagemath-7JFEEo/sagemath-7.5.1/sage/src/doc/en/reference/manifolds/sage/manifolds/chart.rst:1987: WARNING: Exception occurred in plotting chart-11
 from /build/sagemath-7JFEEo/sagemath-7.5.1/sage/src/doc/en/reference/manifolds/sage/manifolds/chart.rst:

Looking at the source code, it does seem indeed that Graphics3d._save_image_png does not support threejs output. Either the docbuild needs to use jmol by default, or threejs could be made to save png output.

(In Debian we explicitly fail the build if the dochtml.log contains "Error". We have to do this because the builder doesn't fail by itself. At this point I'm not sure if this is a Sage bug that also needs to be filed separately, because we patch the Sage dochtml build quite heavily to avoid segfaults and OOM, and I'm unsure if this interferes with any fail-on-error semantics that should be in there.)

egourgoulhon commented 7 years ago
comment:9

Replying to @infinity0:

Looking at the source code, it does seem indeed that Graphics3d._save_image_png does not support threejs output.

Indeed, this is listed in the 'cons' section of the ticket description.

strogdon commented 7 years ago
comment:10

Perhaps a corner case and perhaps not important but if I from a local machine (A) ssh (ssh -Y) to remote machine (B) where Sage is installed and then try to display graphics stuff (show(sphere(), viewer='threejs') on the display for local machine (A); say from sagenb, then the threejs viewer fails. In the JS console I see

THREE.WebGLRenderer 80
THREE.WebGLRenderer: Error creating WebGL context.

and nothing is displayed. Nothing is similarly displayed from the command line or jupyter notebook. The jmol viewer does work. Of course, one could always install Sage on the local machine (A) and get things to work.

egourgoulhon commented 6 years ago
comment:11

Paul, may I ask about any possible update on this, in particular regarding the cons listed in the ticket description? It would be so nice to have Three.js as the default viewer instead of Jmol!

egourgoulhon commented 6 years ago

Description changed:

--- 
+++ 
@@ -9,7 +9,10 @@

 **cons**:

-- some features are still missing in three.js, like mesh plots or converting the view to a png image for printing  
+- some features are still missing in three.js, like 
+  - converting the view to a png image for printing  
+  - mesh plots 
+  - vector field plots 
53959995-fd20-47f5-85e6-5e769b863d1f commented 6 years ago
comment:13

Eric, my main interest in adding Three.js to SageMath was so that I could see it in action in SageMathCell, which for a time I used on my website. There was unfortunately a rather unpleasant conversation earlier this year, the upshot of which was that I was only welcome to use SageMathCell if I did so precisely like everyone else. Not my style.

My response was to write a small library to replicate SageMath interacts in pure JavaScript for embedding in web pages, which is what I now use exclusively on my website. The only other part of SageMath I really used was mpmath for its special functions, so I am developing a second library to provide special functions in pure JavaScript as I need them.

Since I am no longer using SageMath, I have little inclination to work any further on the Three.js backend. That is not to say I never will again, but it is not a priority for me at this time. And if I choose to contribute further I would focus on features useful to the browser experience, not those oriented towards printing, so Three.js is unlikely to replace Jmol if there is no one else interested in developing those features.

kcrisman commented 6 years ago
comment:14

Thanks for this reply, it is very informative about the current status.

Eric, my main interest in adding Three.js to SageMath was so that I could see it in action in SageMathCell, which for a time I used on my website. There was unfortunately a rather unpleasant conversation earlier this year, the upshot of which was that I was only welcome to use SageMathCell if I did so precisely like everyone else. Not my style.

For context, the issue was that running a bunch of cells automatically uses nontrivial amounts of computer resources, which costs money.

My response was to write a small library to replicate SageMath interacts in pure JavaScript for embedding in web pages, which is what I now use exclusively on my website.

That is a great idea - is it completely new code, or based on the Sagecell, or something else? If you don't need the heavy lifting of everything in Sage, this probably does make sense for you, and hopefully others can use it as well.

The only other part of SageMath I really used was mpmath for its special functions, so I am developing a second library to provide special functions in pure JavaScript as I need them.

Is there no other math library in JS? Seems like a lot of work to do correctly.

so Three.js is unlikely to replace Jmol if there is no one else interested in developing those features.

Thanks for the update, that is very helpful to assess the feasibility of this. Maybe it might end up as a summer student project sometime; if so, would you be willing to provide some feedback?

egourgoulhon commented 6 years ago
comment:15

Replying to @paulmasson:

Thanks your detailed answer.

Since I am no longer using SageMath, I have little inclination to work any further on the Three.js backend. That is not to say I never will again, but it is not a priority for me at this time.

I really hope you'll be back in the near future. IMHO, you did such a great job in introducing Three.js in Sage and it would a pity if we remain with such a low quality (or at least unfitted for mathematical graphics) tool as Jmol.

And if I choose to contribute further I would focus on features useful to the browser experience, not those oriented towards printing, so Three.js is unlikely to replace Jmol if there is no one else interested in developing those features.

Yes I understand this. Printing output is a minor feature as compared with interactive display in the browser, but it seems required in at least two cases:

53959995-fd20-47f5-85e6-5e769b863d1f commented 6 years ago
comment:16

Replying to @kcrisman:

My response was to write a small library to replicate SageMath interacts in pure JavaScript for embedding in web pages, which is what I now use exclusively on my website.

That is a great idea - is it completely new code, or based on the Sagecell, or something else? If you don't need the heavy lifting of everything in Sage, this probably does make sense for you, and hopefully others can use it as well.

It is completely my own new open-source code, including an improved version of the Three.js template I wrote for here. It does require some knowledge of JavaScript to use, so it is perhaps not as simple as embedding a SageMathCell. It is really intended to replace the defunct Mathemtatica CDF plugin. Anyone who can write a Mathematica Manipulate will understand how to use my code. And because it works entirely in the browser, there are no server-side delivery lags or hosting issues.

The only other part of SageMath I really used was mpmath for its special functions, so I am developing a second library to provide special functions in pure JavaScript as I need them.

Is there no other math library in JS? Seems like a lot of work to do correctly.

There is one library with complex-valued special functions called Jmat, but it has an explicit caveat as to the accuracy of the code. It is also undocumented and not actively developed. The only other library I know that comes close is a transpilation of the GNU scientific library, but that does not support complex numbers and transpiling produces bloated code. One would expect Math.js to include special functions, but it only has the gamma and error functions, which are the simplest to implement.

edd8e884-f507-429a-b577-5d554626c0fe commented 6 years ago
comment:17

Note that giac (which is a standard package of Sage), was compiled into javascript with emscripten compiler, there might be interesting features to use from there.

egourgoulhon commented 6 years ago
comment:18

See this ask.sagemath question posted today for some example of limitation of Jmol:

https://ask.sagemath.org/question/39903/right-click-options-not-showing-up-for-jsmol/

c4796b8b-df47-4c78-a415-d934347608f6 commented 6 years ago
comment:19

We don't have emscripten in Debian because it uses a fork of LLVM, so until this situation is resolved we wouldn't be able to take advantage of a giac-emscripten viewer. Similarly JSMol isn't available in Debian because its JS tooling can't be packaged.

Neither giac nor jmol have very good code anyways, so I'd recommend against using them independently of the above packaging issues.

kiwifb commented 6 years ago
comment:20

I have made a binary package for jmol/jsmol in sage-on-gentoo. I am fed up with jmol and just want it to go, to be honest.

slel commented 6 years ago
comment:21

There is a corresponding CoCalc issue:

slel commented 6 years ago

Changed keywords from none to 3D, plot3d, threejs, three.js

slel commented 6 years ago

Description changed:

--- 
+++ 
@@ -1,4 +1,5 @@
-After #22389 *(Make Three.js work offline)* and #22261 *(Three.js: Default lighting)* have been merged, it may be time to make three.js the default 3d viewer, in replacement of jmol.
+After #22389 *(Make Three.js work offline)* and #22261 *(Three.js: Default lighting)*
+have been merged, it is time to make three.js the default 3d viewer, in replacement of jmol.

 A discussion about this started in #22389. In addition, one may note the following:

@@ -6,6 +7,7 @@

 - three.js has already capabilities beyond what jmol can offer, notably the axes labels, which are absent in jmol
 - some bugs (crashes) in jmol rendering within the Jupyter notebook are not present with three.js (see e.g. [this report](https://groups.google.com/d/msg/sage-support/KvNuD2DfRz0/VM1BPPDPCAAJ))
+- Jmol is a nightmare for packagers (for Debian, Windows, ...)

 **cons**:

@@ -15,4 +17,3 @@
   - vector field plots 

-
slel commented 6 years ago

Description changed:

--- 
+++ 
@@ -1,13 +1,13 @@
-After #22389 *(Make Three.js work offline)* and #22261 *(Three.js: Default lighting)*
-have been merged, it is time to make three.js the default 3d viewer, in replacement of jmol.
+Now that #22389 *(Make Three.js work offline)* and #22261 *(Three.js: Default lighting)*
+have been merged, it is time to make threejs the default 3d viewer, in replacement of jmol.

 A discussion about this started in #22389. In addition, one may note the following:

 **pros**: 

-- three.js has already capabilities beyond what jmol can offer, notably the axes labels, which are absent in jmol
-- some bugs (crashes) in jmol rendering within the Jupyter notebook are not present with three.js (see e.g. [this report](https://groups.google.com/d/msg/sage-support/KvNuD2DfRz0/VM1BPPDPCAAJ))
-- Jmol is a nightmare for packagers (for Debian, Windows, ...)
+- three.js offers functionality absent in Jmol, notably axis labels
+- three.js works better with Jupyter than Jmol, see e.g. [this report](https://groups.google.com/d/msg/sage-support/KvNuD2DfRz0/VM1BPPDPCAAJ))
+- Jmol is a nightmare for packagers (for Debian, Windows, ...), see e.g. [this discussion on debian-science-sagemath](https://alioth-lists.debian.net/pipermail/debian-science-sagemath/2018-July/001228.html)

 **cons**:

@@ -16,4 +16,3 @@
   - mesh plots 
   - vector field plots 

-
slel commented 6 years ago

Changed keywords from 3D, plot3d, threejs, three.js to 3D, plot3d, threejs, three.js, jmol

videlec commented 5 years ago
comment:26

update milestone 8.3 -> 8.4

embray commented 5 years ago
comment:27

Since the OP of this ticket appears to have flamed out I'll take this over. It appears there are a few other code updates necessary to use threejs in a few areas.

embray commented 5 years ago
comment:28

For 3D graphics in the static documentation I don't know why we don't just use something like matplotlib. It outputs very nice 3D graphics these days. I can write the necessary adapters...

egourgoulhon commented 5 years ago
comment:29

Replying to @embray:

Since the OP of this ticket appears to have flamed out I'll take this over.

This is very good! Many thanks!

egourgoulhon commented 5 years ago
comment:30

Replying to @embray:

For 3D graphics in the static documentation I don't know why we don't just use something like matplotlib. It outputs very nice 3D graphics these days.

Indeed.

I can write the necessary adapters...

This would be nice! Moreover, why not using them not only for the documentation, but for any static display (as we already have with tachyon), by offering a new option: viewer='matplotlib' ?

Besides, one may note that 3D plots in the documentation do not have to be static: an example of (interactive) three.js rendering is at http://doc.sagemath.org/html/en/reference/plot3d/threejs.html So, as soon as threejs becomes the default, we could use those (except probably for pdf documentation).

williamstein commented 5 years ago
comment:31

Replying to @egourgoulhon:

Replying to @embray:

Since the OP of this ticket appears to have flamed out I'll take this over.

This is very good! Many thanks!

I might be able to help a little (I'll keep an eye on this), since I wrote a similar threejs renderer for Sage worksheets, so I have some three.js experience. Also, users of Jupyter notebooks in CoCalc frequently complain when they use Jmol, due to the slow load time or lack of Java support in their browser, so switching to threejs in Jupyter would make CoCalcers happier.

edd8e884-f507-429a-b577-5d554626c0fe commented 5 years ago
comment:32

Replying to @williamstein: [...]

Also, users of Jupyter notebooks in CoCalc frequently complain when they use Jmol, due to the slow load time or lack of Java support in their browser, so switching to threejs in Jupyter would make CoCalcers happier.

At least with Sage's Jupyter notebook they only need javascript support, because jSmol is used.

edd8e884-f507-429a-b577-5d554626c0fe commented 5 years ago
comment:33

Imho, the switch to threejs still requires quite a lot of work to render objects properly, i tried with polytopes, see:

d = dodecahedron()
d.show(viewer='threejs')

The polytope looks rounded.

edd8e884-f507-429a-b577-5d554626c0fe commented 5 years ago

Description changed:

--- 
+++ 
@@ -15,4 +15,9 @@
   - converting the view to a png image for printing  
   - mesh plots 
   - vector field plots 
+  - faces of polytopes look rounded, see

+```
+d = dodecahedron()
+d.show(viewer='threejs')
+```
embray commented 5 years ago

Attachment: 2018-08-29.png

embray commented 5 years ago
comment:35

That might be browser or system related or something. It looks fine to me: I tried it on Chrome, Firefox, and Edge--all fine.

williamstein commented 5 years ago
comment:36

some features are still missing in three.js

For what it is worth, I think the version in Sage worksheets in CoCalc has all of these things that are missing... https://github.com/sagemathinc/cocalc/blob/master/src/smc-webapp/sagews/3d.coffee

To try it, open a cocalc Sage worksheet, then show 3d plots, but don't do (viewer='threejs'), since that will actually break it in Sage worksheets. E.g., just do

dodecahedron(mesh=2, color='green', opacity=.7)

When the mouse moves outside any plot it is replaced by a static png image. This is absolutely CRITICAL since otherwise if there are more than about 10 plots on the page, things break in subtle ways (at least that was the case when I developed this). If Jupyter Sage doesn't do this, then that's a problem.

To see that printing works, click any of the print buttons in the top right. It even should work with print to pdf, which grabs those png's mentioned above, sends them to the backend project, and embeds them in the latex document...

williamstein commented 5 years ago
comment:37

Also possibly relevant: when we were writing this 3d rendering for cocalc, at some point Andrew Ohana (a student of mine at the time) put a fork of the code on github as a standalone project: https://github.com/ohanar/math3d.js

egourgoulhon commented 5 years ago
comment:38

Attachment: dodecahedron_threejs.png

Replying to @embray:

That might be browser or system related or something. It looks fine to me: I tried it on Chrome, Firefox, and Edge--all fine.

On my side, I confirm the issue reported in comment:33: This is with Sage 8.4.beta2 + Firefox running on Ubuntu 18.04.1.

embray commented 5 years ago
comment:40

Then maybe this is a bug with WebGL on Linux, or at least a difference in defaults w.r.t. shading and norms interpolation. Do you know if your Firefox is using ANGLE? There's a setting under about:config for webgl.disable-angle.

embray commented 5 years ago
comment:41

Scratch that, I'm getting the same thing now too. Previously I was running Sage 8.3. So actually maybe something changed between Sage versions.

Update: Okay, really weird. I recently uninstalled and reinstalled Sage 8.3 on Windows (since I was testing a new version of the installer) and now my 8.3 has the same problem, whereas previously it did not. How strange.

I wish I still had my previous example open but it appears not.

embray commented 5 years ago
comment:42

Adding shading: THREE.FlatShading to the polyhedron's material fixes it. I am still mystified as to why this worked for me earlier and then didn't.

dimpase commented 5 years ago
comment:43

Replying to @embray:

Adding shading: THREE.FlatShading to the polyhedron's material fixes it. I am still mystified as to why this worked for me earlier and then didn't.

Given a moving target called "modern web browser", this is not surprising.

embray commented 5 years ago
comment:44

It's entirely possible there was something messed up with my browser's GPU interface that for a moment it wasn't computing vertex normals, which would result in exactly this sort of flat shading??

Anyways, it's an easy patch to make. I've been meaning to get around to it soon. Once that's fixed is there any other reason not to make threejs the default 3D viewer?

egourgoulhon commented 5 years ago
comment:45

Replying to @embray:

Anyways, it's an easy patch to make. I've been meaning to get around to it soon.

Very good!

Once that's fixed is there any other reason not to make threejs the default 3D viewer?

See the first three items in the "cons" list in the ticket description.

egourgoulhon commented 5 years ago
comment:46

Replying to @egourgoulhon:

See the first three items in the "cons" list in the ticket description.

The first of them means that

sage: g = sphere()
sage: g.save("sphere.png")

produces a png file depicting the Jmol initial view, while we have at the moment

sage: g = sphere(viewer='threejs')
sage: g  # OK: opens a threejs window in the browser
Launched html viewer for Graphics3d Object
sage: g.save("sphere.png")
...
ValueError: cannot use viewer=threejs to render image
embray commented 5 years ago
comment:48

Retargeting some of my tickets.

tobihan commented 5 years ago
comment:49

In Debian we don't have jsmol so attempting 3D plotting in the notebooks just gives a message saying that one should use viewer='threejs'. Setting the default 3D viewer to threejs leads to a failing docbuild because of the missing png capability of threejs. Is there some workaround we could apply to fix 3D plotting in the notebooks in Debian? For example to use jmol on the command line and threejs in the notebooks by default?

egourgoulhon commented 5 years ago

Description changed:

--- 
+++ 
@@ -7,14 +7,14 @@

 - three.js offers functionality absent in Jmol, notably axis labels
 - three.js works better with Jupyter than Jmol, see e.g. [this report](https://groups.google.com/d/msg/sage-support/KvNuD2DfRz0/VM1BPPDPCAAJ))
-- Jmol is a nightmare for packagers (for Debian, Windows, ...), see e.g. [this discussion on debian-science-sagemath](https://alioth-lists.debian.net/pipermail/debian-science-sagemath/2018-July/001228.html)
+- Jmol is a nightmare for packagers (for Debian, Windows, ...), see e.g. [this discussion on debian-science-sagemath](https://alioth-lists.debian.net/pipermail/debian-science-sagemath/2018-July/001228.html); in particular jsmol, which is invoked in Jupyter notebooks, is not packaged for Debian (see [comment:49](#comment%3A49))

 **cons**:

 - some features are still missing in three.js, like 
   - converting the view to a png image for printing  
   - mesh plots 
-  - vector field plots 
+  - vector field plots (via the functions `plot_vector_field` and `plot_vector_field3d` only; three.js works fine with the [plot() method of vector fields on manifolds](http://doc.sagemath.org/html/en/reference/manifolds/sage/manifolds/differentiable/vectorfield.html#sage.manifolds.differentiable.vectorfield.VectorField.plot), the latter being based on individual arrow plots)
   - faces of polytopes look rounded, see