Make a three.js backend for 3d plotting

--- 
+++ 
@@ -1,3 +1,5 @@
+SEE https://github.com/mrdoob/three.js/
+
 See the big discussion at http://groups.google.com/group/sage-devel/browse_thread/thread/d3e4a24aebc906a2

 Some resources:

kcrisman commented 11 years ago

comment:4

Jason, William, can you put whatever code exists on here since public worksheets are disabled? We don't want this just in the cloud.sagemath :-)

williamstein commented 11 years ago

comment:5

Attachment: three.js.sws.gz

https://github.com/williamstein/sd48/blob/master/threejs/grout-demo.sagews

kcrisman commented 11 years ago

comment:6

Just as an update, this is now apparently working in cloud.sagemath.

williamstein commented 11 years ago

comment:7

Replying to @kcrisman:

Just as an update, this is now apparently working in cloud.sagemath.

It sort of works, but it isn't included by default. It will be soon though, so that anybody can at least test out some minimal functionality. I'll post when that happens.

jasongrout commented 10 years ago

comment:12

Just FYI, there are now three different threejs backends: SMC, the threejs command in the sage cell, and the pythreejs-based backend that works in the sage cell too.

novoselt commented 10 years ago

comment:13

Jason - what is the status of the last one? Will it be pushed to get "standard" or there is still work to be done there and with your switch nobody is available to steer it?

The first two were definitely lacking features, in particular "things that don't scale with zoom" plus some transparency issues.

jasongrout commented 10 years ago

comment:14

The transparency issues are not fixed---those are inherent in how three.js draws things, and stem from limitations of webgl (there are experiements overcoming them in webgl, which might be interesting to look at, though; the author of mathbox is looking at them, for example).

I think that anything "pushed to standard" should be supported by SMC and the sage cell. Right now, the pythreejs solution requires an communication framework that is/emulates the ipython comm/widget framework. Something like that would need to be implemented for SMC to get it to work there. I think that would be a great project in itself, though---there is starting to be some good development of interactive things for the IPython notebook, and it would be great to use those things from SMC too.

kcrisman commented 10 years ago

comment:15

I think that anything "pushed to standard" should be supported by SMC and the sage cell. Right now, the pythreejs solution requires an communication framework that is/emulates the ipython comm/widget framework. Something like that would need to be implemented for SMC to get it to work there. I think that would be a great project in itself, though---there is starting to be some good development of interactive things for the IPython notebook, and it would be great to use those things from SMC too.

But I thought SMC only uses the js plotting - does it also do Jmols too? (You'll notice I still haven't had time to properly try out SMC, and won't for some time).

See also #16004 as a related ticket I also haven't had enough time to review properly.

61a548cc-63ba-4b67-a2d4-d33f3740a21a commented 10 years ago

comment:16

Replying to @jasongrout:

Just FYI, there are now three different threejs backends: SMC, the threejs command in the sage cell, and the pythreejs-based backend that works in the sage cell too.

Thanks :-) I am interested in enhancing the capabilities of SageManifolds (#14865), which is still under development, so I guess the most natural thing would be to somehow use threejs locally. Probably this question was asked already many times, but what is the current status of threejs in sage notebook (local ipython notebook)?

kcrisman commented 10 years ago

comment:17

Thanks :-) I am interested in enhancing the capabilities of SageManifolds (#14865), which is still under development, so I guess the most natural thing would be to somehow use threejs locally. Probably this question was asked already many times, but what is the current status of threejs in sage notebook (local ipython notebook)?

The Sagenb is not really at all an Ipython notebook, though it may have inspired earlier versions thereof. Anyway, the current status is that you can get it to work - see the examples above - but no one has put time into this. Partly because the sagenb is in maintenance mode currently (though it would be nice to have it more actively developed) but also because of the slight backwards functionality loss involved as you see in previous comments.

53959995-fd20-47f5-85e6-5e769b863d1f commented 7 years ago

Branch: u/paulmasson/12402

53959995-fd20-47f5-85e6-5e769b863d1f commented 7 years ago

Description changed:

--- 
+++ 
@@ -1,3 +1,7 @@
+Usage: `threejs( plot3d )`
+
+---
+
 SEE https://github.com/mrdoob/three.js/

 See the big discussion at http://groups.google.com/group/sage-devel/browse_thread/thread/d3e4a24aebc906a2

53959995-fd20-47f5-85e6-5e769b863d1f commented 7 years ago

Commit: 7cedf75

53959995-fd20-47f5-85e6-5e769b863d1f commented 7 years ago

comment:23

Here's some basic Three.js support for Graphics3d objects. It is modeled on the corresponding function in SageMathCell: simply use threejs() to wrap any combination of points, lines and surfaces. This will save an HTML file with the JavaScript needed to render the scene and open it in a browser.

There are a great many features that can be added in the future, but I first would like to see how much interest there is in this. There have been problems surfacing with Jmol (#20978 for example) and I certainly think it would be preferable to have a native viewer that can be updated more quickly. The HTML format is also more portable than Jmol files, since it only requires a modern browser to run.

This version relies on an external CDN to retrieve the Three.js library, since it does not currently download to the hard drive during the build process. I'll need help on figuring out how to change that at some point, since I'm sure people will want to be able to view saved files offline.

Last 10 new commits:

`3d1cd2a`	`Handle nested lists`
`efb3d0a`	`Handle nested lists`
`f70ac3d`	`Add line plots`
`55f12e6`	`Add point plots`
`b4c8414`	`Adjust centering and camera`
`374809a`	`Rebase to move point bounds off points`
`9a42352`	`Add auto z-aspect`
`d560683`	`Minor rebase`
`aa322ad`	`Add axis labels`
`7cedf75`	`Check for equal bounds`

53959995-fd20-47f5-85e6-5e769b863d1f commented 7 years ago

Author: Paul Masson

egourgoulhon commented 7 years ago

comment:25

Wow! I've just tried it and it's very nice! Thank you very much for this enhancement!

A question: is it possible to customize the axes labels (e.g. via the keyword argument axes_labels, as in 2D plots)?

kcrisman commented 7 years ago

comment:26

A question: is it possible to customize the axes labels (e.g. via the keyword argument axes_labels, as in 2D plots)?

See e.g. here - this would be a very good enhancement, though I guess it would have to then depend on the backend.

Can you think of anything it does not do? I seem to recall that there were things jsmol did with shading etc. that three.js couldn't, when this was discussed ~ 5 years ago.

egourgoulhon commented 7 years ago

comment:27

Replying to @kcrisman:

A question: is it possible to customize the axes labels (e.g. via the keyword argument axes_labels, as in 2D plots)?

See e.g. here

Yes, set_axes_labels is the (not very satisfactory) workaround I wrote to deal with this when using Jmol ;-)

this would be a very good enhancement, though I guess it would have to then depend on the backend.

Maybe not: looking at the source code, in line 155 of src/sage/plot/plot3d/threejs.py, there is

addLabel( 'x=' + ( xMid ).toFixed(d).toString(), xMid, b[1].y+offset, a[2]*b[0].z );

It seems easy to replace 'x=' by another string.

Can you think of anything it does not do?

Not at the moment. I'll play with it further.

egourgoulhon commented 7 years ago

comment:28

PS: Another relevant project is K3D, which was presented at Sage Days 74:

https://wiki.sagemath.org/K3D-tools

53959995-fd20-47f5-85e6-5e769b863d1f commented 7 years ago

comment:29

Replying to @egourgoulhon:

PS: Another relevant project is K3D, which was presented at Sage Days 74:

https://wiki.sagemath.org/K3D-tools

It's good to know about this project, but I'm aiming for something more general and not focused just on notebooks. Matplotlib lets you create and save image files for 2d plots. I'd like something equivalent for 3d plots, which for Three.js is a self-contained HTML file.

53959995-fd20-47f5-85e6-5e769b863d1f commented 7 years ago

comment:30

Replying to @egourgoulhon:

Replying to @kcrisman:

this would be a very good enhancement, though I guess it would have to then depend on the backend.

Maybe not: looking at the source code, in line 155 of src/sage/plot/plot3d/threejs.py, there is
addLabel( 'x=' + ( xMid ).toFixed(d).toString(), xMid, b[1].y+offset, a[2]*b[0].z );
It seems easy to replace 'x=' by another string.

Since this is done in a browswer, and they all understand Unicode now, there won't be issues with missing fonts like in matplotlib. This is an easy customization.

53959995-fd20-47f5-85e6-5e769b863d1f commented 7 years ago

comment:31

Replying to @kcrisman:

Can you think of anything it does not do? I seem to recall that there were things jsmol did with shading etc. that three.js couldn't, when this was discussed ~ 5 years ago.

I would find it difficult to believe that current Three.js can't match anything in Jmol. The transparency issue mentioned above can likely be handled with the alphaMap property that appeared in Three.js about the same time as the comment, and if not that way then by a custom shader which isn't difficult to implement.

53959995-fd20-47f5-85e6-5e769b863d1f commented 7 years ago

comment:32

Replying to @egourgoulhon:

Wow! I've just tried it and it's very nice! Thank you very much for this enhancement!

A question: is it possible to customize the axes labels (e.g. via the keyword argument axes_labels, as in 2D plots)?

Thanks for trying it out! I'm trying to decide between reading the show() options of the plots fed into threejs() or just have a separate dictionary, but should probably do the latter. That would include options for setting axes labels, setting digits after the decimal in labels, turning off axes labels altogether, turning off the bounding box and setting an explicit aspect ratio in all directions. Let me know what other options would be interesting.

There is already automatic scaling in the z-direction to avoid the tall thin box that appears in the current version on the cell server (and SMC), but that can easily be overridden. I can also easily add face colors to support surfaces with color maps. Lights for the scene are meant to be customizable as well and can stand some additional improvement.

The initial camera position is not yet completely adaptive to the possible variety of input plots, so ideas on that would be welcome.

novoselt commented 7 years ago

comment:34

We most definitely want this in, so please keep working!!!

Note that we have optional threejs package (SageMathCell is using it) - if we have a functioning viewer we can easily make is standard as it is small and very fast to install.

Interface-wise let's stick to show for the actual showing of stuff and plot/plot3d for producing plots that can be further manipulated. threejs command if it exists at all should do something like set up a low-level threejs scene that can be tweaked as advanced users please. Presumably, it is possible to just pass some HTML/JavaScript code that will be inserted into generated files?

novoselt commented 7 years ago

comment:35

File-wise: perhaps instead of raw string it is possible to have template_three.js file that will be read/cached by the Python function? Perhaps with template_three.js.html to further wrap it into a standalone page. One of the advantages for development - syntax highlighting of this long template.

novoselt commented 7 years ago

comment:36

So it seems that at the moment opacity is ignored and line thickness is not taken into account - is it possible to implement these?

Also when I scale the image text grows/shrinks and then "becomes normal again" - is it possible to always keep it the same size throughout the transition?

novoselt commented 7 years ago

comment:37

What about embedding the resulting plots into web-page, i.e. how easy it is for someone-not-really-knowing-anything to take a plot from Sage and have it running on their web-page? (No using SageMathCell, just having the three.js stuff properly saved/inserted somewhere.)

53959995-fd20-47f5-85e6-5e769b863d1f commented 7 years ago

comment:38

Replying to @novoselt:

What about embedding the resulting plots into web-page, i.e. how easy it is for someone-not-really-knowing-anything to take a plot from Sage and have it running on their web-page? (No using SageMathCell, just having the three.js stuff properly saved/inserted somewhere.)

The entire aim is to have a self-contained HTML file with everything, all geometric data and the JavaScript to process it. Then it's simply a matter of adding an iframe to a web page and setting its src attribute to the HTML file. Easy peasy!

53959995-fd20-47f5-85e6-5e769b863d1f commented 7 years ago

comment:39

Replying to @novoselt:

So it seems that at the moment opacity is ignored and line thickness is not taken into account - is it possible to implement these?

Opacity should be simple. Line thickness can be included, but will not be consistent across operating systems because of a limitation in Microsoft's implementation of WebGL.

Also when I scale the image text grows/shrinks and then "becomes normal again" - is it possible to always keep it the same size throughout the transition?

Thought I had this figured out. For a plot with the origin at the center it behaves as you expect, but not when the center is displaced. Will definitely work on this.

novoselt commented 7 years ago

comment:40

Replying to @paulmasson:

Line thickness can be included, but will not be consistent across operating systems because of a limitation in Microsoft's implementation of WebGL.

I consider this to be a major flaw, no matter whose fault it is - on the one hand it is nice to use different thicknesses and on the other - single pixel width typically looks bad when used with projectors... Are there any sensible workarounds? Like text seems to be scaled and then scaled back - can it be done with lines too?

53959995-fd20-47f5-85e6-5e769b863d1f commented 7 years ago

comment:41

Replying to @novoselt:

We most definitely want this in, so please keep working!!! Interface-wise let's stick to show for the actual showing of stuff and plot/plot3d for producing plots that can be further manipulated. threejs command if it exists at all should do something like set up a low-level threejs scene that can be tweaked as advanced users please.

A scene in Sage is a collection of Graphics3d objects. Three.js, like Jmol/JSmol, is merely a way to render the scene. I don't see the use for command to set up a separate Three.js scene from scratch.

Until this Three.js viewer is integrated into the Rich Output backend, I don't have a way to display the scene without a command other than show, which will default to Jmol/JSmol. That can change later once an initial version is working acceptably. I've spent enough time reading the Rich Output backend to know where to make those changes, and at that point show can then be set to default to a Three.js viewer if desired. For now I need the separate command!

Presumably, it is possible to just pass some HTML/JavaScript code that will be inserted into generated files?

See the next comment for this.

novoselt commented 7 years ago

comment:42

No problem with having a separate command now, but what I meant is - let's not introduce and then deprecate it in Sage, so using rich output should be done on this ticket as well.

53959995-fd20-47f5-85e6-5e769b863d1f commented 7 years ago

comment:43

Replying to @novoselt:

File-wise: perhaps instead of raw string it is possible to have template_three.js file that will be read/cached by the Python function? Perhaps with template_three.js.html to further wrap it into a standalone page. One of the advantages for development - syntax highlighting of this long template.

I can only think of two ways to communicate data from Python to JavaScript at the command line: either with the Python-formatted string I'm using, or by appending data to a URL as a query string and processing it from there. This second way leads to monstrous URLs that can be bookmarked but not saved as stand-alone files. With what I'm doing all of the data is embedded in the HTML file: saving the file saves the entire scene.

On the server you have other options for processing data, but running something like that at the command line seems to me to be massive overkill. If I'm missing something in your comment, please let me know a better way to transfer the data to the browser.

Also, the raw string is not proper JavaScript because all curly brackets are doubled to become literals during Python formatting. Saving that as a separate file would produce parsing errors.

novoselt commented 7 years ago

comment:44

Sorry I was not clear again: what I meant is take your current template, save it as js file in the same Sage source folder, then in Python code open this file, read it as string, and do the same processing as what you do now. So it will not affect the end result for the user, just, IMHO, makes it easier to maintain this template inside of Sage.

53959995-fd20-47f5-85e6-5e769b863d1f commented 7 years ago

comment:45

Replying to @novoselt:

Replying to @paulmasson:

Line thickness can be included, but will not be consistent across operating systems because of a limitation in Microsoft's implementation of WebGL.

I consider this to be a major flaw, no matter whose fault it is - on the one hand it is nice to use different thicknesses and on the other - single pixel width typically looks bad when used with projectors... Are there any sensible workarounds? Like text seems to be scaled and then scaled back - can it be done with lines too?

Text is done on an HTML canvas, so that's completely different from WebGL lines. There are workarounds, but I'd have to think about it more and then would process lines the same way for all browsers.

53959995-fd20-47f5-85e6-5e769b863d1f commented 7 years ago

comment:46

Replying to @novoselt:

Sorry I was not clear again: what I meant is take your current template, save it as js file in the same Sage source folder, then in Python code open this file, read it as string, and do the same processing as what you do now. So it will not affect the end result for the user, just, IMHO, makes it easier to maintain this template inside of Sage.

When the Python part of this file is integrated with the Rich Output backend, then all that will be left in this file will be the template, so sure. But again, this is not proper JavaScript because of the curly bracket doubling.

novoselt commented 7 years ago

comment:47

Ah, and that's because you want to use .format(..) with this string. An alternative is to use %s substitution if it makes sense or just regular string replacement, but if you prefer doubling so be it since you are doing the actual work ;-)

kcrisman commented 7 years ago

comment:48

Just wanted to say this sounds like a great discussion, keep it up! It would be great to have something reliable. Also you may want to check out how this looks/works in the Jupyter notebook as well as sagenb.

kcrisman commented 7 years ago

comment:49

(I assume it will be fine, just mentioning.)

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 7 years ago

Changed commit from 7cedf75 to f4b1c03

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 7 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

f4b1c03 Move template to separate file

53959995-fd20-47f5-85e6-5e769b863d1f commented 7 years ago

comment:52

Replying to @novoselt:

Ah, and that's because you want to use .format(..) with this string. An alternative is to use %s substitution if it makes sense or just regular string replacement, but if you prefer doubling so be it since you are doing the actual work ;-)

Should have thought of string replacement! Template now separate. Much preferable to have proper JavaScript in it.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 7 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

`df51bde`	`Update initial camera position`
`4f7075d`	`Fix label scaling`

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 7 years ago

Changed commit from f4b1c03 to 4f7075d

53959995-fd20-47f5-85e6-5e769b863d1f commented 7 years ago

comment:54

Replying to @paulmasson:

Replying to @novoselt:

Also when I scale the image text grows/shrinks and then "becomes normal again" - is it possible to always keep it the same size throughout the transition?

Thought I had this figured out. For a plot with the origin at the center it behaves as you expect, but not when the center is displaced. Will definitely work on this.

This should be fixed now.

egourgoulhon commented 7 years ago

comment:55

Replying to @paulmasson:

That would include options for setting axes labels, setting digits after the decimal in labels, turning off axes labels altogether, turning off the bounding box and setting an explicit aspect ratio in all directions. Let me know what other options would be interesting.

It would be nice to have some (customizable) ticks along the bounding box, but this is probably somewhat tricky and should be the topic of another ticket.

Besides, what about text3d?

At the moment, when running threejs from the Jupyter notebook, the 3d plot is rendered in another tab of the browser. Could it be embedded in the notebook instead? (maybe this is related to the Rich Output issue).

Thanks again for your work on this!

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 7 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

62e7a8b Integrate with IPython command line

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 7 years ago

Changed commit from 4f7075d to 62e7a8b

53959995-fd20-47f5-85e6-5e769b863d1f commented 7 years ago

comment:57

Basic integration with the backend for the IPython command line has been added, and the usage has changed accordingly. This was important to understand how options are passed through the backend. Integration with notebooks will be added after basic display options are in place.

sagemath / sage

Make a three.js backend for 3d plotting #12402