scene.capture() does not include labels

[eltos]

The function scene.capture(filename) does not consider labels.

Execute the following code in a jupyter notebook:

from vpython import *
sphere()
label(text='TEXT')
scene.capture('image.png')

Expected output	Actually saved

I noticed that there are two canvas elements in the HTML, one containing the labels, and the other being a placeholder for the 3D objects:

<div class="glowscript-canvas-wrapper ui-resizable" style="display: inline-block; position: relative; float: none;">
    <canvas style="position: absolute;" width="640" height="400"></canvas>
    <canvas style="position: relative; background-color: transparent;" width="640" height="400"></canvas>
    <div class="ui-resizable-handle ui-resizable-e" style="z-index: 90;"></div>
    <div class="ui-resizable-handle ui-resizable-s" style="z-index: 90;"></div>
    <div class="ui-resizable-handle ui-resizable-se ui-icon ui-icon-gripsmall-diagonal-se" style="z-index: 90;"></div>
</div>

It seems that the capturing function here only considers one of them.

[eltos]

BTW it would be useful to have a function actually returning the image (i.e. it's source string from here) in addition to the downloading functionality of capture().

[BruceSherwood]

The issue is that labels are displayed in a transparent 2D canvas in front of the 3D WebGL canvas, and the GlowScript code is only able to capture the 3D canvas. A workaround is to use the 3D text object instead of a label (and note that you can billboard the text object, like a label).

[eltos]

3D text is not an alternative in my opinion, since it scales when zooming, while labels keep the fontsize. Are you closing this as "won't fix"?

A possible solution would be to have a function in glow script returning the rendered image data, and then combining it with the 2D canvas:

var img3D = ... // rendered WebGL image data
var canvas2D = ... // 2D canvas with labels

var c = document.createElement('canvas');
c.width = canvas2D.width;
c.height = canvas2D.height;
c.getContext("2d").putImageData(img3D, 0, 0);
c.getContext("2d").drawImage(canvas2D, 0, 0);
var data = c.toDataURL();
// return or save data

[BruceSherwood]

Thanks for the suggestion. Have you tested your proposed suggestion?

[eltos]

Thanks for the suggestion. Have you tested your proposed suggestion?

@BruceSherwood Yes, I tested this approach successfully. Please find my merge request here: https://github.com/vpython/glowscript/pull/148

[BruceSherwood]

I tried your code with GlowScript VPython 3.2dev, and it hangs for me on this statement:

await new Promise(r => img.onload=r); // wait for image to be loaded

??

[eltos]

@BruceSherwood I tested this code using jupyter notebook 6.3.0 with Firefox 88.0.1 without any issues. I had noticed that the image returned by the renderer's screenshot method was not loaded in time for it to be drawn onto the canvas, hence I added this line of code. Maybe in your environment, it is loaded even before the Promise statement is reached?

[eltos]

You can replace the line with await img.decode();, which is essentially the same (it returns a promise) but should handle either case. I added a second commit to the merge request.