Implement measureText() function

[asteppke]

First of all thanks for this great Jupyter extension!

For combining text and drawing functions it would be very valuable to know the size of rendered text in the current rendering context. The canvas2d API contains the method measureText which returns these properties. Is it possible to implement this into the ipycanvas widget code so that the backend in python can query this method?

[martinRenou]

Thanks for reporting an issue!

This would actually be difficult to implement, at least to my knowledge... I wonder if we could use the await feature from Python 3.5 in order to implement this. The thing is we need to wait for the JavaScript front-end to compute the text measure and then return the value to the Python back-end. I've got no knowledge of these things yet but that would be awesome if you had some ideas!

[asteppke]

Before we get into the possibilities the idea behind this request is two-fold:

• for any graphical application that uses text which is not monospace we do not know how long a given text string will be on screen or canvas before rendering. So it would be ideal to ask the authority here, that is the browser, for the given font (incl. size, kerning, etc.) and then adapt the next drawing steps around this.
• for any longer text when we want to use line breaks in a given fixed box size all algorithms require as input the length of the current line which again from the python perspective is unknown.

I completely agree with you after looking into this a bit that the implementation is not as straightforward as hoped. From a high level view the asynchronous model is not really suited for this. Maybe that is motivation to look a bit deeper into this issue. At least a few other people ran into a similar issue:

https://github.com/jupyter-widgets/ipywidgets/issues/1783 https://github.com/jupyter/notebook/issues/3187 https://github.com/ipython/ipykernel/issues/369 https://stackoverflow.com/questions/60002189/execute-javascript-synchronously-in-jupyter-notebook-cell

Here in particular one could imagine to circumvent this in a hackish way. Instead of

length = canvas.measure_text("abc def")
draw_box(width = length+padding)

it would be possible to do it in two steps:

guessed_length =  canvas.measure_text("abc def")
draw_box(width = guessed_length+padding)
[...] 
draw_other_parts()
update_box(width = canvas.measure_text_trait)

Another alternative is building a small table character <-> width in px. That does not take kerning into account but should be very close to the real value and a pretty good first guess.

[martinRenou]

Thanks a lot for looking into this!

I like the alternative of creating a table, as it would allow to keep the same API as the JavaScript one. Although it would be difficult to make a complete table for all characters at all fonts and font sizes.

I wonder if using PIL to measure the text size Python side would return the same value the web canvas would return? See https://stackoverflow.com/questions/43828955/measuring-width-of-text-python-pil. Note that pillow is already an ipycanvas dependency.

[martinRenou]

Additional links: https://pillow.readthedocs.io/en/stable/reference/ImageFont.html#imagefont-module https://pillow.readthedocs.io/en/stable/reference/ImageDraw.html#PIL.ImageDraw.PIL.ImageDraw.ImageDraw.textsize

[jtpio]

I wonder if we could use the await feature from Python 3.5 in order to implement this. The thing is we need to wait for the JavaScript front-end to compute the text measure and then return the value to the Python back-end. I've got no knowledge of these things yet but that would be awesome if you had some ideas!

It should be possible to transparently create and return Future objects to the caller that can be awaited. And expose the result as part of the future result.

I was experimenting a bit with this lately in https://github.com/jtpio/ipylab/commit/9c43df704e9a994c5683b9684101c25afbf78683 (not merged yet):

import asyncio

from ipylab import JupyterFrontEnd

app = JupyterFrontEnd()

async def sequence():
    await app.ready()
    for i in range(4):
        result = await app.commands.execute('filebrowser:toggle-main')
        await asyncio.sleep(1)

asyncio.create_task(sequence());

However it's not possible to await at the top-level (blocking), but the workaround is to create a new asyncio.Task.

[jtpio]

It should be possible to transparently create and return Future objects to the caller that can be awaited. And expose the result as part of the future result.

And maybe even hooking into the widgets message ids?

[ricky-lim]

Any update, so far ?

[martinRenou]

Nope, unfortunately. But any help is welcome :)

[jesuisse]

I have a suggestion for a fairly simple workaround.

Use Pillow (which you say is already a dependency) to render text to an image in Python (server-side). Then send the image to the client. Yes, that's terribly inefficient, but it can be implemented easily and it has the additional advantage that the user can pick a specific font and be sure it will be available for rendering. If your render text in the Browser (client-side), that's harder (or impossible?) to do. Plus, there is no back-and-forth communication involved that isn't already happening when we draw images onto a canvas. The downside is that we won't get the stroke/fill/shadow functionality of the canvas for text.

For those of us who need this to be efficient, you could offer an API function that lets us pre-render text (for use in an animation, for example). So we call the currently imaginary ipycanvas.prerender_text(text, font, size), which might take a moment, but will create an image, send it to the client and return a handle for it to the python caller. Then a call to a currently imaginary canvas.show_prerendered_text(handle, x ,y) function will tell javascript to render the image onto the target canvas.

Most of this can be implemented as a python module that is completely separate from ipycanvas itself, but it might be more efficient if it's part of ipycanvas.