spatialaudio / nbsphinx

:ledger: Sphinx source parser for Jupyter notebooks
https://nbsphinx.readthedocs.io/
MIT License
457 stars 130 forks source link

Add "alt" to supported image metadata keys #646

Open kandersolar opened 2 years ago

kandersolar commented 2 years ago

Hi @mgeier, this PR allows alt text to be specified in a code cell and end up in the resulting HTML. Alt text is important for accessibility purposes as it is used by screen readers in place of the image itself. It also adds a docs example for width and height (which I think are currently not present in the docs?) as well as alt.

Also, I only looked at HTML output for this as I'm a lot less familiar with the latex side of sphinx output. Not sure it matters, just pointing that out in case it does!

Let me know if anything else is needed here; happy to push updates if so :)

kandersolar commented 2 years ago

Hmm, the doc example is not ideal in that it's calling display() on a figure generated and already displayed in an earlier cell, which probably doesn't match typical usage (only display a figure once). For example a better example might be all this in one cell:

fig, ax = plt.subplots(figsize=[6, 3])
ax.plot([4, 9, 7, 20, 6, 33, 13, 23, 16, 62, 8]);
display(fig, metadata={"image/png": {"width": "150px", "height": "100px",
                                     "alt": "this is my alt text"}})

but this displays the figure twice. Is there some way to get a cell like this to only display the figure once? I'd rather not resort to plt.ioff() for this...

mgeier commented 2 years ago

Well, typical usage would be to not use display() at all, since the plot is displayed automatically.

If you want to avoid the automatic display, you can do this after the plot() call:

plt.close(fig)

But I guess this is even less typical usage.

To add the metadata directly in the same cell as the plot, I guess this would have to be incorporated into the plot() call itself? Like when you use the 'png2x' figure format, which automatically adds the width and height metadata.

kandersolar commented 2 years ago

Thanks @mgeier, maybe I should take a step back and outline my use case. What I want to be able to do is specify figure alt text as unobtrusively as possible in a notebook code cell and use that notebook with nbsphinx in a broader sphinx project to build HTML docs with the alt text included. I see the value in having the alt text be usable elsewhere (e.g. inside jupyterlab itself) and would be happy to work towards that if possible, but I opened this PR only trying to get it working in sphinx's HTML.

To add the metadata directly in the same cell as the plot, I guess this would have to be incorporated into the plot() call itself?

I think this would only work if the matplotlib backends knew what to do with the metadata, right? https://github.com/matplotlib/matplotlib/pull/21328 would add basic aria information to matplotlib's Artist, allowing the alt text and other aria metadata to be set directly on a Figure, but e.g. matplotlib_inline would still need to be changed to pass it through to the output.

Also, is this metadata={mimetype: {...}} structure official in some sense outside of nbsphinx? It seems like the current nbformat schema doesn't know about it: https://github.com/jupyter/nbformat/blob/main/nbformat/v4/nbformat.v4.schema.json#L455-L459

mgeier commented 2 years ago

There seems to be a somewhat related workshop tomorrow: https://mail.python.org/pipermail/matplotlib-users/2022-May/002078.html