search

flogy / gatsby-mdx-tts

🗣 Adds speech output to your Gatsby site using Amazon Polly.
https://gatsby-mdx-tts.netlify.com
MIT License
9 stars 2 forks source link

Create speech output for images #18

Open ghost opened 4 years ago

ghost commented 4 years ago

It would be nice to generate speech output for images using the alt attribute. I think this is the relevant AST node: https://github.com/syntax-tree/mdast#image.

I'm not quite sure what consequences that would have for the word marker. Perhaps add a border around the image?

flogy commented 4 years ago

Good idea! I think a border around the image would be nice. Could also be another highlighting effect, but we must pay attention that it does not change the dimensions of the image or it will flicker around.

A problem I stumbled across when playing around with this was that we don't know if the embedded image is just a simple <img> element or a gatsby-image (or even something else). Depending on how images are implemented we will have to use different approaches to manipulate their styles.

Generally, I think it would be nice to somehow abstract the way the word highlighter handles different kinds of elements. Like that we could also provide an option to the plugin users to inject custom handlers for elements we have not yet foreseen. What do you think about this?

ghost commented 4 years ago

You're right, applying the style might be a problem.

A friend who is blind recently showed me her JAWS setup and how she browses the web. JAWS seems to apply a border with a padding to the images but I didn't look at the actual HTML. Since JAWS isn't open source (afaik) perhaps we could look at how NVDA does it?

Making the highlighting customizable seems like a good idea.