[Enhancement]: Improve accessibility for screen readers

[ndarilek]

Describe the feature/enhancement

I'm a blind screen reader user and am having a lot of issues trying to use audiobookshelf.

I'd like to make time to address some of these myself one day, but as a blind person myself, it can be a bit hard to figure out exactly what's going on in the interface to fix it. But a lot of it is either adding attributes or syncing ARIA attributes with the interface status itself, so I'll make a checklist here, and if folks can help with some of these then things will get incrementally better. Here are some challenges/fixes:

• [ ] Lots of controls could use aria-label="" as they have unintuitive names (E.g. format_list, file_download, chevron_right.)
• [ ] If something is expanded it should have its aria-expanded attribute set accordingly. It's hard to tell whether menus are expanded/collapsed.
• [ ] Region roles would be very helpful. E.g. role="main" around the main interface area, role="region" aria-label="Continue listening" around specific sections, etc.)
• [ ] aria-hidden="true" might be useful for hidden interface elements, which may or may not be a thing but it's tough to tell.

Again, I'll try to get to some of these but any little bit helps. Definitely enjoying using the app on my home lab and over wireguard when out and about but accessibility is definitely a struggle. I think at least labeling the icons and buttons would be a fairly easy change for folks to make that would help a lot, because right now I'm not always clear what some of these buttons do.

Thanks!

[advplyr]

I agree, this needs to be improved. There is a related issue open for this #441

Some aria labels have been added but not many. We were discussing accessibility in Discord the other day.

[Timtam]

Hi there, I motivated the discussion on Discord back then. I started adding labels, aria-hidden and stuff in some places within the UI already, but far from everywhere as I'm just slowly starting to discover what ABS can do and how it does it. One major point that I want to add to the checklist above is that we most likely need to rewrite, or at least fix, the lists. It would be great if @ndarilek could give this a try as well, but right now filtering or sorting my library is really hard to do accessibly, and sometimes even impossible, depending on the amount of categories/keywords you have etc. I don't know what exactly is broken there, but that would be my top priority right now. Not knowing what a button does is one thing, but at least you can use it if you know what it does, but not being able to use a feature entirely is something else ;).

[ndarilek]

Wow, I didn't know filtering lists was even a thing.

I'll work on adding more labels and regions in the coming weeks. Also, the mobile app is a big priority for me so I want to help fix that too. Does that share any components with the desktop? I can sort of poke around that but it's very difficult to do more than play the latest episodes of a podcast.

[Timtam]

The mobile app certainly looks kinda similar to the actual app for me, although I cannot see the interface myself. I can get audio books playing and everything just fine, its just pretty weird to fiddle around with the UI and it can definitely appreciate some accessibility improvements, I just don't have any experience with development for mobile apart from some Xamarin tests back in 2020 or so ;).

[Timtam]

Another thing that comes to mind, and sorry for the repeatitive posts, is that we need some way to indicate things that right now are shown visually only. Best example would be if a book is abbrigiated or not. The only way we can find out right now is by editing it and checking the checkbox, but the indicator on the book's main view just doesn't get announced by the screen reader. It might be helpful to have some kind of CSS styling for an element that works the other way around to aria-hidden: instead of hiding something from the screen reader, we might require some elements that are detectable for screen readers, but invisible for sighted users who only need to use the visual UI.

[advplyr]

@ndarilek Great, thanks for your help. The mobile app does share a lot of components although they are not 1 to 1, they are customized a bit in places.

[nichwall]

I was going to test out a workflow for updating strings across the server and app repositores. I assume these aria labels will also need translations?

I just wanted to double check before putting all the work in and realizing the workflow didn't handle screen readers. I am not planning to edit the aria labels right now, just want to make sure I take them into account.

[ndarilek]

Yeah, they'll need to be translatable as well. Since presumably your framework is handling the translation, it'd just be writing the translated strings to the supported attributes.

[nichwall]

I was taking some time tonight to look at the aria labels and was curious how the browser works using a screen reader since I have never used a screen reader or aria-labels before. I installed NVDA, and it works well, but it is definitely a very different way to use a computer. I saw in the referenced issue Aria Dev Tools is a good way to see how a page looks to a screen reader, and it has been helpful with my experimentation.

On the topic of adding new labels, are there any things that should be ignored? For example, there are currently aria labels to change the size of cover images on books. In this specific case, the label is still useful for someone who is visually impaired, but is an unnecessary label for someone who is completely blind.

Making those sorts of distinctions would be difficult, but I don't know what would cause "information overload" in finding specific information since after 3 minutes of using a screen reader I had to take my headphones off since I couldn't focus with words cutting out every time I navigated too fast or moved my mouse over another element or paragraph.

[Timtam]

@nichwall Thanks for looking into this and experimenting with a screen reader yourself to get a better grasp of the things that are going on for us SR users specifically. Yep, there are a few things that should be ignored here. The first and most prominent thing would be the thing you already said, the cover image resizer. On almost any ABS page we have the three elements:

• button, increase cover size
• text, defaults to 120, probably the current size or something?
• button, decrease cover size

All of this isn't of much use to us as it stands, people who are still using the screen visually can still use it even with the aria-hidden attribute on those elements, so I think those should just be marked as hidden for screen readers.

Another thing is something that is an annoyance and thus can be fixed, some texts that probably have something to do with styling but still result in text output. An example would be the system settings page. Just read that through real quick with a screen reader and you'll notice a text like "outlined" or something after every setting (I don't know the exact text as i've already labelled them as hidden in my fork). These styling elements can certainly be hidden from screen readers as they don't hold any useful information.

[ndarilek]

Sorry, was away for a bit, but I want to also thank you for using a screen reader to experience how things behave.

I do strongly oppose hiding elements that are otherwise accessible just because they aren't useful to totally blind users for a few reasons:

1. Blindness is a spectrum. Someone may be using a screen reader temporarily, or to supplement existing vision and reduce eyestrain. In that situation, they may benefit from resizing covers so they can more easily see and/or click on their library items.
2. I am totally blind, but have been considering replacing my TV with a media PC and larger monitor. In that case I'd have a screen reader running, but also sighted folks using the same PC from a greater distance. I could imagine someone wanting the covers to be larger when looking at the library from the couch, and it would be annoying if I couldn't resize those images for them because the controls were deliberately hidden.

That said, this isn't a bad idea in general. aria-hidden is probably a good idea on the cover images themselves. My sense, possibly incorrect, is that clicking them either does nothing or behaves identically to clicking the title. In that case, I don't really benefit at all from knowing that there's a cover image as it's essentially 2 controls leading to the same place. If you wanted to make the interface less verbose, I'd hide the images if they don't do anything additional.

Thanks again.

[Timtam]

@ndarilek If they are using a screen reader to enlarge covers so that they can click on them to open the library item, why not just click on the library item via the screen reader in the first place? I can however see that those people might want to enlarge the covers just to properly view them. I don't oppose having the controls visible, thats fine, but I'd love them to be arranged in a different way. Right now the size controls take up three controls and 3 lines that you have to pass through with the arrow keys, that feels a bit clumsy to me. I don't know if its necessary to see the current size of the covers via a screen reader, but i'd like to have it in a more compact way if we want to keep them visible all the time.

[ndarilek]

@ndarilek If they are using a screen reader to enlarge covers so that they can click on them to open the library item, why not just click on the library item via the screen reader in the first place?

🤷 Maybe the cover is a larger hit target and easier to click? If I don't know an answer, I don't like to just take something away. If you hide that option completely, then someone doesn't even have that choice to make and is subject to arbitrary limitations imposed by someone else. As a screen reader user, I don't expect the world to tuck away every single option someone doesn't think I'd need just because I'm using a screen reader. I'd rather those options be accessible to me if only so I know they exist, but often so I can collaborate with non-screen-reader users. Accessibility should add choices, not take them away.

Refactoring the control to be more compact does sound like a good idea, though. How about a slider?

[nichwall]

Currently, clicking on a cover navigates to the book details page, which is the same as clicking on the book title. When the mouse hovers over the cover images, some additional information and buttons are displayed as follows:

• button to Play book
• button to Open edit modal
• button for Drop-down list for additional options like Marking Completed and Discard Progress
• Check box for selecting multiple books
• Ebook format label if applicable

All of these buttons and information can be accessed through the book details page, so the cover could be completely hidden to screen readers to make navigating the library simpler by reducing duplicate information. However, this would prevent you from using the "play" button on the cover and would make editing books in bulk more tedious, but I imagine it is tedious with a screen reader anyway.

Do you think it would be better to hide the cover or just the book title from the screen reader? Or hide neither?

I think changing the size control to a slider would make it more difficult to use for both the screen reader and with a mouse, but maybe the control could appear later? It is displayed in the bottom corner, so making it appear after all of the other information would get it out of the way but still be accessible. I'm not sure how to do that since I don't do much with web development, but I can add labels to things. 🙂

[nichwall]

Oh, and there is also a little progress bar at the bottom of the cover to display the listening progress on the book, but once again that information is also on the book details page.

[ndarilek]

Hmm, maybe the cover could be a combobox?

[nichwall]

I've been taking a look at this again to try and get some things figured out in the web client. I have successfully moved the cover size widget to appear at the end of the page, so that is more out of the way but still accessible.

I am working on figuring out the rows of books on the main page. Right now there are a few rows such as: Continue Listening, Recently Added, and Recent Series. Based on the ARIA dev tools extension, it looks like all of these books are not clickable as of server version 2.7.1. I have not checked previous versions for this behavior. It is also interesting because the entire row is rendered in the screen reader, even if there are more elements in the row than are displayed on the screen. The "chevron left" and "chevron right" buttons are to scroll through the list on the screen and do not seem to do anything from the screen reader perspective since all of the books, series, and authors are visible to the screen reader.

Can one of you who use a screen reader confirm that you cannot navigate to a book page from the main screen by using the title on 2.7.1, or am I misunderstanding what the dev tools are telling me?

[Timtam]

@nichwall Thanks for looking into this. I can confirm that the books are clickable, even though they might not be tagged as such. In fact I just changed the aria role on those in my branch:

• I made the headings (continue listening, recently added and such) a h2 heading, so that we can quickly reach them via heading navigation
• I made the books themselves h3 headings, so that they are headings too, but can directly be addressed by jumping to level 3 headings

Those two changes made the interface much easier to navigate for me already.

Regarding the amount of books displayed: I guess the decision if a book should be visible or not is made via CSS, books that are not on screen are visually hidden, but they still exist and don't get tagged with any hiding indicator, thus they are still visible for screen readers, and I assume they'd be clickable as well. You're right, the next and previous buttons exist and don't work for us. I'd suggest either:

• to make them work by really hiding the books that aren't currently visible, which would make the interface work the same way it does for sighted people. That would probably be recommended I guess, but I personally don't like the idea to have to press previous/next all the time too much.
• or stick with showing all books to screen readers all the time and hide the previous/next buttons, which would distinguish the visual and sr perspectives further, which is most likely not what we want.

HTH a bit.

[nichwall]

I took a look at your fork to see exactly what you meant, but couldn't find the headings.

At one point there was talk about making the shelves scroll infinitely to the right, so more books would be loaded like on the Discovery and Recent shelves, but not sure if that will actually happen and how that would affect the screen reader decisions.

I think hiding the chevrons would be a good option if you can navigate between shelves and then scroll through them like normal with the screen reader anyway. But, that makes it more difficult to navigate in the case of a sighted person/visually impaired since then they couldn't make the UI scroll through the shelf.

Do you need to press "previous/next" to go between books on a shelf, or does it read through all of them automatically?

[linuxdaemon]

Hi folks, what's the current status of this work? I don't see any recent updates related to it and as a visually impaired ABS user myself with a blind partner, I am interested in these improvements.

If the work has not been undertaken, I can take a look at some of it.

[Timtam]

I've got some improvements on my fork already, but its far from finished, basically some aria-labels, aria-levels and roles and such. I'll have to see what I can improve further.