search

libretro / RetroArch

Cross-platform, sophisticated frontend for the libretro API. Licensed GPLv3.
http://www.libretro.com
GNU General Public License v3.0
10.31k stars 1.83k forks source link

Narrator feature has very high latency on Windows #9855

Open pitermach opened 4 years ago

pitermach commented 4 years ago

Description

The menu narration feature takes an extremely long time to speak on Windows as compared to Mac OS.

Expected behavior

When accessibility is turned on and you move the highlight from option to option, it should be spoken almost immediately, which is the case when using Retroarch on Mac OS (tested on a 2014 Mac Mini with a Core I5 processor and Mac OS Catalina)

Actual behavior

On Windows, there is a rather large delay. When you press a key to move the cursor from one option to another, speech is stopped immediately which is great, but it takes over a second for the new menu item to be spoken. On my computer (Core I7-6700, Geforce960 with Windows and Retroarch running off of an SSD) the delay is about a second. On a friends computer running an AMD A8-4555M, this delay is closer to 3 seconds. Both of these systems are running Windows 10. This delay happens both when using a keyboard to navigate the menus as well as a USB gamepad (Logitech F510 in xInput mode). Other things I tried to see if they would improve the situation include lowering the audio latency, changing the audio driver from xAudio to wasapi and SDL, and the video driver from GL to D3d 12. Interface sounds and game audio do not have this delay. When using the Windows voices in other applications (NVDA and Narrator) they have almost no delay, so the issue seems to be how Retroarch is accessing them.

Steps to reproduce the bug

  1. In RetroArch, enable accessibility, either from commandline or the menus.
  2. Use the keyboard or gamepad to navigate the menus and observe how long it takes for the speech to keep up.

Version/Commit

Environment information

BarryJRowe commented 4 years ago

This is a bit tricky since the delay is not present on all windows installs (my windows 8 runs it fine). However, we can try using a dll instead of the power shell way. This is the relevant doc: https://docs.microsoft.com/en-us/uwp/api/windows.media.speechsynthesis.speechsynthesizer

Phoenix1991-lab commented 4 years ago

How am I supposed to use a dll for the narrator in Windows 10 for retroarch? Thank you!

pitermach commented 4 years ago

@Phoenix1991-lab you can't, this isn't something that's been implemented yet. @BarryJRowe That sounds like something worth trying. If you're considering using a dll, perhaps it might also be worth considering using Tolk

The advantage of this aproach rather than just using SAPI directly is that tolk sends the spoken text to the scrreen reading software that someone is using, or to SAPI if it can't find one. This is better because the speech will come out with whatever settings a user has configured in the screen reader to fit their personal preference. It's also worth noting that the current SAPI implementation in RetroArch also seems to completely ignore what voice is set to default in Windows and always uses Microsoft David which makes other interface languages not sound right.

As a sidenote, Tolk also offers a function to check if any screen reading software is currently running. This could be useful to automatically turn on the menu narration if someone starts Retroarch while also running a screen reader, it would greatly simplify first-time setup for a blind user as there would be no need to memorise the button sequence to navigate the menus or read up on the command line.

blindndangerous commented 4 years ago

I tried to post this through email, but I guess it didn't go through. Would this be of any help? https://github.com/dkager/tolk

Phoenix1991-lab commented 4 years ago

how am I supposed to use tolk with retroarch? The instructions do not sound clear to me. I read through the readme and downloaded the directory. What do I need to do next?

On 12/22/19, blindndangerous notifications@github.com wrote:

I tried to post this through email, but I guess it didn't go through. Would this be of any help? https://github.com/dkager/tolk

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/libretro/RetroArch/issues/9855#issuecomment-568306397

pitermach commented 4 years ago

@Phoenix1991-lab Again, this is not something you can do yet because it's not implemented. @blindndangerous and myself are nearly throwing potential suggestions around to solve the unresponsive speech on Windows. The only time you'd be able to use any of them is when the Retroarch team implements one of them.