[Accessibility] Allow to trigger reading out aloud via keybinding

[jooyoungseo]

Type: Bug

CC @meganrogge

1. Ask any question in Chat.

2. Press "read aloud (alt+ctrl+shift+U)" and see if the keybinding works.

Current Behavior

No response until the read aloud button is manually clicked.

Expected Behavior

Read aloud needs to be triggered via keybinding.

VS Code version: Code - Insiders 1.94.0-insider (dc9412125d4e0a480593962ae2687e74e64af728, 2024-09-09T17:10:54.809Z) OS version: Windows_NT x64 10.0.22631 Modes:

System Info

|Item|Value| |---|---| |CPUs|11th Gen Intel(R) Core(TM) i5-1145G7 @ 2.60GHz (8 x 2611)| |GPU Status|2d_canvas: enabled
canvas_oop_rasterization: enabled_on
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
skia_graphite: disabled_off
video_decode: enabled
video_encode: enabled
vulkan: disabled_off
webgl: enabled
webgl2: enabled
webgpu: enabled
webnn: disabled_off| |Load (avg)|undefined| |Memory (System)|15.71GB (4.45GB free)| |Process Argv|--crash-reporter-id b05b88e5-8894-4031-ae34-fa034ebddea9| |Screen Reader|yes| |VM|0%|

Extensions (125)

Extension|Author (truncated)|Version ---|---|--- vscode-openapi|42C|4.28.1 zotenote|A-W|1.0.1 android-dev-ext|ade|1.4.0 aiprm-lang|AIP|0.0.2 Bookmarks|ale|13.5.0 openscad|Ant|1.2.2 spellright|ban|3.0.136 mermaid-markdown-syntax-highlighting|bpr|1.6.0 external-pdf|cha|1.2.0 doxdocgen|csc|1.4.0 vscode-markdownlint|Dav|0.53.0 vscode-eslint|dba|3.0.10 vscode-quick-select|dba|0.2.9 vscode-deno|den|3.39.0 gitlens|eam|14.6.1 EditorConfig|Edi|0.16.4 prettier-vscode|esb|10.1.0 figma-vscode-extension|fig|0.3.5 vscode-firefox-debug|fir|2.9.10 shell-format|fox|7.2.5 vscode-google-translate|fun|1.4.13 codespaces|Git|1.17.2 copilot|Git|1.229.1095 copilot-chat|Git|0.21.2024090602 remotehub|Git|0.64.0 vscode-github-actions|git|0.26.2 vscode-pull-request-github|Git|0.97.2024090514 cloudcode|goo|2.17.0 overleaf-workshop|iam|0.13.2 cslpreview|igo|0.2.2 path-autocomplete|ion|1.25.0 latex-workshop|Jam|10.3.0 lilypond-syntax|jea|0.1.1 scheme|jea|0.2.0 better-cpp-syntax|jef|1.17.2 commitlint|jos|2.6.0 language-julia|jul|1.122.1 google-search|kam|0.0.1 vscode-lua-format|Koi|1.3.8 lilypond-formatter|lhl|0.2.3 lilypond-pdf-preview|lhl|0.2.8 lilypond-snippets|lhl|0.1.1 vslilypond|lhl|1.7.3 language-matlab|Mat|1.2.5 git-graph|mhu|1.30.0 azure-dev|ms-|0.8.3 vscode-azureappservice|ms-|0.25.3 vscode-azurecontainerapps|ms-|0.6.1 vscode-azurefunctions|ms-|1.15.3 vscode-azureresourcegroups|ms-|0.8.3 vscode-azurestaticwebapps|ms-|0.12.2 vscode-azurestorage|ms-|0.16.1 vscode-azurevirtualmachines|ms-|0.6.5 vscode-cosmosdb|ms-|0.22.0 vscode-docker|ms-|1.29.2 vscode-edge-devtools|ms-|2.1.5 black-formatter|ms-|2024.3.12071014 debugpy|ms-|2024.11.2024082901 flake8|ms-|2023.13.12291011 isort|ms-|2023.13.12321012 python|ms-|2024.15.2024090406 vscode-pylance|ms-|2024.9.1 jupyter|ms-|2024.9.2024090801 jupyter-keymap|ms-|1.1.2 jupyter-renderers|ms-|1.0.19 vscode-jupyter-cell-tags|ms-|0.1.8 vscode-jupyter-slideshow|ms-|0.1.5 remote-containers|ms-|0.385.0 remote-ssh|ms-|0.115.2024090921 remote-ssh-edit|ms-|0.86.0 remote-wsl|ms-|0.81.8 vscode-remote-extensionpack|ms-|0.25.0 azure-account|ms-|0.12.0 azure-repos|ms-|0.40.0 cmake-tools|ms-|1.19.51 cpptools|ms-|1.22.2 cpptools-extension-pack|ms-|1.3.0 js-debug-nightly|ms-|2024.9.517 remote-explorer|ms-|0.5.2024011009 remote-repositories|ms-|0.42.0 remote-server|ms-|1.6.2024011109 vscode-github-issue-notebooks|ms-|0.0.130 vscode-node-azure-pack|ms-|1.2.0 vscode-selfhost-test-provider|ms-|0.3.25 vscode-serial-monitor|ms-|0.12.0 vscode-speech|ms-|0.10.0 vscode-speech-language-pack-en-ca|ms-|0.4.0 vscode-speech-language-pack-en-gb|ms-|0.4.0 vscode-speech-language-pack-ko-kr|ms-|0.4.0 vsliveshare|ms-|1.0.5936 windows-ai-studio|ms-|0.5.2024090301 autodocstring|njp|0.6.1 pandocciter|not|0.10.4 typst-lsp|nva|0.13.0 publisher|pos|1.1.6 shiny|Pos|1.1.0 shinyuieditor|pos|0.5.0 quarto|qua|1.114.0 r-debugger|RDe|0.5.5 java|red|1.34.0 vscode-xml|red|0.27.1 vscode-yaml|red|1.14.0 r|REd|2.8.4 multi-command|ryu|1.6.0 AudioQ|Seh|0.0.2 vscode-deepl|soe|1.1.1 abc-music|sof|0.4.0 lua|sum|3.10.5 latex-utilities|tec|0.4.14 cmake|twx|0.0.17 vscode-terminal-here|Tyr|0.2.4 windows-terminal|Tyr|0.7.0 errorlens|use|3.16.0 intellicode-api-usage-examples|Vis|0.2.8 vscodeintellicode|Vis|1.2.30 vscode-conventional-commits|viv|1.26.0 vscode-arduino|vsc|0.7.1 vscode-gradle|vsc|3.16.4 vscode-java-debug|vsc|0.58.0 vscode-java-dependency|vsc|0.24.0 vscode-java-pack|vsc|0.29.0 vscode-java-test|vsc|0.42.0 vscode-maven|vsc|0.44.0 markdown-all-in-one|yzh|3.6.2 grammarly|znc|0.25.0 (1 theme extensions excluded)

A/B Experiments

``` vsliv368:30146709 vspor879:30202332 vspor708:30202333 vspor363:30204092 vscod805cf:30301675 vsaa593:30376534 py29gd2263:31024238 c4g48928:30535728 a9j8j154:30646983 962ge761:30841072 pythongtdpath:30726887 welcomedialog:30812478 pythonnoceb:30776497 asynctok:30898717 dsvsc014:30777825 dsvsc015:30821418 pythonmypyd1:30859725 h48ei257:31000450 pythontbext0:30879054 accentitlementst:30870582 dsvsc016:30879898 dsvsc017:30880771 dsvsc018:30880772 cppperfnew:30980852 pythonait:30973460 01bff139:31013167 a69g1124:31018687 dvdeprecation:31040973 dwnewjupyter:31046869 impr_priority:31057980 nativerepl1:31134653 refactort:31084545 pythonrstrctxt:31093868 flighttreat:31119334 wkspc-onlycs-t:31132770 nativeloc1:31118317 wkspc-ranged-c:31125598 cf971741:31111988 jh802675:31132134 e80f6927:31120813 autoexpandse:31133494 ei213698:31121563 ```

[meganrogge]

I can reproduce this not working, though the keyboard shortcuts troubleshooting show the command is, in fact, invoked.

[meganrogge]

response is undefined here

[bpasero]

I would think this is a regression from refactorings that happened and maybe a bug in how the response is resolved when invoked from a command.

[bpasero]

@jooyoungseo @meganrogge I wonder, was this ever working? Do we define alt+ctrl+shift+U anywhere? I am asking because if triggered via keybinding, there is no context being passed in and probably never was 🤔

[jooyoungseo]

@bpasero Oh, my bad! Yes, it was added to my custom keybinding. However, why is this not working?

  {
    "key": "alt+ctrl+shift+u",
    "command": "workbench.action.chat.readChatResponseAloud"
  }

[meganrogge]

I don't think it was ever working @bpasero

[bpasero]

Not everything that is a command in VS Code can just work with keybindings without further work to know the context. Treating as feature request.

[bpasero]

https://github.com/microsoft/vscode/pull/228314 implements this with the following logic:

• the keybinding will always go at the last focused chat
• if a specific response is focused, that one will be read out aloud
• otherwise the last response will be read out aloud

Is there a standard expected keybinding that people would assume here to work? We could in theory assign a keybinding then, maybe only when the speech extension is installed.

[meganrogge]

I wonder if we should have a setting for whether the chat response is automatically read aloud. https://github.com/microsoft/vscode/issues/228151

If we have that, then maybe we don't need a default keybinding here?

[meganrogge]

If so, I'd be happy to add this - and will make sure when that's enabled, we don't do an aria alert.

[bpasero]

@meganrogge we do have accessibility.voice.autoSynthesize, doesn't that cover this?

[meganrogge]

That is only when speech was used as input

[meganrogge]

If you don't think non-screen reader users will want to have chat responses read out loud automatically when they use non-voice input, then I am fine with tweaking that setting description to indicate that for screen reader users, it will apply regardless of voice or text input.

[bpasero]

Yeah we can tweak this and adopt as needed for screen reader users, happy to review a PR there 🙏

[vs-code-engineering[bot]]

This bug has been fixed in the latest release of VS Code Insiders!

@jooyoungseo, you can help us out by commenting /verified if things are now working as expected.

If things still don't seem right, please ensure you're on version 46c3cc46a159f9140637ef86de34b27144ee09e3 of Insiders (today's or later - you can use Help: About in the command palette to check), and leave a comment letting us know what isn't working as expected.

Happy Coding!