Allow VS Code extensions to trigger audio cues

[therobinkim]

👋

Feature request: I'm creating a VS Code extension, and I wish I could trigger audio cues!

These audio cues could be the ones that already exist.

It'd also be amazing if I could trigger custom audio cues (eg, a 440 Hertz tone for 150 milliseconds).

This would be particularly helpful for me when my VS Code extension fails early out of a command (eg, maybe there are no results to show for my custom search command) and an audio cue gets triggered so that I'm not relying only on hard to see visual cues.

[meganrogge]

cc @isidorn for thoughts

[VSCodeTriageBot]

This feature request is now a candidate for our backlog. The community has 60 days to upvote the issue. If it receives 20 upvotes we will move it to our backlog. If not, we will close it. To learn more about how we handle feature requests, please see our documentation.

Happy Coding!

[isidorn]

Fair feature request, but backlog candidates for now is the right course of action imho. @therobinkim can you provide more details about your extension? How would you use audio cues?

[meganrogge]

a use case of @jooyoungseo #176521

[balajip881]

Hi vscode members,

We should not treat accesssibility issue and normal user issues same. As screen reader users are very less in number compared to normal developer. So please consider this issue and move it to backlog so that team can implement soon.

Please consider this issue even if it does not get 20 upvotes. Thanks.

[meganrogge]

Hi @balajip881 what is your use case/ which extension would you like this to apply to?

[balajip881]

extention name : bookmarks https://github.com/alefragnani/vscode-bookmarks/issues/601

[vadimkholodilo]

This feature would be very useful for the Bookmarks extension discussed above. In Visual Studio this feature is already implemented. Bookmarks speed up code navigation for screen reader users. Especially, in pare programming. Please, move this feature to the backlog

[jooyoungseo]

@meganrogge -- I have encountered the need of this feature again while proposing this feature: https://github.com/iamhyc/Overleaf-Workshop/issues/44

CC @iamhyc

[meganrogge]

@hediet what do you think about an API for this?

[hediet]

Generally, I think an API could make sense, but needs a lot of thoughts. In particular, the API should prevent audio-cue reuse by sound (1) and encourage audio-cue reuse by semantics (2), especially if we ever want to consider contributable sound packs.

(1) e.g. when some extension author likes to use the "error" sound as indicator that some process finished successfully. (2) when a process finished successfully, it should use an audio cue that reflects this semantics.

Maybe an extension could statically declare new audio cues and assign them a default sound (which could be changed by the user or sound-packs). Then, an extension could only play audio cues that it contributed itself.

[davidacm]

Hi. I'm not sure if it's related but my needs are related with accessibility. I'm totally new in vsCode extensions, I started to research today. Having said that, I need to create an extension that produces tones of different frequencies, shape and duration. As a visually impaired developer, I frequently use tones for various cues. But it would be more useful to be able to implement this in an extension instead of coding it in my test files. Although I can implement my own wave generator, the audio output is a problem. So, an API should allow playing audio from files, or from a stream in case the audio waves are created in real time. I'm sure sighted developers could also benefit from something like this, extensions shouldn't be restricted to just visual aspects.

Well, I think I can solve my needs by creating a Webview and communicating with it, but it doesn't seem like an optimal solution to me because I don't need a webview, it shouldn't even be visible!

In a webview I have access to audioContext, I have done tests and can access the oscillators, I have been able to play my own tones without implementing any wave generator, simply using audioContext from VSCode itself.

So why not make a small part of that API available to extensions without so much overhead? I will be happy with an interface to play from file, from stream, and stop the current sound. An event that fires when the sound has stopped would be useful too. Parallel sounds should be allowed. Access to the audioContext oscillators would be even better, but not essential.

Sorry if there is an approach to do what I want without webviews, on the webviews section it's suggested to use alternatives whenever they are available but I haven't found alternatives to play sounds from an extension.

[derekriemer]

I am trying to build an accessibility extension that plays sounds for each level of indent.

• cursor moves to a new line.
• if that line contains a different number of indent from the previous line, play indent tone.
  • 0 indents plays 220 hz.
  • Each indent level (as configured by vscode) beyond that, is 1 half tone higher.
  • Timbre changes based on attributes of the line, I.E. method start, end of an if statement, if statement, for loop, etc.

Vscode's capabilities for playing sounds would be greatly enhanced if this were added. In the perfect world I'd like to be able to contribute signal sounds generated by the web audio api, but I could deal with generating signal sounds from a stream and providing a buffer. Right now, I would either be stuck with low latency options that are not cross platform and don't work on web, or need to use a web view to get web audio.

The web audio api is not generally available for extensions in event handlers, etc. Spinning up web views is expensive, so I'd prefer to have a more efficient way of doing this.

Please lmk what you think HTH in designing further implementation.