Capture audio output to speakers and headphones - not just Tab audio

[guest271314]

Right now only Firefox captures monitor devices on Linux. Chrome/Chrome refuses to capture monitor devices on Linux. Thus there is no Web API that provides a means to capture what you hear.

[ljharb]

If there's no web API then how could WinterCG include it?

[guest271314]

There is Screen Capture API where navigator.mediaDevices.getDisplayMedia() is specified. That specification even uses the term "systemAudio", however that usage is questionable as system audio is not captured. The Tab audio is captured when audio constraint is set.

That means that window.speechSynthesis.speak() will not be captured on Chrome/Chromium browsers.

The rationale is explained here https://github.com/GoogleChrome/developer.chrome.com/pull/3947#issuecomment-1274250421

I thank @guest271314 for his contribution, but I am still against this addition. I think a very small percentage of the readership will be aware of speechSynthesis, and so most of our readers will just be confused by this.

Thereafter I filed https://github.com/GoogleChrome/developer.chrome.com/issues/3957 so users in the field would not think that systemAudio literally means system audio, as it does not.

This is a long-standing issue. The details and context can be found here: https://github.com/guest271314/captureSystemAudio#background.

Web Audio API deferred to Media Capture and Streams API and Screen Capture API https://github.com/WebAudio/web-audio-api/issues/2478 - in the meantime nothing has changed since I filed this https://lists.w3.org/Archives/Public/public-speech-api/2017Jun/0000.html.

Do not hesitate to ask if you have any question.

[ljharb]

How would a non-browser based JS environment utilize these APIs? Most servers are headless and there's no screen or audio to capture.

[guest271314]

Well, WPT performs all sorts of tests with headless browsers. I can stream audio - and capture said audio - in a headless browser https://github.com/guest271314/sw-extension-audio.

[ljharb]

WinterCG encapsulates far more than headless browsers - iow, there's not necessarily a user sitting at the machine.

[guest271314]

Who said I need to be sitting at the machine?

[guest271314]

Look at your mission statement, in pertinent part https://wintercg.org/:

This community group aims to provide a space for JavaScript runtimes to collaborate on API interoperability. We focus on documenting and improving interoperability of web platform APIs across runtimes (especially non-browser ones).

This appeared to be the perfect place to file some long-standing interoperability issues for the folks who know each other and span vendors, specifications, implementors to finally address.

If what you claim on your site is not true and correct I can just keep hacking away creating workarounds for the omissions of standards bodies and implementers.

[ljharb]

"especially non-browser ones" means, to me, audio and video capture doesn't apply. The interoperability is between WinterCG implementations, not between browsers (who, at the moment, do not participate in WinterCG)

[guest271314]

Read your mission statement again. Remove the term web apis if you don't mean it.

[ljharb]

"web APIs" does not imply "all web APIs".

[guest271314]

Sure it does that's what you said on your site in the broadest sense. If you don't mean that then remove that claim from the site.

[guest271314]

"web APIs" does not imply "all web APIs".

Put that language right on your site with a list of the web apis you are talking about and are not talking about. So you can't make arbitrary calls on the fly.

[ljharb]

It's unnecessary as it's already stated. If it were all/every API, that qualifier would be present - since it's not, it's already apparent what it means - a subset.

[guest271314]

It is not clear. You don't spell it out.

And are for some reason not yet capable of hearing feedback about your omissions and failures. The exact opposite of interoperability.

You need to learn how to listen to constructive feedback.