SpeechRecognition events don't trigger

[jkolokotronis]

Describe the Bug SpeechRecognition start/result/end events don't seem to trigger, neither through interface properties, nor through addEventListener api

Steps to Reproduce Our current code, with unnecessary steps removed `

  const SpeechRecog =
    window.SpeechRecognition || window.webkitSpeechRecognition;

  data.recognition = new SpeechRecog();
  data.recognition.continuous = true;
  data.recognition.interimResults = true;

  data.recognition.onstart = () => {
    logger.log("recognition begun", data.recognition!.lang);
    data.recognizing = true;
  };

  data.recognition.onend = () => {
    logger.log("recognition stopped");
    data.recognizing = false;
    nodeCleanup(element);
  };

  data.recognition.onerror = (err: SpeechRecognitionErrorEvent) => {
    logger.warn("recognition error:", err);
    data.recognizing = false;
    nodeCleanup(element);
  };

  data.recognition.onresult = (event: SpeechRecognitionEvent) => {...}

 // somewhere further in code
data.recognition!.start()

` There are no related errors in SEB log (or any mention of SpeechRecognition at all), none of the events ever trigger, onresult is never called, nothing gets recorded, but the system does recognize that start() is being called because if I call it again I get InvalidStateError: recognition has already started.

Tried to rewrite it with addEventListener, got the same issue. Config allowAudioCapture is enabled (and WebRTC audio recording is working fine elsewhere on site), allowDictation enabled.

Recognition works fine in regular Chrome browser, when our site is not loaded through SEB, so there is no obvious logic error. Is there anything else that has to be enabled for SpeechRecognition api to work under SEB?

Expected Behavior recognition.start() should trigger onstart event actual voice capture should trigger onresult event

Version Information

• OS: Windows 11 Home, 22H2, also confirmed with our clients on Win10
• SEB-Version 3.4.1 (x64), Build 3.4.1.505

[dbuechel]

It may very well be that this isn't working (yet) in the browser engine we use (CEF / CefSharp), or that it needs to be activated manually first.

You could try to test the speech recognition with the CEF sample application, that will indicate where the problem lies: https://cef-builds.spotifycdn.com/index.html

[jkolokotronis]

Thanks for the quick response.

Our app doesn't work under their sample client out of the box, but with added -enable-speech-input command flag it starts recording, just to stop it split second later with error code network. so there seems to still be another lingering issue. Their own suggested test (google speech demo) doesn't seem to work better either.

Would it be possible to turn that flag on through SEB config or such?

[dbuechel]

At the moment not, but there is a feature request regarding browser configuration flags (see #551). But it would then appear that there is either way another issue regarding support of speech recognition in CEF itself. You might be able to get some more information in the CEF forum: https://magpcss.org/ceforum/.

[dbuechel]

We'll track the issue regarding browser configuration flags with #551, thus I'll close this issue for now. Please feel free to comment or reopen in case there are new insights available.