Unable to use cognitive services speech in iOS webview

[Dpetters]

I enabled cognitive speech services in our webchat view and it worked when testing in Chrome on desktop but when viewing through the iOS simulator where the same web page is loaded in a webview, I get a TypeError: Super expression must either be null or a function error. I attempted to dig into it a bit but am not sure if the issue is in this package or "web-speech-cognitive-services". Please see attached screenshots. Here is how I create the webSpeechPonyfillFactory method:

await createCognitiveServicesSpeechServicesPonyfillFactory({
        subscriptionKey: this.props.cognitiveServicesKey,
        region: 'westus'
      }).then(webSpeechPonyfillFactory => {
        this.setState(() => ({ webSpeechPonyfillFactory }));
      });

and here is how I pass it to the React component:

<ReactWebChat
            directLine={directLine}
            webSpeechPonyfillFactory={this.state.webSpeechPonyfillFactory}
            styleSet={this.getStyleSet()}
            user={user}
            userID={userId}
            store={store}
          />

To clarify, I do not get this when running on desktop Safari or chrome on a MacBook. Any tips are appreciated. Thank you in advance!

[corinagum]

Can you test the application in the following scenarios?

1. Safari
2. Chrome or Edge on iOS
3. The app you are developing

Because apps on iOS don't have full support of WebRTC, I suspect this problem will occur in Chrome, Edge, and your own app, but won't happen in Safari. Unfortunately this is a limitation of iOS, and Cognitive Services will not work on any app (other than Safari).

Please let us know if you're hitting a different problem. I tested this on my iPhone (v12.0.1) and confirmed that Cognitive Services doesn't work on Chrome, but does work on Safari.

[Dpetters]

Thanks Corina. This issue does not occur on Chrome or Safari on desktop, or Safari on iOS. It does occur on Chrome and Edge on iOS, as well as in our WKWebView in the app.

If you happen to know of any mitigation that would enable us to render the page in some kind of web view on iOS, please do let me know.

[corinagum]

Good to know the root cause.

According to this bug on Chromium, Google is waiting for Apple to add WebRTC to WKWebView.

If you are willing to write native code (Objective C++ or Swift), you can consider this approach:

• Use SFSpeechRecognizer to implement speech-to-text for iOS
• Expose this native component via your bridge to your WKWebView
• On the JavaScript side, write a JavaScript component that talks to your native counterpart
  • The API must comply to W3C Web Speech API
• Pass your API to Web Chat

On the other hand, text-to-speech should work, as Cognitive Services is using Web Audio API.

If you do implement this, please let us know! We would love to include this in our repo.

[stevengum]

Tracking issue for the core issue in WebKit: https://bugs.webkit.org/show_bug.cgi?id=208667 It was indicated that the bug should have been fixed in iOS 13.4 beta 1, but a cross-linked bug indicates that it's still an issue as of iOS 13.4 beta 4.

WebKit Bugzilla 2020/10/19 Search results for "getUserMedia ios 14"

It appears that short of custom native code being written, this issue is still blocked.