Does not work on iOS browsers

[dan7geo]

This does not work properly on iOS browsers including Safari and Chrome. The live transcription works well for the first 10-20 seconds on iOS. Then it loses the websocket connection and says "Deepgram is connecting..." After that it shows "Deepgram connection open!" but then stops working. No further captions are produced after this.

You can reproduce this issue by running your Vercel demo on iOS browsers.

Whenever the websocket disconnects, the Close event seems to be triggered correctly because isListening is set to false. It seems a new websocket connection is also correctly established after this because isListening gets changed back to true when the connection Open event triggers.

Not sure what's going wrong here just on iOS because this demo works well on Android, Windows, Mac.

[theotarr]

Are there any updates on this issue?

I'm experiencing a similar problem. The websocket connection remains open across all browsers and platforms, and the microphone appears to be active, continuously sending data to Deepgram. However, Deepgram doesn't respond with any interim results or final transcription; it only sends back metadata. I can't reliably reproduce this, but every now and then reloading the web page, thereby resetting the websocket connection seems to fix this.

This seems to be a different aspect of the problem you're facing, where on iOS browsers like Safari and Chrome, the live transcription works initially but then stops after the websocket connection is lost and reestablished.

I'm also not sure what going wrong here, but maybe there's a fix to reset the websocket connection when the microphone volume is above a certain threshold and there are no results from Deepgram.

[1trackprojects1]

for me it doesnt work on chrome (Version 121.0.6167.160 (Official Build) (arm64)) on macOS. It works fine for a few seconds until it reconnects to deepgram and then it just stops working

[theotarr]

for me it doesnt work on chrome (Version 121.0.6167.160 (Official Build) (arm64)) on macOS. It works fine for a few seconds until it reconnects to deepgram and then it just stops working

By chance do you have React strict mode on? I had that problem but turning it off seemed to fix it.

[hackthehackerman]

+1 this fails frequently on mobile browser. Websocket is active and it is sending blobs to deepgram, but sometimes it just won't receive anything back.

[lukeocodes]

I believe it's to do with the media type, but I've not had time to look into it. We used this as a base for https://github.com/deepgram-devs/deepgram-conversational-demo and we have that working in iOS. If anyone wants to fix it, happy to throw some t-shirts or stickers your way.

[christophmeise]

@lukeocodes

I also suspected the mime type as it's different on mobile browser. Here is the function I use:

const getSupportedMimeType = useMemo(() => {
  if (MediaRecorder.isTypeSupported("audio/wav")) {
    console.log("audio/wav");
    return "audio/wav";
  }
  if (MediaRecorder.isTypeSupported("audio/webm")) {
    console.log("audio/webm");
    return "audio/webm";
  }
  if (MediaRecorder.isTypeSupported("audio/ogg")) {
    console.log("audio/ogg");
    return "audio/ogg";
  }
  // seems to be necessary for ios
  if (MediaRecorder.isTypeSupported("video/mp4")) {
    console.log("video/mp4");
    return "video/mp4";
  }
  console.log("no supported mime type found");
  return "";
}, []);

I debugged with my iPhone on Safari and the result was video/mp4 - no audio type was working.

Can you confirm that video/mp4 works on live streaming? The fun part is that the first few seconds are transcribed just fine so it seems like it's working.

The connection closes without any error though.

[celestk]

Hi all, i've also faced the same issue when building a project on Nuxt.js.

Damien on discord has provided a solution:

When sending Webm Opus audio which is containerized the first packet contains the headers. If you do not send those headers again on reconnection Deepgram does not know how to process the audio I made an example here that stores this first packet of data and resends it on reconnection See: https://github.com/DamienDeepgram/node-live-example/commit/a80e548b646e88251a323c24b74aafe80980382f Note this code intentionally disconnects after 10 sec to simulate reconnection but you likely do not want to do that in your own code.

Maybe someone can give it a go?