livekit / client-sdk-flutter

Flutter Client SDK for LiveKit
https://docs.livekit.io
Apache License 2.0
262 stars 132 forks source link

[bug] Intermittent Issue with Missing Audio Information in SDP Offer on iOS #598

Open kendo6666 opened 1 month ago

kendo6666 commented 1 month ago

There is an intermittent issue we have encountered while using the flutter SDK for audio and video calls on iOS devices.

Issue Summary: We have observed that approximately 10%-20% of the time, the remote participant cannot hear any audio during a call. Upon investigation, we found that the SDP offer generated by the createAndSendOffer method sometimes lacks any audio information. This issue does not occur on Android devices using the same SDK versions.

Details: iOS Devices: iPhone X (iOS 16.4) iPhone 11 (iOS 15.0.2)

Livekit Server Versions: 1.6.0 1.7.0

Livekit SDK Versions: 1.5.3 to 2.2.2 (all versions tested)

Steps to Reproduce:

  1. Initiate a call using the Livekit SDK on an iOS device.
  2. Publish an audio track. Observe the generated SDP offer. Notice that approximately 10%-20% of the time, the SDP offer lacks audio information, resulting in the remote participant not hearing any audio.

Observed Behavior: The SDP offer generated by the createAndSendOffer method sometimes lacks audio information. The remote participant cannot hear any audio during the call.

Thank you for your attention to this matter. We look forward to your prompt response and assistance.

`
      await _room?.connect(url, token,
          connectOptions: connectOptions,
          roomOptions: RoomOptions(
              dynacast: true,
              adaptiveStream: true,
              defaultCameraCaptureOptions: const CameraCaptureOptions(params: VideoParametersPresets.h540_169),
              defaultVideoPublishOptions: VideoPublishOptions(
                  simulcast: !e2eeEnabled,
                  videoCodec: e2eeOptions != null ? 'VP8' : 'VP9',
                  backupVideoCodec: BackupVideoCodec(),
                  videoEncoding: videoEncoding),
              e2eeOptions: e2eeOptions));
      if (!mounted) return;
      _room?.addListener(_onRoomDidUpdate);
      if (null != _listener) _setUpListeners();
      if (null != _room) roomDidUpdateSubject.add(_room!);
      _sortParticipants();
      if (CallState.call == callState || CallState.connecting == callState) {
        widget.onWaitingAccept?.call();
      }
      WidgetsBindingCompatible.instance?.addPostFrameCallback((_) {
        _publish();
      });

  void _publish() async {
    // video will fail when running in ios simulator
    try {
      final enabled = widget.callType == CallType.video;
      await _room?.localParticipant?.setCameraEnabled(enabled);
    } catch (error, stackTrace) {
      Logger.print('could not publish video: $error $stackTrace');
    }
    try {
      await _room?.localParticipant?.setMicrophoneEnabled(enabledMicrophone);
    } catch (error, stackTrace) {
      Logger.print('could not publish audio: $error $stackTrace');
    }
    await Hardware.instance.setPreferSpeakerOutput(false);
    await Hardware.instance.setSpeakerphoneOn(true);
  }
      `
kendo6666 commented 1 month ago
      if (widget.callType == CallType.video) {
        await Future.delayed(const Duration(seconds: 1));
      }
      await _room?.localParticipant?.setMicrophoneEnabled(enabledMicrophone);

After adding 1 second delay the problem was solved.Perhaps it would be correct to publish the audio track first, as fastConnection does.