livekit / client-sdk-swift

LiveKit Swift Client SDK. Easily build live audio or video experiences into your mobile app, game or website.
https://livekit.io
Apache License 2.0
174 stars 84 forks source link

Can't publish video track and preview in SwiftUIVideoView doesn't work #330

Open creativecbr opened 5 months ago

creativecbr commented 5 months ago

Describe the Bug

I am encountering an issue with screen share previews using the following code:

SwiftUIVideoView(track, layoutMode: .fit,
                 renderMode: .auto,
                 debugMode: false)

The video tracks are configured in the same manner as in the sample Swift app. I am publishing video tracks gathered by the following method:

let source = MacOSScreenCapturer.sources(for: .display, includeCurrentApplication: true)[0]!

let localVideoTrackForMainDisplay = LocalVideoTrack.createMacOSScreenShareTrack(source: source, options: SessionController.defaultScreenShareCaptureOptions)

guard let capturer = localVideoTrackForMainDisplay.capturer as? MacOSScreenCapturer,
      let mainWindowSource = capturer.captureSource else {

    logger.debug("Couldn't receive display source, error")
    completionHandler?(false)
    return
}

completionHandler?(true)
self.connect(using: configuration, with: mainWindowSource)

The publishing method returns an immediate error: "timed out," and previews do not return any frames from the screen. Notably, it worked correctly on the released version 1.1.6, and there have been no changes to the code for this preview and initialization.

On macOS 14.1, it works correctly from the main branch.

SDK Version

I pulled the SDK version directly from the main branch to incorporate changes from https://github.com/livekit/client-sdk-swift/issues/266, as it has not been released yet. Version 1.1.6 functions correctly.

iOS/macOS Version macOS 12.7.1

Steps to Reproduce

  1. Download the sample macOS app.
  2. Change LiveKit version to gather directly from the main branch.
  3. Adjust the required code to fit into the new LiveKit version.
  4. Launch the macOS sample and attempt to preview or publish the video track.

Expected Behavior The preview should work, and publishing should also function correctly.

Logs I logged information during the restart of tracks:

let sources = try await MacOSScreenCapturer.sources(for: .display, includeCurrentApplication: true)
let options = ScreenShareCaptureOptions(dimensions: .h360_43, fps: 5)//, includeCurrentApplication: true)
let _newTracks = sources.map { LocalVideoTrack.createMacOSScreenShareTrack(source: $0, options: options) }

logger.info("Found \(_newTracks.count) display sources for remote access")

It appears that the tracks are available; the issue may be located further in the code.

creativecbr commented 5 months ago

The problem occurs also on just released 2.0.0.

@hiroshihorie