Livekit js sdk

[jinleileiking]

livekit-react

https://github.com/livekit/client-sdk-js 是基础库，livekit-react是基于此库的react component，后续livekit团队进行了使用next.js重构：https://github.com/livekit/components-js， 重构后的项目我无法将console里的log跳转到相应文件，slack上也问了一下，https://livekit-users.slack.com/archives/C01KVTJH6BX/p1677037479199229 他们也无法解决，于是用livekit-react进行流程调试

调试livekit-react过程中，有时需要调试client-sdk-js，方法如下

• clone https://github.com/livekit/client-sdk-js
• cd client-sdk-js
• yarn link 这个命令会把这个库关联到全局, 可以用ls ~/.config/yarn/link来看link情况
• yarn build 后续每改一次库代码，要build一下
• cd livekit-react
• yarn link livekit-client
• yarn start

client sdk里面的log 打不出来文件名 显示installhook.js

原因是装了react chrome插件，停了就好了 😢

sdk 代码分析

graph TD;
  RTCEngine.this.client.onOffer-->this.subscriber.setRemoteDescription-->this.subscriber.createAndSetAnswer-->this.client.sendAnswer

RTCEngine

启动

RoomPage.tsx
  const RoomPage
    return 
       <LiveKitRoom

LiveKitRoom.tsx
  const LiveKitRoom
    useRoom(roomOptions) core:hooks/useRoom.ts
      connect: connectFn
    useEffect
      roomState.connect
        connectFn
          await room.connect(url, token, options);
            const connectFn = async
              `connected to Livekit Server LOG

ingress流推过来，拉流播放

sequenceDiagram

    Server-->>Browser: update 
    Browser->>Browser: [event].room.participantConnected
    Browser->>Browser: [event].room.participantPermissionsChanged
    Server-->>Browser: update
    Server-->>Browser: update 带着音频信息
    Browser->>Browser: [event].participant.trackPublished
    Browser-->>Server: subscriptionPermissionUpdate
    Server-->>Browser: connectionQuality
    Browser->>Browser: [event].participant.connectionQualityChanged
    Server-->>Browser: offer (sdp带着音频信息,o不变)
    Browser->>Browser: subpc.setRemoteDescription
    Browser->>Browser: subscriber.pc.ontrack
    Browser->>Browser: [event].participant.trackSubscribed
    Browser->>Browser: [event].room.trackSubscriptionStatusChanged
    Browser->>Browser: pc.setLocalDescription
    Browser-->>Server: answer
    Browser->>Browser: [event].room.audioPlaybackChanged
rect rgb(191, 223, 255)
    Browser-->>Server: update setting (video)
    Server-->>Browser: streamStateUpdate
    Browser->>Browser: [event].participant.trackStreamStateChanged
    Browser->>Browser: [event].rooom.trackStreamStateChanged
end

ontrack流程

----------
[注册]
private maybeCreateEngine()
  this.engine
    │   │ .on(
    │   │   EngineEvent.MediaTrackAdded,
    │   │   (mediaTrack: MediaStreamTrack, stream: MediaStream, receiver?: RTCRtpReceiver) => {
    │   │   │ this.onTrackAdded(mediaTrack, stream, receiver);
    │   │   },
...
this.subscriber.pc.ontrack
  this.emit(EngineEvent.MediaTrackAdded
...
  private onTrackAdded (room.ts)
    participant.addSubscribedMediaTrack
      publication.setTrack(track);
        this.emit(TrackEvent.Subscribed
        emitSubscriptionUpdateIfChanged
          this.emit(TrackEvent.SubscriptionStatusChanged
...
  this.emit(ParticipantEvent.TrackSubscribed
...
  this.emit(RoomEvent.TrackSubscribed
...
  .on(RoomEvent.TrackSubscribed, onSubscribedTrackChanged)
  onSubscribedTrackChanged [core/src/hooks/useRoom.ts]
    onParticipantsChanged
      setParticipants (States)
...
[state注册]
 AudioRenderer
   track.attach();
    super.attach();
      this.emit(TrackEvent.AudioPlaybackStarted);
...
emitSubscriptionUpdateIfChanged
  TrackEvent.SubscriptionStatusChanged
  this.emit(ParticipantEvent.TrackSubscriptionStatusChanged
    publication.on(TrackEvent.Subscribed)
      this.emit(ParticipantEvent.TrackSubscribed
[视频]
...
VideoRenderer
  useEffect
    track.attach(el); 
      this.observeElementInfo(elementInfo);
        this.updateVisibility();
          this.emit(TrackEvent.VisibilityChanged, isVisible, this);
          ...
            handleVisibilityChange
              this.emitTrackUpdate();   
                this.emit(TrackEvent.UpdateSettings, settings);
          ...
            this.signalClient.sendUpdateTrackSettings(settings);

this.onParticipantUpdate(msg.update.participants ?? []);
    ∏
      this.emitWhenConnected(RoomEvent.ParticipantConnected) --- ParticipantConnected
      participant.updateInfo(info);
        this.setPermissions(info.permission)
          this.emit(ParticipantEvent.ParticipantPermissionsChanged, prevPermissions);
    remoteParticipant.updateInfo(info);
      info.tracks.forEach
        publication.updateInfo
          super.updateInfo
            log.debug('update publication info'

注册：
connectToRoom: async
  room.on(RoomEvent.ParticipantConnected, participantConnected)

接收ws消息 update：
[WS]received update
  见上ws触发消息 ParticipantConnected room event participantConnected  sid: "PA_TKakMka3HExN"  
...
  participantConnected
    participant.on(ParticipantEvent.TrackMuted ... 注册一堆track event回调

第三次update，带audio信息： 
remoteParticipant.updateInfo(info)
   newTracks.forEach((publication) => {
      this.emit(ParticipantEvent.TrackPublished, publication);

第一次sdp

第二次offer的sdp

可以看到，o， ice-ufrag没变，就是给这个加上了音频信息

第三次update，带了audio信息了

断流

调整窗口大小