Forward Rtp packets

[Codefa]

Thank you for this awesome Webrtc Implementation.

I'm using your SFU server, updated it to use the latest werift, and it works fine, I want to create a MESH network of SFU servers so for that I want to forward/send Rtp data to remote server, with that i can sync video calls ( 1 user from server A & another user from Server B on the same room ). I can succesfully transfer the rtp data to remote server but on another side it feels little out of sync and if the data size get's increase ( stream from more users ) video & audio gets out of sync ( maybe JitterBuffer ? but don't know how to use that in werift ). I send Rtp data's through data-channels, also tried with hyperswarm ( which is DHT service )but other side get's the out of sync data. here is the structure to send rtp data to remote server

const { unSubscribe } = msTrack.onReceiveRtp.subscribe(async (rtp) => {
      sendSFUData(JSON.stringify({
        type: 'rtpData',
        roomName: this.roomName,
        info: this.info,
        quality: this.info.simulcast ? msTrack.rid : 'single',
        packet: rtp.serialize(),
      }))
    })

then on subscribe the media, i do JSON.parse the object then

const newRtp = RtpPacket.deSerialize(Buffer.from(packet))
sender.sender.sendRtp(newRtp)

so basically i serialize & deSerialize the Rtp packets just like in the example rtp_forward. but still the data get's out of sync, if you can give any suggestion that will be great, thank you

[Codefa]

oops I forgot to do receiver.sendRtcpPLI(ssrc) on every 2 seconds just like in your normal SFU server , now i feel like it's almost in sync, but not sure it's the right way, any suggestions on this ?

[shinyoshiaki]

werift's Track does not lip-sync on a per-stream basis.

Please refer to the Recording sample code for an example of lip-sync. https://github.com/shinyoshiaki/werift-webrtc/blob/cc5ebf194db5193095af353ca1531a7083361f61/examples/save_to_disk/pipeline.ts#L48

The lip-sync module used in the source code above is not suitable for your use since it does not interface with RTP. Please use it with appropriate modifications.

[Codefa]

Thank you for your reply. I'm trying to implement mediasoup like pipeTransports between different SFUs ( SFU-->SFU data transfer but on different hosts ). currently I achived it with sending media-info, rtp packets via data-channel this way i keep all the connected (MESH network of SFU servers ) sync in realtime. it works fine even the server have 30mbps(total) internet speed, but when the server bandwidth reduce the video or audio get out-of sync but within some seconds it get in sync again ( just like the user have bad internet) If werift can bring the mediasoup like pipeTransports feature or some examples will be great( maybe something like pion to pion example )