Compatibility with Unity VideoRenderStreaming

[felchs]

The actual library appears to not be compatible with the Unity Video Render Streaming because of the protocols and VP8 the format supported by Unity/WEBRtc: https://docs.unity3d.com/Packages/com.unity.renderstreaming@3.1/manual/video-streaming.html

about vp8 decoding: https://docs.unity3d.com/Packages/com.unity.webrtc@2.4/manual/videostreaming.html

Would it be possible to create an example with a connection with Unity SDP and VP8 architecture?

example of protocols:

{ type: 'offer',
  from: 'ac75d7e3-b1ba-4cbe-a48b-7639a3995508',
  data:
   { sdp:
      'v=0\r\no=- 1140300157692296514 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0\r\na=extmap-allow-mixed\r\na=msid-semantic: WMS\r\nm=application 9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:YFRD\r\na=ice-pwd:Lz6Ai8m/WKGy0CzqNr9OxM4I\r\na=ice-options:trickle\r\na=fingerprint:sha-256 14:25:B1:60:C7:99:D6:B6:90:2C:D7:C1:A1:04:22:D8:82:C9:E1:10:E2:4C:57:00:48:D4:5A:3C:9B:EA:40:6F\r\na=setup:actpass\r\na=mid:0\r\na=sctp-port:5000\r\na=max-message-size:262144\r\n',
     connectionId: 'ac75d7e3-b1ba-4cbe-a48b-7639a3995508' } }
{ type: 'candidate',
  from: 'ac75d7e3-b1ba-4cbe-a48b-7639a3995508',
  data:
   { candidate:
      'candidate:1204296370 1 udp 2113937151 3d10699d-4d2f-4c6b-bf2a-8898648e8cb9.local 54800 typ host generation 0 ufrag YFRD network-cost 999',
     sdpMLineIndex: '0',
     sdpMid: 0,
     connectionId: 'ac75d7e3-b1ba-4cbe-a48b-7639a3995508' } }
{ type: 'candidate',
  from: 'ac75d7e3-b1ba-4cbe-a48b-7639a3995508',
  data:
   { candidate:
      'candidate:1947584492 1 udp 2113939711 c3797242-3097-49ab-a676-d8f9daae14a5.local 54901 typ host generation 0 ufrag YFRD network-cost 999',
     sdpMLineIndex: '0',
     sdpMid: 0,
     connectionId: 'ac75d7e3-b1ba-4cbe-a48b-7639a3995508' } }
{ type: 'candidate',
  from: 'ac75d7e3-b1ba-4cbe-a48b-7639a3995508',
  data:
   { candidate:
      'candidate:842163049 1 udp 1677729535 189.36.4.75 54800 typ srflx raddr 0.0.0.0 rport 0 generation 0 ufrag YFRD network-cost 999',
     sdpMLineIndex: '0',
     sdpMid: 0,
     connectionId: 'ac75d7e3-b1ba-4cbe-a48b-7639a3995508' } }
{ type: 'offer',
  from: 'ac75d7e3-b1ba-4cbe-a48b-7639a3995508',
  data:
   { sdp:
      'v=0\r\no=- 1140300157692296514 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0\r\na=extmap-allow-mixed\r\na=msid-semantic: WMS\r\nm=application 54800 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 189.36.4.75\r\na=candidate:1204296370 1 udp 2113937151 3d10699d-4d2f-4c6b-bf2a-8898648e8cb9.local 54800 typ host generation 0 network-cost 999\r\na=candidate:1947584492 1 udp 2113939711 c3797242-3097-49ab-a676-d8f9daae14a5.local 54901 typ host generation 0 network-cost 999\r\na=candidate:842163049 1 udp 1677729535 189.36.4.75 54800 typ srflx raddr 0.0.0.0 rport 0 generation 0 network-cost 999\r\na=ice-ufrag:YFRD\r\na=ice-pwd:Lz6Ai8m/WKGy0CzqNr9OxM4I\r\na=ice-options:trickle\r\na=fingerprint:sha-256 14:25:B1:60:C7:99:D6:B6:90:2C:D7:C1:A1:04:22:D8:82:C9:E1:10:E2:4C:57:00:48:D4:5A:3C:9B:EA:40:6F\r\na=setup:actpass\r\na=mid:0\r\na=sctp-port:5000\r\na=max-message-size:262144\r\n',
     connectionId: 'ac75d7e3-b1ba-4cbe-a48b-7639a3995508' } }
{ type: 'offer',
  from: 'ac75d7e3-b1ba-4cbe-a48b-7639a3995508',
  data:
   { sdp:
      'v=0\r\no=- 1140300157692296514 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0\r\na=extmap-allow-mixed\r\na=msid-semantic: WMS\r\nm=application 54800 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 189.36.4.75\r\na=candidate:1204296370 1 udp 2113937151 3d10699d-4d2f-4c6b-bf2a-8898648e8cb9.local 54800 typ host generation 0 network-cost 999\r\na=candidate:1947584492 1 udp 2113939711 c3797242-3097-49ab-a676-d8f9daae14a5.local 54901 typ host generation 0 network-cost 999\r\na=candidate:842163049 1 udp 1677729535 189.36.4.75 54800 typ srflx raddr 0.0.0.0 rport 0 generation 0 network-cost 999\r\na=ice-ufrag:YFRD\r\na=ice-pwd:Lz6Ai8m/WKGy0CzqNr9OxM4I\r\na=ice-options:trickle\r\na=fingerprint:sha-256 14:25:B1:60:C7:99:D6:B6:90:2C:D7:C1:A1:04:22:D8:82:C9:E1:10:E2:4C:57:00:48:D4:5A:3C:9B:EA:40:6F\r\na=setup:actpass\r\na=mid:0\r\na=sctp-port:5000\r\na=max-message-size:262144\r\n',
     connectionId: 'ac75d7e3-b1ba-4cbe-a48b-7639a3995508' } }

[paullouisageneau]

Like any WebRTC implementation, libdatachannel does not enforce a specific signaling protocol. You have to adapt the signaling messages in your client to be compatible. The SDP descriptions and candidates themselves are compatible so you should just forward them without changes. Note sdpMLineIndex with libdatachannel is always 0.

Regarding the VP8 codec, there are two possibilities:

• If you already have a VP8 RTP stream, you may send RTP packets directly in a track.
• If you only have VP8 raw data, you need to implement your own packetization handler or wait for https://github.com/paullouisageneau/libdatachannel/issues/558.

[felchs]

Ow thank you @paullouisageneau now I complete understood how the structure is.