search

kclyu / rpi-webrtc-streamer

This repo's objective is providing something like Web Cam server on the most popular Raspberry PI hardware. By integrating [WebRTC](https://webrtc.org/native-code/) and Raspberry PI, we can stream the Raspberry camera feed to browser or native client which talks WebRTC.
Other
626 stars 108 forks source link

SessionDescription compatibility with flutterWebRtc #97

Closed ropha closed 4 years ago

ropha commented 4 years ago

A lot thanks to your great working.

I'm doing a project about low latency '4G fpv picam',i used a proxy program connect to rws with tcp direct,then send all msg from rws to a public network server,then server send to phone(flutter app),but i get an error when the flutter webrtc app setRemoteSessionDescription(using sdp from rws),the sdp created by flutter webrtc is look different to rws's ,but i don't know how to fix it,pls help!

here is the error:

[ERROR:flutter/lib/ui/ui_dart_state.cc(157)] Unhandled Exception: Unable to RTCPeerConnection::setRemoteDescription: Failed to set remote offer sdp: Failed to set remote video description send parameters.

is1

and here is the sdp look like from two platform:

flutter webrtc app sdp:

v=0 o=- 201384015615429976 2 IN IP4 127.0.0.1 s=- t=0 0 a=group:BUNDLE video a=msid-semantic: WMS m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 127 c=IN IP4 0.0.0.0 a=rtcp:9 IN IP4 0.0.0.0 a=ice-ufrag:IDFJ a=ice-pwd:toqiwQMS4y/jJk1ZsKFF6sMd a=ice-options:trickle renomination a=fingerprint:sha-256 1E:BE:BC:5B:DB:A7:AB:DC:F9:62:DA:37:0C:81:9A:23:63:70:13:A4:44:8A:F4:1D:54:8A:11:02:C1:9C:56:EA a=setup:actpass a=mid:video a=extmap:14 urn:ietf:params:rtp-hdrext:toffset a=extmap:13 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time a=extmap:3 urn:3gpp:video-orientation a=extmap:2 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01 a=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing a=extmap:8 http://tools.ietf.org/html/draft-ietf-avtext-framemarking-07 a=extmap:9 http://www.webrtc.org/experiments/rtp-hdrext/color-space a=recvonly a=rtcp-mux a=rtcp-rsize a=rtpmap:96 VP8/90000 a=rtcp-fb:96 goog-remb a=rtcp-fb:96 transport-cc a=rtcp-fb:96 ccm fir a=rtcp-fb:96 nack a=rtcp-fb:96 nack pli a=rtpmap:97 rtx/90000 a=fmtp:97 apt=96 a=rtpmap:98 VP9/90000 a=rtcp-fb:98 goog-remb a=rtcp-fb:98 transport-cc a=rtcp-fb:98 ccm fir a=rtcp-fb:98 nack a=rtcp-fb:98 nack pli a=rtpmap:99 rtx/90000 a=fmtp:99 apt=98 a=rtpmap:100 red/90000 a=rtpmap:101 rtx/90000 a=fmtp:101 apt=100 a=rtpmap:127 ulpfec/90000

rws sdp('offer')

v=0 o=- 3145301544693357634 2 IN IP4 127.0.0.1 s=- t=0 0 a=group:BUNDLE 0 a=msid-semantic: WMS stream_id m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 127 125 104 124 106 c=IN IP4 0.0.0.0 a=rtcp:9 IN IP4 0.0.0.0 a=ice-ufrag:ImsQ a=ice-pwd:XOS7Mw1EiuuvqZexDLz/uoIh a=ice-options:trickle a=fingerprint:sha-256 D0:8F:D1:9B:6B:21:3F:80:BD:77:01:34:A8:85:55:7B:C8:DD:AD:D5:0C:DF:37:4E:B6:D8:E7:ED:F8:EC:0A:1B a=setup:actpass a=mid:0 a=extmap:14 urn:ietf:params:rtp-hdrext:toffset a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time a=extmap:13 urn:3gpp:video-orientation a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01 a=extmap:12 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay a=extmap:11 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing a=extmap:8 http://tools.ietf.org/html/draft-ietf-avtext-framemarking-07 a=extmap:9 http://www.webrtc.org/experiments/rtp-hdrext/color-space a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id a=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id a=sendrecv a=msid:stream_id video_label a=rtcp-mux a=rtcp-rsize a=rtpmap:96 H264/90000 a=rtcp-fb:96 goog-remb a=rtcp-fb:96 transport-cc a=rtcp-fb:96 ccm fir a=rtcp-fb:96 nack a=rtcp-fb:96 nack pli a=fmtp:96 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f a=rtpmap:97 rtx/90000 a=fmtp:97 apt=96 a=rtpmap:98 H264/90000 a=rtcp-fb:98 goog-remb a=rtcp-fb:98 transport-cc a=rtcp-fb:98 ccm fir a=rtcp-fb:98 nack a=rtcp-fb:98 nack pli a=fmtp:98 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f a=rtpmap:99 rtx/90000 a=fmtp:99 apt=98 a=rtpmap:100 H264/90000 a=rtcp-fb:100 goog-remb a=rtcp-fb:100 transport-cc a=rtcp-fb:100 ccm fir a=rtcp-fb:100 nack a=rtcp-fb:100 nack pli a=fmtp:100 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f a=rtpmap:101 rtx/90000 a=fmtp:101 apt=100 a=rtpmap:127 H264/90000 a=rtcp-fb:127 goog-remb a=rtcp-fb:127 transport-cc a=rtcp-fb:127 ccm fir a=rtcp-fb:127 nack a=rtcp-fb:127 nack pli a=fmtp:127 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f a=rtpmap:125 rtx/90000 a=fmtp:125 apt=127 a=rtpmap:104 red/90000 a=rtpmap:124 rtx/90000 a=fmtp:124 apt=104 a=rtpmap:106 ulpfec/90000 a=ssrc-group:FID 1514350685 2123720248 a=ssrc:1514350685 cname:R3s9Fu9IFSb+GDTT a=ssrc:1514350685 msid:stream_id video_label a=ssrc:1514350685 mslabel:stream_id a=ssrc:1514350685 label:video_label a=ssrc:2123720248 cname:R3s9Fu9IFSb+GDTT a=ssrc:2123720248 msid:stream_id video_label a=ssrc:2123720248 mslabel:stream_id a=ssrc:2123720248 label:video_label

kclyu commented 4 years ago

RWS currently only supports H.264 codec. VP8/9 video codec is not supported by Raspberry PI hardware due to CPU load problem. There is no H.264 video description in flutter SDP offer, but it seems that h.264 codec is not enabled in flutter webrtc.

ropha commented 4 years ago

RWS currently only supports H.264 codec. VP8/9 video codec is not supported by Raspberry PI hardware due to CPU load problem. There is no H.264 video description in flutter SDP offer, but it seems that h.264 codec is not enabled in flutter webrtc.

thank you to show me the way.