search

ossrs / srs

SRS is a simple, high-efficiency, real-time media server supporting RTMP, WebRTC, HLS, HTTP-FLV, HTTP-TS, SRT, MPEG-DASH, and GB28181.
https://ossrs.io
MIT License
25.43k stars 5.35k forks source link

Publish WebRTC Firefox error SDP #2913

Open thegobot opened 2 years ago

thegobot commented 2 years ago

Note: Before asking a question, please read the FAQ (Please read FAQ before filing an issue) https://github.com/ossrs/srs/issues/2716

Description pc.setRemoteDescription(sdp).catch() Generate DOMException: Answer and offer have different media types at m-line 0

I noticed that Firefox puts the video media first (mid:0) in the SDP, then the audio (mid:1). SRS is the opposite, but the numbers are the same...

I found the line of code that generates this error https://github.com/mozilla/gecko-dev/blob/c12a59323ee46b29b90c9917a3a7a70ea714ffec/dom/media/webrtc/jsep/JsepSessionImpl.cpp#L1884

Please describe your issue here

  1. SRS version: 4.0.238

  2. SRS log:

[2022-02-12 03:59:28.931][Trace][14882][fg4i314f] HTTP #0 127.0.0.1:50332 POST http://my.host.com/rtc/v1/publish/, content-length=3041
[2022-02-12 03:59:28.998][Trace][14882][fg4i314f] http: on_publish ok, client_id=fg4i314f, url=http://my.host.com/auth, request={"server_id":"vid-192711d","action":"on_publish","client_id":"fg4i314f","ip":"115.34.130.119","vhost":"webrtc","app":"webrtc","tcUrl":"https://my.host.com/webrtc?key=UC5r3BJNSg&vhost=webrtc","stream":"safasdf_adsfdasfdas_15893_198","param":"?key=UC5r3BJNSg&vhost=webrtc"}, response=0
[2022-02-12 03:59:28.998][Trace][14882][fg4i314f] RTC publish https://my.host.com/webrtc?key=UC5r3BJNSg&vhost=webrtc/safasdf_adsfdasfdas_15893_198, api=, tid=safasdf_adsfdasfdas_15893_198, clientip=115.34.130.119, app=webrtc, stream=safasdf_adsfdasfdas_15893_198, offer=2728B, eip=, codec=
[2022-02-12 03:59:28.999][Trace][14882][fg4i314f] new source, stream_url=webrtc/webrtc/safasdf_adsfdasfdas_15893_198
[2022-02-12 03:59:28.999][Trace][14882][fg4i314f] RTC publisher nack=1, nnc=1, pt-drop=0, twcc=1/7
[2022-02-12 03:59:28.999][Trace][14882][fg4i314f] new source, stream_url=webrtc/webrtc/safasdf_adsfdasfdas_15893_198
[2022-02-12 03:59:29.000][Trace][14882][fg4i314f] RTC: Init tracks {track: {318c119b-54f1-4058-8ecb-40d6d32cd29b}, is_active: 0=>1},{track: {dd22be7c-e210-4372-822c-acd9a82d0f85}, is_active: 0=>1}, ok
[2022-02-12 03:59:29.000][Trace][14882][fg4i314f] RTC: Use candidates 64.8.95.36
[2022-02-12 03:59:29.001][Trace][14882][fg4i314f] RTC init session, user=j38372ii:1ad7b5d3, url=webrtc/webrtc/safasdf_adsfdasfdas_15893_198, encrypt=1/1, DTLS(role=passive, version=auto), timeout=30000ms, nack=1
[2022-02-12 03:59:29.001][Trace][14882][fg4i314f] RTC username=j38372ii:1ad7b5d3, offer=2728B, answer=1317B
[2022-02-12 03:59:29.001][Trace][14882][fg4i314f] RTC remote offer: v=0\r\no=mozilla...THIS_IS_SDPARTA-97.0 4906095697022403286 0 IN IP4 0.0.0.0\r\ns=-\r\nt=0 0\r\na=fingerprint:sha-256 7A:CF:57:8C:69:FA:C3:7B:E7:B0:BC:5C:09:C2:22:02:ED:12:10:70:FE:61:65:DF:EB:19:27:C5:B8:09:87:F7\r\na=group:BUNDLE 0 1\r\na=ice-options:trickle\r\na=msid-semantic:WMS *\r\nm=video 9 UDP/TLS/RTP/SAVPF 120 124 121 125 126 127 97 98\r\nc=IN IP4 0.0.0.0\r\na=sendonly\r\na=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid\r\na=extmap:4 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=extmap:5 urn:ietf:params:rtp-hdrext:toffset\r\na=extmap:6/recvonly http://www.webrtc.org/experiments/rtp-hdrext/playout-delay\r\na=extmap:7 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=fmtp:126 profile-level-id=42e01f;level-asymmetry-allowed=1;packetization-mode=1\r\na=fmtp:97 profile-level-id=42e01f;level-asymmetry-allowed=1\r\na=fmtp:120 max-fs=12288;max-fr=60\r\na=fmtp:124 apt=120\r\na=fmtp:121 max-fs=12288;max-fr=60\r\na=fmtp:125 apt=121\r\na=fmtp:127 apt=126\r\na=fmtp:98 apt=97\r\na=ice-pwd:c9e7840989f4cd0d14388361e93a2c99\r\na=ice-ufrag:1ad7b5d3\r\na=mid:0\r\na=msid:{ce2664c1-5ab1-4726-834a-197fb0995664} {318c119b-54f1-4058-8ecb-40d6d32cd29b}\r\na=rtcp-fb:120 nack\r\na=rtcp-fb:120 nack pli\r\na=rtcp-fb:120 ccm fir\r\na=rtcp-fb:120 goog-remb\r\na=rtcp-fb:120 transport-cc\r\na=rtcp-fb:121 nack\r\na=rtcp-fb:121 nack pli\r\na=rtcp-fb:121 ccm fir\r\na=rtcp-fb:121 goog-remb\r\na=rtcp-fb:121 transport-cc\r\na=rtcp-fb:126 nack\r\na=rtcp-fb:126 nack pli\r\na=rtcp-fb:126 ccm fir\r\na=rtcp-fb:126 goog-remb\r\na=rtcp-fb:126 transport-cc\r\na=rtcp-fb:97 nack\r\na=rtcp-fb:97 nack pli\r\na=rtcp-fb:97 ccm fir\r\na=rtcp-fb:97 goog-remb\r\na=rtcp-fb:97 transport-cc\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:120 VP8/90000\r\na=rtpmap:124 rtx/90000\r\na=rtpmap:121 VP9/90000\r\na=rtpmap:125 rtx/90000\r\na=rtpmap:126 H264/90000\r\na=rtpmap:127 rtx/90000\r\na=rtpmap:97 H264/90000\r\na=rtpmap:98 rtx/90000\r\na=setup:actpass\r\na=ssrc:2695188846 cname:{0845f19d-5124-4ef6-80d0-f3cd900d84f4}\r\na=ssrc:4273023225 cname:{0845f19d-5124-4ef6-80d0-f3cd900d84f4}\r\na=ssrc-group:FID 2695188846 4273023225\r\nm=audio 9 UDP/TLS/RTP/SAVPF 109 9 0 8 101\r\nc=IN IP4 0.0.0.0\r\na=sendonly\r\na=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level\r\na=extmap:2/recvonly urn:ietf:params:rtp-hdrext:csrc-audio-level\r\na=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid\r\na=fmtp:109 maxplaybackrate=48000;stereo=1;useinbandfec=1\r\na=fmtp:101 0-15\r\na=ice-pwd:c9e7840989f4cd0d14388361e93a2c99\r\na=ice-ufrag:1ad7b5d3\r\na=mid:1\r\na=msid:{ce2664c1-5ab1-4726-834a-197fb0995664} {dd22be7c-e210-4372-822c-acd9a82d0f85}\r\na=rtcp-mux\r\na=rtpmap:109 opus/48000/2\r\na=rtpmap:9 G722/8000/1\r\na=rtpmap:0 PCMU/8000\r\na=rtpmap:8 PCMA/8000\r\na=rtpmap:101 telephone-event/8000/1\r\na=setup:actpass\r\na=ssrc:3948349321 cname:{0845f19d-5124-4ef6-80d0-f3cd900d84f4}\r\n
[2022-02-12 03:59:29.001][Trace][14882][fg4i314f] RTC local answer: v=0\r\no=SRS/4.0.238(Leo) 94807242081952 2 IN IP4 0.0.0.0\r\ns=SRSPublishSession\r\nt=0 0\r\na=ice-lite\r\na=group:BUNDLE 1 0\r\na=msid-semantic: WMS webrtc/safasdf_adsfdasfdas_15893_198\r\nm=audio 9 UDP/TLS/RTP/SAVPF 109\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:j38372ii\r\na=ice-pwd:333a0e49521ok134e5yjq2311tb940gh\r\na=fingerprint:sha-256 8D:4A:E9:58:40:45:2C:5F:42:ED:61:20:32:77:43:EE:D5:56:16:C5:2C:15:CB:6D:8E:B2:8C:44:F2:BD:AE:61\r\na=setup:passive\r\na=mid:1\r\na=recvonly\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:109 opus/48000/2\r\na=fmtp:109 ;useinbandfec=1\r\na=candidate:0 1 udp 2130706431 64.8.95.36 9999 typ host generation 0\r\nm=video 9 UDP/TLS/RTP/SAVPF 126\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:j38372ii\r\na=ice-pwd:333a0e49521ok134e5yjq2311tb940gh\r\na=fingerprint:sha-256 8D:4A:E9:58:40:45:2C:5F:42:ED:61:20:32:77:43:EE:D5:56:16:C5:2C:15:CB:6D:8E:B2:8C:44:F2:BD:AE:61\r\na=setup:passive\r\na=mid:0\r\na=extmap:7 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=recvonly\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:126 H264/90000\r\na=rtcp-fb:126 nack\r\na=rtcp-fb:126 nack pli\r\na=rtcp-fb:126 transport-cc\r\na=fmtp:126 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\na=candidate:0 1 udp 2130706431 64.8.95.36 9999 typ host generation 0\r\n
[2022-02-12 03:59:29.001][Trace][14882][fg4i314f] TCP: before dispose resource(HttpConn)(0x563a07faeea0), conns=1, zombies=0, ign=0, inz=0, ind=0
[2022-02-12 03:59:29.001][Trace][14882][fg4i314f] client finished.
[2022-02-12 03:59:29.001][Trace][14882][19634w7d] TCP: clear zombies=1 resources, conns=1, removing=0, unsubs=0
[2022-02-12 03:59:29.001][Trace][14882][fg4i314f] TCP: disposing #0 resource(HttpConn)(0x563a07faeea0), conns=1, disposing=1, zombies=0
[2022-02-12 03:59:29.035][Trace][14882][5569jgi6] Hybrid cpu=0.00%,13MB, cid=1,1, timer=63,0,0, clock=0,49,0,0,0,0,0,0,0
[2022-02-12 03:59:29.035][Trace][14882][5569jgi6] RTC: Server conns=1
^C[2022-02-12 03:59:31.084][Trace][14882][0c969895] sig=2, user terminate program, fast quit
[2022-02-12 03:59:32.046][Trace][14882][0c969895] cleanup for quit signal fast=1, grace=0
[2022-02-12 03:59:32.046][Warn][14882][0c969895][11] main cycle terminated, system quit normally.
[2022-02-12 03:59:32.147][Trace][14882][0c969895] srs disposed
[2022-02-12 03:59:32.147][Trace][14882][0c969895] srs terminated
  1. SRS configuration:
xxxxxxxxxxxx

Replay

How to replay bug?

  1. Start Mozilla Firefiox (ver 97, Windows 7)
  2. createOffer()
  3. send offer
  4. Got answer from SRS
  5. setRemoteDescription()

Expect setRemoteDescription() success Please describe your expectation

TRANS_BY_GPT3

thegobot commented 2 years ago

I was able to run in ff by modifying SDP I just put the video block at the beginning and the audio at the end of the SDP

workaround

let remoteSDP = json.sdp;
let remoteSDPForFF = "";

let startAudio = remoteSDP.match(/(m=audio)/);
let startVideo = remoteSDP.match(/(m=video)/);

if(startAudio && startVideo){

    let audioBlock = remoteSDP.slice(startAudio.index, startVideo.index);
    let videoBlock = remoteSDP.slice(startVideo.index);

    remoteSDPForFF = remoteSDP.slice(0, startAudio.index);

    remoteSDPForFF += videoBlock;
    remoteSDPForFF += audioBlock;
}
winlinvip commented 2 years ago

It seems a bug introduced by yourself? So we can't replay it by Firefox only?

johzzy commented 2 years ago

In this question, I don't know how you operate the "add track" function on PC corresponding to "publish". By default, SRS assumes that audio is added before video.

TRANS_BY_GPT3

thegobot commented 2 years ago

The code above is a workaround You can make your own publish in FF

The fact is that FF(Win7) ignores the MID-N and determines the types in ascending order.