search

AlexxIT / go2rtc

Ultimate camera streaming application with support RTSP, RTMP, HTTP-FLV, WebRTC, MSE, HLS, MP4, MJPEG, HomeKit, FFmpeg, etc.
https://github.com/AlexxIT/Blog
MIT License
4.99k stars 399 forks source link

WebRTC no sound #678

Closed spawn451 closed 1 year ago

spawn451 commented 1 year ago

Hello, I have an issue using Realtek AMB82-Mini IoT AI Camera Arduino Dev. board . Video and audio works when using MSE mode (auto select mode) but there is no audio when using WebRTC mode. Bellow go2rtc.yaml and info.json. 

  AMB82:
    - rtsp://192.168.10.58:554/
    - ffmpeg:rtsp://192.168.10.58#audio=opus

MSE

{
  "producers": [
    {
      "type": "RTSP active producer",
      "url": "rtsp://192.168.10.58:554",
      "remote_addr": "192.168.10.58:554",
      "user_agent": "go2rtc/1.7.1",
      "sdp": "v=0\r\no=- 10045164 0 IN IP4 192.168.10.58\r\ns=ameba\r\nc=IN IP4 192.168.10.11\r\nt=0 0\r\nm=video 0 RTP/AVP 96\r\na=rtpmap:96 H264/90000\r\na=control:streamid=0\r\na=fmtp:96 packetization-mode=0\r\nm=audio 0 RTP/AVP 97\r\na=rtpmap:97 mpeg4-generic/8000/1\r\na=fmtp:97 streamtype=5; profile-level-id=15; mode=AAC-hbr; config=1588; sizeLength=13; indexLength=3; indexDeltaLength=3; constantDuration=1024; Profile=1\r\na=control:streamid=1\r\n",
      "medias": [
        "video, recvonly, H264",
        "audio, recvonly, MPEG4-GENERIC/8000"
      ],
      "receivers": [
        "96 H264, bytes=8293020, senders=1",
        "97 MPEG4-GENERIC/8000, bytes=33283, senders=1"
      ],
      "recv": 8402587
    },
    {
      "url": "ffmpeg:rtsp://192.168.10.58#audio=opus"
    }
  ],
  "consumers": [
    {
      "type": "MSE/WebSocket active consumer",
      "remote_addr": "192.168.10.50, 192.168.10.1:3213",
      "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, MPEG4-GENERIC, PCMA, PCMU, L16, PCML, OPUS"
      ],
      "senders": [
        "96 H264, bytes=3184164, receivers=1",
        "97 MPEG4-GENERIC/8000, bytes=33283, receivers=1"
      ],
      "send": 3182875
    }
  ]
}

WebRTC

{
  "producers": [
    {
      "type": "RTSP active producer",
      "url": "rtsp://192.168.10.58:554",
      "remote_addr": "192.168.10.58:554",
      "user_agent": "go2rtc/1.7.1",
      "sdp": "v=0\r\no=- 10115111 0 IN IP4 192.168.10.58\r\ns=ameba\r\nc=IN IP4 192.168.10.11\r\nt=0 0\r\nm=video 0 RTP/AVP 96\r\na=rtpmap:96 H264/90000\r\na=control:streamid=0\r\na=fmtp:96 packetization-mode=0\r\nm=audio 0 RTP/AVP 97\r\na=rtpmap:97 mpeg4-generic/8000/1\r\na=fmtp:97 streamtype=5; profile-level-id=15; mode=AAC-hbr; config=1588; sizeLength=13; indexLength=3; indexDeltaLength=3; constantDuration=1024; Profile=1\r\na=control:streamid=1\r\n",
      "medias": [
        "video, recvonly, H264",
        "audio, recvonly, MPEG4-GENERIC/8000"
      ],
      "receivers": [
        "96 H264, bytes=175261, senders=1"
      ],
      "recv": 178213
    },
    {
      "url": "ffmpeg:rtsp://192.168.10.58#audio=opus"
    }
  ],
  "consumers": [
    {
      "type": "WebRTC/WebSocket async passive consumer",
      "remote_addr": "udp4 host 192.168.10.50:59185",
      "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36",
      "medias": [
        "video, sendonly, VP8, RTX, VP9, H264, AV1, RED, ULPFEC, FLEXFEC-03",
        "audio, sendonly, OPUS/48000/2, RED/48000/2, G722/8000, PCMU/8000, PCMA/8000, CN/8000, TELEPHONE-EVENT/48000, TELEPHONE-EVENT/8000, L16, PCML"
      ],
      "senders": [
        "102 H264, bytes=175261, receivers=1"
      ],
      "send": 178544
    }
  ]
}
AlexxIT commented 1 year ago

Have you tried WebRTC mode? Not auto select

spawn451 commented 1 year ago

Find the solution

  AMB82:
    - ffmpeg:rtsp://192.168.10.58:554#video=copy#audio=opus
AlexxIT commented 1 year ago

What viewer you using? What OS? What Browser?

spawn451 commented 1 year ago

No Audio

  AMB82_test:
  - rtsp://192.168.10.58:554
  - ffmpeg:rtsp://192.168.10.58#audio=opus
 {
  "producers": [
    {
      "type": "RTSP active producer",
      "url": "rtsp://192.168.10.58:554",
      "remote_addr": "192.168.10.58:554",
      "user_agent": "go2rtc/1.7.1",
      "sdp": "v=0\r\no=- 10163858 0 IN IP4 192.168.10.58\r\ns=ameba\r\nc=IN IP4 192.168.10.50\r\nt=0 0\r\nm=video 0 RTP/AVP 96\r\na=rtpmap:96 H264/90000\r\na=control:streamid=0\r\na=fmtp:96 packetization-mode=0\r\nm=audio 0 RTP/AVP 97\r\na=rtpmap:97 mpeg4-generic/16000/1\r\na=fmtp:97 streamtype=5; profile-level-id=15; mode=AAC-hbr; config=1408; sizeLength=13; indexLength=3; indexDeltaLength=3; constantDuration=1024; Profile=1\r\na=control:streamid=1\r\n",
      "medias": [
        "video, recvonly, H264",
        "audio, recvonly, MPEG4-GENERIC/16000"
      ],
      "receivers": [
        "96 H264, bytes=566363, senders=1"
      ],
      "recv": 578879
    },
    {
      "url": "ffmpeg:rtsp://192.168.10.58#audio=opus"
    }
  ],
  "consumers": [
    {
      "type": "WebRTC/WebSocket async passive consumer",
      "remote_addr": "udp4 host 192.168.142.1:62271",
      "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36",
      "medias": [
        "video, sendonly, VP8, RTX, VP9, H264, AV1, RED, ULPFEC, FLEXFEC-03",
        "audio, sendonly, OPUS/48000/2, RED/48000/2, G722/8000, PCMU/8000, PCMA/8000, CN/8000, TELEPHONE-EVENT/48000, TELEPHONE-EVENT/8000, L16, PCML"
      ],
      "senders": [
        "102 H264, bytes=566363, receivers=1"
      ],
      "send": 580242
    }
  ]
} 
21:32:36.361 DBG [webrtc] new consumer src=AMB82_test
21:32:36.402 TRC [rtsp] client request:   
DESCRIBE rtsp://192.168.10.58:554 RTSP/1.0
User-Agent: go2rtc/1.7.1
CSeq: 1
Accept: application/sdp
Require: www.onvif.org/ver20/backchannel  

21:32:36.457 TRC [rtsp] client response:
RTSP/1.0 200 OK
Cseq: 1
Content-Type: application/sdp
Content-Base: rtsp://192.168.10.58:554
Content-Length: 417

v=0
o=- 10339798 0 IN IP4 192.168.10.58
s=ameba                                                                                                                                                   
c=IN IP4 192.168.10.50                                                                                                                                    
t=0 0                                                                                                                                                     
m=video 0 RTP/AVP 96                                                                                                                                      
a=rtpmap:96 H264/90000                                                                                                                                    
a=control:streamid=0                                                                                                                                      
a=fmtp:96 packetization-mode=0                                                                                                                            
m=audio 0 RTP/AVP 97                                                                                                                                      
a=rtpmap:97 mpeg4-generic/16000/1                                                                                                                         
a=fmtp:97 streamtype=5; profile-level-id=15; mode=AAC-hbr; config=1408; sizeLength=13; indexLength=3; indexDeltaLength=3; constantDuration=1024; Profile=1
a=control:streamid=1                                                                                                                                      

21:32:36.457 TRC [rtsp] client request:
SETUP rtsp://192.168.10.58:554/streamid=0 RTSP/1.0
Transport: RTP/AVP/TCP;unicast;interleaved=0-1
CSeq: 2

21:32:36.522 TRC [rtsp] client response:
RTSP/1.0 200 OK
Cseq: 2
Session: 10339798
Transport: RTP/AVP/TCP;unicast;interleaved=0-1

21:32:36.533 DBG [exec] run url="exec:ffmpeg -hide_banner -v error -allowed_media_types audio -fflags nobuffer -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -rtsp_flags prefer_tcp -i rtsp://192
.168.10.58 -c:a libopus -application:a lowdelay -frame_duration 20 -min_comp 0 -vn -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp {output}"
rtsp://192.168.10.58: Invalid data found when processing input
21:32:41.802 DBG [streams] can't stop none producer
21:32:41.802 DBG [streams] start producer url=rtsp://192.168.10.58:554
21:32:41.802 TRC [rtsp] client request:
PLAY rtsp://192.168.10.58:554 RTSP/1.0
CSeq: 3
Session: 10339798

21:32:41.879 TRC [rtsp] client response:
RTSP/1.0 200 OK
Cseq: 3
Session: 10339798

Audio OK

  AMB82:
    - ffmpeg:rtsp://192.168.10.58:554#video=copy#audio=opus
{
  "producers": [
    {
      "type": "RTSP passive producer",
      "url": "rtsp://127.0.0.1:8554/b69dacfcb8eb620c0f5344929fa2e706",
      "remote_addr": "127.0.0.1:45788",
      "user_agent": "ffmpeg/go2rtc",
      "sdp": "v=0\r\no=- 0 0 IN IP4 127.0.0.1\r\ns=ameba\r\nc=IN IP4 127.0.0.1\r\nt=0 0\r\na=tool:libavformat 60.3.100\r\nm=video 0 RTP/AVP 96\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1; sprop-parameter-sets=J2QAM6wTGqBQBboQAAADABAAAAMDwPFiW4A=,KO4CkZIZAA==; profile-level-id=640033\r\na=control:streamid=0\r\nm=audio 0 RTP/AVP 97\r\nb=AS:64\r\na=rtpmap:97 opus/48000/2\r\na=control:streamid=1\r\n",
      "medias": [
        "video, recvonly, H.264 High 5.1",
        "audio, recvonly, OPUS/48000/2"
      ],
      "receivers": [
        "96 H264, bytes=345130, senders=1",
        "97 OPUS/48000/2, bytes=100282, senders=1"
      ],
      "recv": 459740
    }
  ],
  "consumers": [
    {
      "type": "WebRTC/WebSocket async passive consumer",
      "remote_addr": "udp4 host 192.168.142.1:60362",
      "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36",
      "medias": [
        "video, sendonly, VP8, RTX, VP9, H264, AV1, RED, ULPFEC, FLEXFEC-03",
        "audio, sendonly, OPUS/48000/2, RED/48000/2, G722/8000, PCMU/8000, PCMA/8000, CN/8000, TELEPHONE-EVENT/48000, TELEPHONE-EVENT/8000, L16, PCML"
      ],
      "senders": [
        "102 H264, bytes=345130, receivers=1",
        "111 OPUS/48000/2, bytes=100282, receivers=1"
      ],
      "send": 460265
    }
  ]
}
21:30:55.684 DBG [webrtc] new consumer src=AMB82
21:30:55.695 DBG [exec] run url="exec:ffmpeg -hide_banner -v error -allowed_media_types video+audio -fflags nobuffer -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -rtsp_flags prefer_tcp -i rtsp
://192.168.10.58:554 -c:v copy -c:a libopus -application:a lowdelay -frame_duration 20 -min_comp 0 -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp {output}"
[h264 @ 0000020e48c75d80] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0000020e48c75d80] decode_slice_header error    
[h264 @ 0000020e48c75d80] no frame!
[h264 @ 0000020e48c75d80] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0000020e48c75d80] decode_slice_header error    
[h264 @ 0000020e48c75d80] no frame!
[h264 @ 0000020e48c75d80] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0000020e48c75d80] decode_slice_header error    
[h264 @ 0000020e48c75d80] no frame!
[h264 @ 0000020e48c75d80] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0000020e48c75d80] decode_slice_header error    
[h264 @ 0000020e48c75d80] no frame!                    
[h264 @ 0000020e48c75d80] non-existing PPS 0 referenced
    Last message repeated 1 times                      
[h264 @ 0000020e48c75d80] decode_slice_header error    
[h264 @ 0000020e48c75d80] no frame!                    
[h264 @ 0000020e48c75d80] non-existing PPS 0 referenced
    Last message repeated 1 times                      
[h264 @ 0000020e48c75d80] decode_slice_header error    
[h264 @ 0000020e48c75d80] no frame!                    
21:30:57.187 TRC [rtsp] server request:                                
OPTIONS rtsp://127.0.0.1:8554/b69dacfcb8eb620c0f5344929fa2e706 RTSP/1.0
Cseq: 1                                                                
User-Agent: ffmpeg/go2rtc

21:30:57.187 TRC [rtsp] server response:
RTSP/1.0 200 OK
Public: OPTIONS, SETUP, TEARDOWN, DESCRIBE, PLAY, PAUSE, ANNOUNCE, RECORD
Cseq: 1

21:30:57.188 TRC [rtsp] server request:
ANNOUNCE rtsp://127.0.0.1:8554/b69dacfcb8eb620c0f5344929fa2e706 RTSP/1.0
User-Agent: ffmpeg/go2rtc
Content-Length: 372
Content-Type: application/sdp
Cseq: 2

v=0
o=- 0 0 IN IP4 127.0.0.1
s=ameba
c=IN IP4 127.0.0.1
t=0 0
a=tool:libavformat 60.3.100
m=video 0 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 packetization-mode=1; sprop-parameter-sets=J2QAM6wTGqBQBboQAAADABAAAAMDwPFiW4A=,KO4CkZIZAA==; profile-level-id=640033
a=control:streamid=0
m=audio 0 RTP/AVP 97
b=AS:64
a=rtpmap:97 opus/48000/2
a=control:streamid=1

21:30:57.189 TRC [rtsp] server response:
RTSP/1.0 200 OK
Cseq: 2

21:30:57.189 TRC [rtsp] server request:
SETUP rtsp://127.0.0.1:8554/b69dacfcb8eb620c0f5344929fa2e706/streamid=0 RTSP/1.0
Transport: RTP/AVP/TCP;unicast;interleaved=0-1;mode=record
Cseq: 3
User-Agent: ffmpeg/go2rtc

21:30:57.189 TRC [rtsp] server response:
RTSP/1.0 200 OK
Transport: RTP/AVP/TCP;unicast;interleaved=0-1
Cseq: 3
Session: 62333356;timeout=60

21:30:57.190 TRC [rtsp] server request:
SETUP rtsp://127.0.0.1:8554/b69dacfcb8eb620c0f5344929fa2e706/streamid=1 RTSP/1.0
Cseq: 4
User-Agent: ffmpeg/go2rtc
Session: 62333356
Transport: RTP/AVP/TCP;unicast;interleaved=2-3;mode=record

21:30:57.190 TRC [rtsp] server response:
RTSP/1.0 200 OK
Transport: RTP/AVP/TCP;unicast;interleaved=2-3
Cseq: 4
Session: 80076123;timeout=60

21:30:57.191 TRC [rtsp] server request:
RECORD rtsp://127.0.0.1:8554/b69dacfcb8eb620c0f5344929fa2e706 RTSP/1.0
Range: npt=0.000-
Cseq: 5
User-Agent: ffmpeg/go2rtc
Session: 62333356

21:30:57.191 TRC [rtsp] server response:
RTSP/1.0 200 OK
Cseq: 5
Session: 80076123

21:30:57.191 DBG [exec] run launch=1.4956638s
21:30:57.192 DBG [streams] start producer url=ffmpeg:rtsp://192.168.10.58:554#video=copy#audio=opus
AlexxIT commented 1 year ago

This is output from ffmpeg:

rtsp://192.168.10.58: Invalid data found when processing input

Try this:

sources:
  AMB82:
    - rtsp://192.168.10.58:554/
    - ffmpeg:AMB82#audio=opus
spawn451 commented 1 year ago

Thanks, this works :)

streams:
  AMB82:
    - rtsp://192.168.10.58:554
    - ffmpeg:AMB82#audio=opus
AlexxIT commented 1 year ago

So I think it's camera/ffmpeg problem.