No audio in rtsp stream on WebRTC Camera Card

AlexxIT / WebRTC

Home Assistant custom component for real-time viewing of almost any camera stream using WebRTC and other technologies.

https://github.com/AlexxIT/Blog

MIT License

1.35k stars 161 forks source link

No audio in rtsp stream on WebRTC Camera Card #634

Open hapklaar opened 5 months ago

hapklaar commented 5 months ago

As far as I understand audio and video should both be active when using the WebRTC Camera Card as both media types are standard, but there is no audio. When I open the stream in VLC I have both audio and video.

My config is very simple:

WebRTC Camera Card configuration:

type: custom:webrtc-camera
url: rtsp://user:pass@192.168.x.x:554/h264Preview_01_main

Also tried using the go2rtc.yaml route:

streams:
  reo_doorbell: rtsp://user:pass@192.168.x.x:554/h264Preview_01_main

with WebRTC Camera Card configuration:

type: custom:webrtc-camera
streams:
  - url: reo_doorbell
    media: video,audio

What am I missing?

AlexxIT commented 5 months ago

https://github.com/AlexxIT/go2rtc?tab=readme-ov-file#codecs-madness

hapklaar commented 5 months ago

Sweet, it works now. I'll document what I did here for other to see.

According to VLC in my stream AAC is used for audio, this can't be changed unfortunately. As far as I understand now audio doesn't work because this codec is not supported by 'Desktop Edge' which is what I'm using, so I need to transcode.

Changed my go2rtc.yaml to this:

streams:
  reo_doorbell: 
    - rtsp://user:pass@192.168.x.x:554/h264Preview_01_main
    - ffmpeg:rtsp://user:pass@192.168.x.x:554/h264Preview_01_main#audio=opus

After restarting HA, audio works

AlexxIT commented 5 months ago

AAC should works via MSE technology in all browsers

hapklaar commented 5 months ago

I had no luck with mse. Only a frozen image or completely black. Only webrtc works for me.

AlexxIT commented 5 months ago

This is because you haven't read carefully https://github.com/AlexxIT/go2rtc?tab=readme-ov-file#source-rtsp

Sometimes Reolink RTSP is broken trash. MSE can't handle it.

hapklaar commented 5 months ago

I actually did see that, it doesn't mention anything specific about Reolink and MSE, just that RTSP on Reolink is unreliable, which we can agree on.

I'll sum up my experience:

RTSP HD + MSE works in HA app on my iPad, with flaky AAC sound
RTSP HD + MSE doesn't work in Edge on my Windows desktop. No sound, no video
RTSP HD + RTC works fine on both, but without sound.
RTSP HD + RTC works with sound on Edge, when transcoded opus (only one I tried)
RTMP with HD stream works, but very bad framerate
RTMP with SD stream works fine

All with same Reolink cam (Doorbell cam Wifi)

AlexxIT commented 5 months ago

MSE can't handle broken stream from any camera brand.

I don't know why you have problems via MSE on edge. This can be because H265. But if RTC works for you, this mean you have H264.

RTMP also has problems on Reolink cameras. Better to use HTTP FLV.

deezid commented 1 month ago

MSE can't handle broken stream from any camera brand.

I don't know why you have problems via MSE on edge. This can be because H265. But if RTC works for you, this mean you have H264.

RTMP also has problems on Reolink cameras. Better to use HTTP FLV.

Switching to Https/FLV with Opus audio did the trick for me with mode set to webrtc in webrtc custom card. Works flawlessly.