search

JurajNyiri / HomeAssistant-Tapo-Control

Control for Tapo cameras as a Home Assistant component
Apache License 2.0
1.12k stars 87 forks source link

Is there a way to get Audio from the camera streams? #58

Closed krasatos closed 3 years ago

krasatos commented 3 years ago

Hey there, and thank you for this great integration. Mainly a question, secondly, if possible, a feature request.

I was wondering what's going on with sound. I have seen several posts in the community saying that several onvif streams have audio and others are muted by code.

What can i do about it? Is there some way to enable sound? Is it impossible?

Thank you

JurajNyiri commented 3 years ago

RTSP stream does support sound via VLC, I will look into this.

JurajNyiri commented 3 years ago

Audio from my other cameras work via Onvif integration (aac codec) Audio from Tapo via Onvif integration (and this one) doesn't work.

So this is the codec information of Tapo camera: image

This is the PR which added audio capability to Home Assistant.

Quote: This PR muxes the audio stream into the output HLS stream if the codec is supported. If not, no audio stream is sent along with the video, saving bandwidth. Note that PCM codecs such as G.711 are not supported as they are not supported in mp4 containers.

Supported audio codecs are "aac", "ac3" and "mp3".

Unfortunately not possible. Support will have to come from Home Assistant side.

mrwsl commented 3 years ago

Since the PR got merged into dev: can we expect audio to be working in a future home assistant release? Never mind, can't read..

krasatos commented 3 years ago

@JurajNyiri sorry for bringing this up again, could you instruct me on what further steps i could take to make this happen? If i understand correctly, we need the PCM ALAW codec to be muxed into the stream. Right now only "aac", "ac3" and "mp3" codecs are supported. So this could be solved by including PCM codecs in the stream, from the HA side, or Tapo firmware to use different codec right?

How could one go and request this? In the community forum of HA or somewhere in github, could you please point me to that direction? I am guessing that changing the tapo firmware would be out of the question.

JurajNyiri commented 3 years ago

Yes you are correct, change would have to happen either from Home Assistant or Tapo firmware.

You can request a new feature here: https://community.home-assistant.io/c/feature-requests/13 .

Browsers seem to support PCM ALAW in webrtc https://developer.mozilla.org/en-US/docs/Web/Media/Formats/WebRTC_codecs but not in audio/video tags afaik.

krasatos commented 3 years ago

Thank you, i will go start a feature request on the HA-side.

RestOp commented 3 years ago

But is it possible just to extract this audio stream and record it into file? I need only audio recordings not real time listening.

Or maybe you know any other way to record audio from Tapo?

Thanks a lot!

JurajNyiri commented 3 years ago

Yes but out of scope of this integration. You can use ffmpeg directly to output audio to file.

RestOp commented 3 years ago

Yes but out of scope of this integration. You can use ffmpeg directly to output audio to file.

Thanks a lot! But can i use your itegration (i like it very much!) together with ffmpeg? Will it make any interference\problems?

Thanks a lot!

VasekPurchart commented 3 years ago

Hi, I don't have a camera or HA yet - doing preliminary research. From what I was reading about showing streams from cameras using RTSP and showing the stream in browser, there is currently no direct ( and native+universal) solution (https://flashphoner.com/browser-based-webrtc-stream-from-rtsp-ip-camera-with-low-latency/).

So I gather that HA is already doing some transcoding (probably using ffmpeg since it is based on the stream integration). And from what I understand it is quite simple for ffmpeg to convert the PCM audio to something usable (AAC)?

I tried this today and had a proof of concept with working sound (but only with PCM ulaw stream, could not find a alaw stream), basically using https://video.stackexchange.com/questions/29887/ffmpeg-convert-g-711-audio-to-aac-audio .

And since my result was just an browser-consumable stream, which is processed by ffmpeg in the backgound, isn't it basically the same HA does? And if so, could I just point HA to this new stream and use it directly (ideally without another ffmpeg processing)? Or is there something I am missing?

h0jeZvgoxFepBQ2C commented 1 year ago

@krasatos did you open a feature issue on Home Assistant? Would be nice to know the status

eerison commented 1 year ago

Was it solved?

Audio just works when I run the command link to save the video. But it doesn't work on browser , just the video is reproduced 😢

ccdunder commented 1 year ago

Awesome project! I'd love to start using it with C320WSs. I noticed this issue and want to share what I've learned over the years regarding PCM audio:

Upstream HomeAssistant cannot "fix" or change what audio codecs MP4 allows. It's defined by ISO 14496-14.

So it looks like there's at least two ways forward:

  1. Transcode the audio using gstreamer, which is available in hass. I've done this for my C320WS. Happy to share the code if it would help.
  2. Use a different container or stream protocol that supports PCM audio e.g. RTSP or WebRTC.

Which approach sounds best to you? (no pun intended)

markrutten commented 1 year ago

Awesome project! I'd love to start using it with C320WSs. I noticed this issue and want to share what I've learned over the years regarding PCM audio:

Upstream HomeAssistant cannot "fix" or change what audio codecs MP4 allows. It's defined by ISO 14496-14.

So it looks like there's at least two ways forward:

  1. Transcode the audio using gstreamer, which is available in hass. I've done this for my C320WS. Happy to share the code if it would help.
  2. Use a different container or stream protocol that supports PCM audio e.g. RTSP or WebRTC.

Which approach sounds best to you? (no pun intended)

Hi, I am interested in how you solved this, could you please share your solution with some explanation? Does this solution work with the TAPA add-on from github? I have two C225 camera's.

Thanks in advance, kind regards Mark