Webrtc crackling sound on multiple camera brands

[borgqueenx]

Whether i send audio mp3 files using the home assistant webrtc integration(and add the cameras in configuration.yaml as media players) or send tts voice messages, the sound is crackling like crazy. Sometimes the tts messages are hard to understand and sounds are very clunky and crackling. A weird note to make is that this does not happen on the reolink duo 3, it plays sounds perfectly. But on annke, hikvision and tapo cameras, the problem is the same. Sometimes(but rarely) sounds play normally without crackling up. I already tried to add a delay to the sound of 2 seconds without luck.

I would love to find out with the community here how this is solvable.

[AlexxIT]

This can happen if you have chosen the wrong codec for your camera

[borgqueenx]

Which ones should i use for tapo and annke/hikvision and how do i select the codec? I assume annke and hikvision share the same codec. Maybe i need to transcode the file from mp3 to something else?

Is it also correct that the audio you can select in annke and hikvision software is just for recording, not for sending audio towards them?

[AlexxIT]

You can find codec in stream info in go2rtc WebUI. You selecting codec while adding camera as media player.

[borgqueenx]

Under active producers it says the following at annke and hikvision cameras: Profile=1;\r\na=Media_header:MEDIAINFO=494D4B48010300000400050010710110401F000000FA000000000000000000000000000000000000;\r\na=appversion:1.0\r\n", "medias": [ "video, recvonly, H265", "audio, recvonly, PCMU/8000", "audio, sendonly, PCMU/8000, G726/8000, MPEG4-GENERIC/16000" ], "receivers": [ "96 H265, bytes=37299853942, senders=1", "0 PCMU/8000, bytes=325004160, senders=1" ], "senders": [ "0 PCMU/8000, bytes=18858, receivers=3" ], "recv": 37973941648, "send": 20362 },

However when in configuration.yaml i tried MPEG4-GENERIC/16000 and after G726/8000 as codec, i get a unknown error when executing the send audio command.

I used pcmu/8000 which worked, but is crackling.

[AlexxIT]

PCMU is right codec for this camera. Unfortunately, I don't have any Hikvision cameras to run a test.

[borgqueenx]

It is weird that also tapo cameras suffer from this, while using a other codec then hikvision and annke. Is there a way to debug this?

[AlexxIT]

I have tapo. They used PCMA and works fine.

[borgqueenx]

But pretty much all my cameras are crackling. The only one that never did was my reolink duo 3. I have 3 annke cameras, 2 hikvision cameras and 9 tapo cameras (different models)

[NickM-27]

What is your go2rtc config

[borgqueenx]

What is your go2rtc config

A tapo and a annke camera. I dont have a pc to easily copy larger parts of the huge config file so hope this helps. The annke camera has the mainstream(h265), substream(h264) and the tapo cams (only support h264) have the rtsp mainstream and the tapo protocol mainstream. Both cameras also have a ffmpeg aac encoding.

[NickM-27]

There is a known bug when using hikvision cameras that rtsp backchannel will have crackling. You should use isapi instead

[borgqueenx]

There is a known bug when using hikvision cameras that rtsp backchannel will have crackling. You should use isapi instead

Is this a bug with hikvision and annke or with webrtc? How do i set up isapi with webrtc to send audio to cameras with home assistant?

[NickM-27]

See https://github.com/AlexxIT/go2rtc/issues/124

[borgqueenx]

See #124

I just tried to put backchannel=0 to the streams, but it breaks sending sound to the hikvision and annke cameras. I have seen the reference to isapi in the thread you linked but its just for two-way audio it seems..for talking to people...i am using the webrtc integration to send audio files to the cameras.

[NickM-27]

for talking to people...i am using the webrtc integration to send audio files to the cameras.

It is used for both just like rtsp backchannel and that you what you need to use

Its what I've been using for a year

[borgqueenx]

for talking to people...i am using the webrtc integration to send audio files to the cameras.

It is used for both just like rtsp backchannel and that you what you need to use

Its what I've been using for a year

Does this integrate with webrtc and/or frigate? So it works seamless.

[NickM-27]

It works exactly the same as before it's just a different way of getting audio to the camera

[borgqueenx]

It works exactly the same as before it's just a different way of getting audio to the camera

Yes but how do we get this in there? The issue you linked has a python example script. But how do we use this for hikvision and annke?

[NickM-27]

https://github.com/AlexxIT/go2rtc#source-isapi

[borgqueenx]

Thanks! I just played around but when i press the mic button in frigate, a empty video icon shows

I already tried to comment out the h264 substream without luck.

Edit: i realise its getting a bit offtopic, but i didnt know isapi:// is working in go2rtc. I hope this can fix the issue for sending audio to all my cameras.

I would still love it if there is a way to debug the tapo cameras after this. Maybe its not going to be possible then i'l also accept that. I know the hardware of tapo is weak, but just a small audio file? Even the weakest hardware in 2024 should be able to handle that. But maybe the protocol or backchannel of tapo is simply something unsolvable, but perhaps i can then email tapo in order to fix this...if we find something straight forward to tell them what the problem is.