search

w23 / zenki

Lightweight video surveillance
25 stars 10 forks source link

add audio stream? #2

Open MACscr opened 4 years ago

MACscr commented 4 years ago

First off, this is pretty awesome. Nice work and thanks for sharing it!

So my camera seems to have a h264 stream for video and an aac for audio. Any way to combine them both to be used within your solution?

w23 commented 4 years ago

Thanks! :D

The -avcodec copy argument should also handle audio similarly to video: pass it to HLS without transcoding. So I'd say it should work as is, but we can try to diagnose it further if it doesn't work for you.

First, can you hear the sound in VLC or mpv when:

There can be insurmountable browser compatibility issues with the stream camera produces. You can check that by forcing the script to reencode audio stream to various other expected to be compatible codecs by replacing -acodec copy by:

I haven't tried these options with this script, and not sure whether they behave well within HLS or MPEG-TS container, but it's worth a try. Even if it's the only option to make the sound work, audio encoding is not that compute-intensive by modern standards, so it should be mostly fine.

MACscr commented 4 years ago

They are literally two different streams from the camera as you can see below and I can indeed play them in VLC individually, but obviously i want a single browser stream with both.

image

If i try your solution with just the video, it does work for just the video. The h264 url does not contain sound. No idea why they send them separately. Definitely made it much harder for me to find a solution. Ultimately this particular camera is going to be our baby monitor, so obviously sound is important. Flashed the camera with custom firmware to add the RTSP support as we got sick of the cloud only solution that the vendor provided and we would lose access to the camera any time internet was lost, so wanted to connect direct.

w23 commented 4 years ago

Ah, I didn't get it the first time. So it literally is two different RTSP streams for audio and video, and not two muxed streams within one RTSP transport going wrong.

This should be generally possible, although I don't have an easy way to test this specific scenario locally right now. Can you try doing something simple, like replacing the first line with ffmpeg -i "$SOURCE" \ with just ffmpeg -i "<audio_url>" -i "<video_url>" \? Note that it is very likely there will be a noticeable (think seconds) desync.

Yeah, the "cloud" solution is what I'm trying to avoid with this little project. It just doesn't make any sense to me. Also, I'd advise running cameras in a separate isolated VLAN so that they can't access neither internet (they tend to try to communicate with random Chinese addresses for some reason :D), nor the rest of LAN with regular devices. But it's a bit painful and likely requires special network setup (e.g. not all home routers support VLANs).