search

elsampsa / valkka-core

Valkka - Create video surveillance, management and analysis programs with PyQt
GNU Lesser General Public License v3.0
181 stars 35 forks source link

Anticipated audio streaming support timeframe? #10

Closed tylertroy closed 3 years ago

tylertroy commented 4 years ago

Love this library thus far! Thank you for sharing with the world.

You mention in the docs that audio streaming is not currently supported. Is there any plan to include this in the near future?

I have very little experience in C++ but I would be pleased to help in any way I can to support this project.

elsampsa commented 4 years ago

Thanks for the offer! Help is very much appreciated.

We have 100+ ideas for this library, but unfortunately only very limited amount of time to dedicate for it.

For RTSP streams, this is where the fun starts: https://github.com/elsampsa/valkka-core/blob/21bfb39a47fad0006fd83d3dd28f449d6c27fb34/src/live.cpp#L500

As you can see, we have disabled audio there. You may enable it for testing. PCMU is the only accepted audio format for the moment, but that's very easy to change. Just add a new "else-if" section and fill in the BasicFrame structure correctly.

This could also be made in such a way that it seamlessly accepts all ffmpeg codecs, but I haven't spent a thought for that yet.

There might be some places in the library, where the audio packets might cause problems..

..so, once you create your own build, you may use this script for semi-automated testing: https://github.com/elsampsa/valkka-core/blob/master/tools/build/run_tests.bash (I like to link that file into my build directory)

Implementing audio would be (very vaguely) something like this:

I might be able to do some research about this in the coming months and write more specific guidelines (but can't really promise that)

tylertroy commented 4 years ago

Awesome. I only have bandwidth in my evenings. That coupled with my limited C++ knowledge will mean slow going but start I shall. These guidelines are a great starting point, thank you. I'll be sure to keep you abreast of any progress I make.