At moment the video stream is sended using a "unknown" fourCC. Ninjam clients are accepting the OGGv fourCC only. JamTaba video streams are discarded (because I'm using JTBv as fourCC) when received by another ninjam clients. Off course JamTaba can detect the JTBv fourCC and render the video stream.
Using this fourCC trick musicians can use video in JamTaba without being incompatible with another ninjam clients. The trick works, but the audio and video are separated streams, and in general are not in sync.
My Experiments
First I opened an audio file saved by ReaNinjam in NotePad++. The first 4 bytes are OggS, and after some bytes becomes the vorbis header.
So, I realized ninjam was always using OGG as container, and vorbis audio inside the container. This is very nice because is possible explore the OGG container possibilites.
I started sending a OGV (vorbis audio and theora video) file to ReaNinjam, and got nothing in the other side. No audio or video. I was expecting hear the audio.
I compiled the ninjam winclient to debug the vorbis decoder. When debuging I see the decoder is assuming all streams in the container are vorbis (the theora stream is messing the decoder).
To get the vorbis decoder working (decoding the audio stream but ignoring the theora) I just removed some lines in the decoder code.
I generated an OGV file with the audio stream first, so the second stream (theora) is ignored by vorbis decoder and the audio is rendered without problems. It's working, using this approach is possible send audio + video inside an OGG container.
My proposal
When ninjam client receive an OGG stream containing audio + video no audio is rendered/decoded.
In this commit the problematic piece code in vorbis decoder was removed. Testing in ninjam winclient this solution works.
If an OGV file (audio + video) is streamed the ninjam winclient can decode the audio. The video stream is just ignored.
Important: The "solution" is not working if the video stream appears first in the OGG container. Is mandatory put the audio stream first in OGG container.
Benefits
JamTaba can show video streams in sync with vorbis audio (nice!!! 😄 )
Another ninjam clients can render video too.
Issues
I compiled the ReaNinjam (thanks for share the code 👍 😄 ). The changes in vorbis decoder are not working even after undefine WDL_VORBIS_INTERFACE_ONLY. ReaNinjam is always dicarding audio and video. Maybe ReaNinjam is using another decoder implementation?
I'm proposing a change in a piece of code inside WDL, maybe this will break other codes?
Hello Mr. Frankel!
First of all it's a honor contact you! Thanks for all work in ninjam, Reaper, etc.
Some context for this PR
I'm the mantainer of JamTaba, a ninjam client. JamTaba is streaming video (I'm using the FFmpeg lib).
http://jamtaba-music-web-site.appspot.com/img/Jamtaba_2_1_0_jamming.png
At moment the video stream is sended using a "unknown" fourCC. Ninjam clients are accepting the OGGv fourCC only. JamTaba video streams are discarded (because I'm using JTBv as fourCC) when received by another ninjam clients. Off course JamTaba can detect the JTBv fourCC and render the video stream.
Using this fourCC trick musicians can use video in JamTaba without being incompatible with another ninjam clients. The trick works, but the audio and video are separated streams, and in general are not in sync.
My Experiments
First I opened an audio file saved by ReaNinjam in NotePad++. The first 4 bytes are OggS, and after some bytes becomes the vorbis header.
So, I realized ninjam was always using OGG as container, and vorbis audio inside the container. This is very nice because is possible explore the OGG container possibilites.
I started sending a OGV (vorbis audio and theora video) file to ReaNinjam, and got nothing in the other side. No audio or video. I was expecting hear the audio.
I compiled the ninjam winclient to debug the vorbis decoder. When debuging I see the decoder is assuming all streams in the container are vorbis (the theora stream is messing the decoder).
To get the vorbis decoder working (decoding the audio stream but ignoring the theora) I just removed some lines in the decoder code.
I generated an OGV file with the audio stream first, so the second stream (theora) is ignored by vorbis decoder and the audio is rendered without problems. It's working, using this approach is possible send audio + video inside an OGG container.
My proposal
When ninjam client receive an OGG stream containing audio + video no audio is rendered/decoded.
In this commit the problematic piece code in vorbis decoder was removed. Testing in ninjam winclient this solution works. If an OGV file (audio + video) is streamed the ninjam winclient can decode the audio. The video stream is just ignored.
Important: The "solution" is not working if the video stream appears first in the OGG container. Is mandatory put the audio stream first in OGG container.
Benefits
Issues
I compiled the ReaNinjam (thanks for share the code 👍 😄 ). The changes in vorbis decoder are not working even after undefine WDL_VORBIS_INTERFACE_ONLY. ReaNinjam is always dicarding audio and video. Maybe ReaNinjam is using another decoder implementation?
I'm proposing a change in a piece of code inside WDL, maybe this will break other codes?