search

teltek / Galicaster

The Galicaster Project is an open initiative to provide flexible, state-of-the-art solutions for recording educational multimedia contents like lectures and conferences
galicaster.teltek.es
Other
35 stars 31 forks source link

NDI multi stream stuttering #617

Open andiempettJISC opened 5 years ago

andiempettJISC commented 5 years ago

when trying to use multiple NDI sources in galicaster. the ndivideosrc and ndiaudiosrc will stutter. stuttering will happen in both preview and when recording to filesink

if you play the ndi sources from gst-launch command line at the same time, they work fine with no stuttering (also recording to filesink works OK as well)

its hard to pin down the cause of the stuttering, but i dont think its compute resource as the machine running this is very powerful.

gst-launch-1.0 --verbose ndivideosrc stream-name="BIRDDOG-xxxxx (HDMI)" ! queue ! videoconvert ! videobox  top=0 bottom=0 ! videorate ! xvimagesink async=true qos=true sync=true force-aspect-ratio=true

gst-launch-1.0 --verbose ndiaudiosrc stream-name="BIRDDOG-xxxxx (comms)" ! queue ! autoaudiosink

example profile:

[data]
name = NDI

[track1]
name = NDI
device = ndi
location = BIRDDOG-xxxxx (HDMI)
file = presentation.avi
flavor = presentation
audio = False

[track3]
name = NDI_Audio
device = ndi_audio
location = BIRDDOG-xxxxx (comms)
file = presenter.mp3
flavor = presenter
player = True
vumeter = True
smarquard commented 5 years ago

The difference in the Galicaster gstreamer pipelines is that they include a tee for both preview and filesink recording, so it's worth reproducing that pipeline with gst-launch.

We had a datapath problem once that only showed up with a tee pipeline but not a simpler pipeline.

andiempettJISC commented 5 years ago

@smarquard raises a good point, but sadly with more complex pipelines on the command line i still cant replicate the issue.

gst-launch-1.0 --verbose ndiaudiosrc stream-name="BIRDDOG-xxxxx (comms)" do-timestamp=true ! queue ! audiorate ! decodebin async-handling=true ! tee name=tee-aud \
tee-aud. ! queue ! audioconvert ! wavenc ! filesink location=testndi.wav async=false \
tee-aud. ! queue ! autoaudiosink

gst-launch-1.0 --verbose ndivideosrc stream-name="BIRDDOG-xxxxx (HDMI)" ! queue ! videoconvert ! videobox  top=0 bottom=0 ! videorate ! tee name=gc-ndi-tee ! queue ! xvimagesink async=false \
gc-ndi-tee. ! queue ! valve drop=false ! videoconvert ! queue ! \
x264enc pass=5 quantizer=22 speed-preset=4 ! queue ! avimux ! \
queue ! filesink location=testndi.avi async=false
Alfro commented 5 years ago

Hi @androidwiltron

A more accurate way to test it would be to record both sources through the same pipeline. That may reproduce the issue better, since I believe Galicaster has all sources on the same process, as opposed to launching two different gst-launch-1.0 commands.

andiempettJISC commented 5 years ago

@Alfro good suggestion really i need to make a minimal python script to do this

do you think using videomixer gstreamer element with gst-launch would be sufficient?

Alfro commented 5 years ago

I meant something like this:

gst-launch-1.0 --verbose ndiaudiosrc stream-name="BIRDDOG-xxxxx (comms)" do-timestamp=true ! queue ! audiorate ! decodebin async-handling=true ! tee name=tee-aud \ tee-aud. ! queue ! audioconvert ! wavenc ! filesink location=testndi.wav async=false \ tee-aud. ! queue ! autoaudiosink ndivideosrc stream-name="BIRDDOG-xxxxx (HDMI)" ! queue ! videoconvert ! videobox top=0 bottom=0 ! videorate ! tee name=gc-ndi-tee ! queue ! xvimagesink async=false \ gc-ndi-tee. ! queue ! valve drop=false ! videoconvert ! queue ! \ x264enc pass=5 quantizer=22 speed-preset=4 ! queue ! avimux ! \ queue ! filesink location=testndi.avi async=false

Let me know if it reproduces like this!

andiempettJISC commented 4 years ago

OK @Alfro your launch command replicates the problem. the audio stutters, the video is OK