voc / voctomix

Full-HD Software Live-Video-Mixer in python
https://c3voc.de/
MIT License
578 stars 107 forks source link

voctomix1: audiostram mapping broken with recent GStreamer version #257

Open danimo opened 4 years ago

danimo commented 4 years ago

Assume a config with sources like this:

[source.cam1]
...
audiostream[0] = 0
audiostream[1] = 1

This worked perfectly with Debian 9 based installations (gstreamer 1.10). Now starting with version 1.13.90, this commit "breaks" Voctomix1, although it aims to correct the change of video and audio caps while a matroska mux is already running. I think this is directly related because the error first introduced in this patch ("Caps changed are not supported by Matroska") can be found in the debug output. This breaks Voctomix 1 on Buster.

Here is the difference in startup output (out.working is the output when audiostream lines are commented out):

--- /tmp/out.working    2019-12-21 02:26:42.330442450 +0200
+++ /tmp/out    2019-12-21 01:16:48.811866929 +0200
@@ -16,16 +16,17 @@
        ../voctocore-config.ini
    DEBUG root: initializing Voctocore
    DEBUG Clock: Obtaining System-Clock
-    INFO Clock: Using System-Clock for all Pipelines: <Gst.SystemClock object at 0x7fb5a77c91f8 (GstSystemClock at 0x24b3130)>
+    INFO Clock: Using System-Clock for all Pipelines: <Gst.SystemClock object at 0x7f9b132af048 (GstSystemClock at 0x184d130)>
     INFO Clock: Starting NetTimeProvider on Port 9998
    DEBUG Voctocore: creating GLib-MainLoop
    DEBUG Voctocore: creating A/V-Pipeline
     INFO Pipeline: Video-Caps configured to: video/x-raw,format=I420,width=1920,height=1080,framerate=25/1,pixel-aspect-ratio=1/1
     INFO Pipeline: Audio-Caps configured to: audio/x-raw,format=S16LE,channels=2,layout=interleaved,rate=48000
     INFO Pipeline: Creating 3 AVSources: ['cam1', 'cam2', 'slides']
-    INFO DecklinkAVSource[cam1]: audiostream_map: {}
-    INFO DecklinkAVSource[cam1]: no audiostream-mapping defined,defaulting to mapping channel 0+1 to first stream
+    INFO DecklinkAVSource[cam1]: audiostream_map: {0: '0', 1: '1'}
     INFO DecklinkAVSource[cam1]: configuring decklink-input to 2 channels
+    INFO DecklinkAVSource[cam1]: mapping decklink input-channel 0 as left and right to output-stream 0
+    INFO DecklinkAVSource[cam1]: mapping decklink input-channel 1 as left and right to output-stream 1
    DEBUG DecklinkAVSource[cam1]: Launching Source-Pipeline:

             decklinkvideosrc
@@ -43,9 +44,21 @@

                     device-number=1
                     connection=embedded
-                    name=aout
+                    ! deinterleave name=aout

-                    aout. !
+                        interleave name=i0
+                        aout.src_0 ! tee name=t0
+
+                        t0. ! queue ! i0.sink_0
+                        t0. ! queue ! i0.sink_1
+                    
+                        interleave name=i1
+                        aout.src_1 ! tee name=t1
+
+                        t1. ! queue ! i1.sink_0
+                        t1. ! queue ! i1.sink_1
+                    
+                    i0. !
                     audio/x-raw,format=S16LE,channels=2,layout=interleaved,rate=48000 !
                     queue !
                     tee name=atee_stream0
@@ -59,6 +72,20 @@
                         atee_stream0. ! queue ! interaudiosink
                             channel=audio_cam1_mirror_stream0

+                    i1. !
+                    audio/x-raw,format=S16LE,channels=2,layout=interleaved,rate=48000 !
+                    queue !
+                    tee name=atee_stream1
+                
+                        atee_stream1. ! queue ! interaudiosink
+                            channel=audio_cam1_mixer_stream1
+                    
+                        atee_stream1. ! queue ! interaudiosink
+                            channel=audio_cam1_preview_stream1
+                    
+                        atee_stream1. ! queue ! interaudiosink
+                            channel=audio_cam1_mirror_stream1
+                    
                 vout. !
                 video/x-raw,format=I420,width=1920,height=1080,framerate=25/1,pixel-aspect-ratio=1/1 !
                 queue !
@@ -766,3 +793,27 @@
    DEBUG ControlServer: Setting GObject io-watch on Socket
    DEBUG root: running Voctocore
     INFO Voctocore: running GLib-MainLoop
+   DEBUG AVRawOutput[cam1_mirror]: Received Error-Signal on Output-Pipeline
+   DEBUG AVRawOutput[cam1_mirror]: Error-Details: #10: matroska-mux.c(1810): gst_matroska_mux_audio_pad_setcaps (): /GstPipeline:pipeline1/GstMatroskaMux:mux:
+Caps changed are not supported by Matroska
+   DEBUG AVRawOutput[cam1_mirror]: Received Error-Signal on Output-Pipeline
+   DEBUG AVRawOutput[cam1_mirror]: Error-Details: #10: matroska-mux.c(1810): gst_matroska_mux_audio_pad_setcaps (): /GstPipeline:pipeline1/GstMatroskaMux:mux:
+Caps changed are not supported by Matroska
+   DEBUG AVRawOutput[cam1_mirror]: Received Error-Signal on Output-Pipeline
+   DEBUG AVRawOutput[cam1_mirror]: Error-Details: #1: gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline1/GstInterAudioSrc:interaudiosrc0:
+streaming stopped, reason not-negotiated (-4)
+   DEBUG AVRawOutput[cam1_mirror]: Received Error-Signal on Output-Pipeline
+   DEBUG AVRawOutput[cam1_mirror]: Error-Details: #1: gstqueue.c(988): gst_queue_handle_sink_event (): /GstPipeline:pipeline1/GstQueue:queue17:
+streaming stopped, reason not-negotiated (-4)
+   DEBUG AVPreviewOutput[cam1_preview]: Received Error-Signal on Output-Pipeline
+   DEBUG AVPreviewOutput[cam1_preview]: Error-Details: #10: matroska-mux.c(1810): gst_matroska_mux_audio_pad_setcaps (): /GstPipeline:pipeline2/GstMatroskaMux:mux:
+Caps changed are not supported by Matroska
+   DEBUG AVPreviewOutput[cam1_preview]: Received Error-Signal on Output-Pipeline
+   DEBUG AVPreviewOutput[cam1_preview]: Error-Details: #10: matroska-mux.c(1810): gst_matroska_mux_audio_pad_setcaps (): /GstPipeline:pipeline2/GstMatroskaMux:mux:
+Caps changed are not supported by Matroska
+   DEBUG AVPreviewOutput[cam1_preview]: Received Error-Signal on Output-Pipeline
+   DEBUG AVPreviewOutput[cam1_preview]: Error-Details: #1: gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline2/GstInterAudioSrc:interaudiosrc2:
+streaming stopped, reason not-negotiated (-4)
+   DEBUG AVPreviewOutput[cam1_preview]: Received Error-Signal on Output-Pipeline
+   DEBUG AVPreviewOutput[cam1_preview]: Error-Details: #1: gstqueue.c(988): gst_queue_handle_sink_event (): /GstPipeline:pipeline2/GstQueue:queue20:
+streaming stopped, reason not-negotiated (-4)

Why do the caps change after the demuxer is established? Also, the audiocaps are always forced before passing the audio to the matroska muxer.

danimo commented 4 years ago

Debug logs: https://gist.github.com/danimo/29b560be0cee46c0811ee026a74b8393