Sound problem with a local record

[thierrygayet]

Hi Mathieu,

I am testing a local record using the cefsrc and cefdemux modules. What I observe is a problem out of the sound and I would like to have your opinion on what can happen.

The pipeline I use for a local recording is:

$ export GST_DEBUG = 4
$ export GST_PLUGIN_PATH = $ PWD
$ export GST_DEBUG_DUMP_DOT_DIR = $ PWD
$ gst-launch-1.0 cefsrc url = "https://soundcloud.com/platform/sama" ! cefdemux name = d d.video ! video / x-raw, format = BGRA, framerate = 30/1! tail! videoconvert! videorate max-rate = 3000! videoscale! x264enc tune = speed-preset zerolatency = superfast key-int-max = 60 ref = 2! mux. d.audio_0! audio / x-raw, rate = 48000! tail! audioconvert! audiorate! voaacenc bitrate = 96000! mux. matroskamux name = mux! filesink location = out-sample01.mkv

As you can see, I get an empty file:

$ ls -al sample01.mkv
-rw-rw-r-- 1 tgayet tgayet 0 Oct 25 14:03 sample01.mkv

$ file sample01.mkv
sample01.mkv: empty

$ gst-discoverer-1.0 out-cef.mkv
Analyzing sample01.mkv
Done discovering sample01.mkv
An error was encountered while discovering the file
 Stream contains no data.

What I can see is that I do not get a dot file for the PLAYING state; i just get READY then PAUSE.

However, when I use another source for the sound, namely the gstreamer module audiotestsrc, what I get seems ok with sound this time:

$ export GST_DEBUG = 4
$ export GST_PLUGIN_PATH = $ PWD
$ export GST_DEBUG_DUMP_DOT_DIR = $ PWD
$ gst-launch-1.0 cefsrc url = "https://soundcloud.com/platform/sama" ! cefdemux name = d d.video ! video/x-raw,format=BGRA,framerate=30/1 ! queue ! videoconvert ! videorate max-rate=3000 ! videoscale ! x264enc tune = speed-preset zerolatency = superfast key-int-max = 60 ref = 2 ! mux. audiotestsrc ! audio/x-raw ! queue ! audioconvert ! audiorate ! voaacenc bitrate = 96000 ! mux. matroskamux name = mux ! filesink location = sample02.mkv

$ ls -al sample02.mkv
-rw-rw-r-- 1 tgayet tgayet 1194840 Oct 25 14:08 sample02.mkv

$ file sample02.mkv
sample02.mkv: Matroska data

$ gst-discoverer-1.0 sample02.mkv
Analyzing sample02.mkv
Done discovering sample02.mkv

Topology:
  container: Matroska
    audio: MPEG-4 AAC
    video: H.264 (High 4: 4: 4 Profile)

Properties:
  Duration: 0: 00: 04.566000001
  Seekable: yes
  Live: no
  tags:
      container format: Matroska
      audio codec: MPEG-4 AAC audio
      language code: in
      video codec: H264

Do you have an idea what could happen? Any idea?

Thank you in advance for your return.

Thierry

[michael-riha]

@thierrygayet just saw your issue and I wanted to ask if you already have an idea or answer?

I used this very nice plugin for video only in python and docker and it works like a charm. Today I wanted to add audio, so I used the cefdemux and nothing works anymore, even if I use the demux for just video.

As audio_%u is a sometimes-pad I even added a SIGNAL callback in python

cefdemux.connect("pad-added", on_incoming_audio_pad)

but this is function on_incoming_audio_pad is never called.

@MathieuDuponchelle any idea? CLI like @thierrygayet shown does not work either.

Thank you in advance, for this nice work and would appreciate if you have an answer as well.

[thierryg]

Hi,

The problem with a webrtc source is that the audio may come late compared to the video. One of the solutions to get around this problem was to add a fake audio with zero volume and to mix it with the audio from the webrtc :

gst-launch-1.0 rtspclientsink stream-name='stream_@channelid@' name=r user-agent='cef-player/1.0' location=@rtsp_url@ cefsrc url='' ! queue ! cefdemux name=t t. ! video/x-raw ! videoconvert ! x264enc threads=4 tune=zerolatency speed-preset=ultrafast ! video/x-h264 ! queue ! r. audiotestsrc volume=0.0 is-live=true ! queue ! audiomixer name=m ! audioconvert ! voaacenc bitrate=128000 ! aacparse ! queue ! r. t. ! audio/x-raw ! queue ! m."

Hope it will help!

BR Thierry

[michael-riha]

@thierryg okay, I see. Very hacky but I will try it, thank you so far!

@MathieuDuponchelle what would be the common way to handle this challenge? The cefmuxer has this sometimes-pad and mp4mux needs to request_pad before getting into PLAYING- OR PAUSED-state?

[MathieuDuponchelle]

@michael-riha using audiomixer is an appropriate workaround