search

tinyzimmer / go-gst

Gstreamer bindings and utilities for golang
GNU Lesser General Public License v2.1
130 stars 37 forks source link

Add support for adding audio meta data to a buffer #42

Open danjenkins opened 2 years ago

danjenkins commented 2 years ago

If you have a buffer with non interleaved audio inside it, you need to add meta data to the buffer to say it's not interleaved and how to deal with it...

gst_buffer_add_audio_meta

https://gstreamer.freedesktop.org/documentation/audio/gstaudiometa.html?gi-language=c#GstAudioMeta

danjenkins commented 2 years ago

@tinyzimmer I made a start on this but the whole gsize pointer logic is confusing me.... any chance you can tell me what I've done wrong?

Branch diff here: https://github.com/tinyzimmer/go-gst/compare/main...broadcastervc:go-gst:audio-metadata-non-interlaced?expand=1

These are the errors coming back from gstreamer

(<unknown>:33979): GStreamer-Audio-CRITICAL **: 22:15:37.327: GstAudioMeta properties would cause out-of-bounds memory access on the buffer: max_offset 4535117456, samples 960, bps 4, buffer size 7680

and

(<unknown>:33979): GStreamer-Audio-CRITICAL **: 22:15:37.348: GstAudioMeta properties would cause channel memory areas to overlap! offsets: 8 (0), 214 (1) with plane size 3840

In my case I'm calling the new function with gstAudio.BufferAddAudioMeta(b, info, int64(samples), []int{0, 3840}) b is a gst buffer created with b := gst.NewBufferFromBytes(data) and info is created from the caps I'm passing in... info, _ := gstAudio.InfoFromCaps(caps) which are "audio/x-raw, format=F32LE, channels=%d, rate=%d, layout=non-interleaved" (obviously with values...)

Obviously once I know it works I'll make the PR :)

Oh and this is the test data for gst_buffer_add_audio_meta... https://github.com/GStreamer/gstreamer/blob/main/subprojects/gst-plugins-base/tests/check/libs/audio.c#L1368-L1401 for an idea of what it's expecting...

danjenkins commented 2 years ago

Most recent commit I removed the offset parameter and passed in NULL because my audio samples are tightly packed. And it seems to work.... so just need to sort out the passing in off the offsets (and allow it to be null if someone chooses)