avcodec_fill_audio_frame does not fill

[wangjia184]

This is not an issue of rsmpeg. I am trying to seek some assistance here.

I try to fill AVFrame with samples as below.

let bgm_samples : AVSamples = ...;
assert_eq!(bgm_samples.nb_channels, 1);
assert_eq!(bgm_samples.sample_fmt, AV_SAMPLE_FMT_FLTP);

let mut frame = AVFrame::new();
frame.set_nb_samples(bgm_samples.nb_samples);
frame.set_ch_layout(AVChannelLayout::from_nb_channels(1).into_inner());
frame.set_format(bgm_samples.sample_fmt);
frame.set_sample_rate(22050);

frame
    .alloc_buffer()
    .context("Could not allocate frame buffer")?;

let raw_ptr = frame.into_raw();
unsafe {
    let err = ffi::avcodec_fill_audio_frame(
        raw_ptr.as_ptr(),
        1,
        bgm_samples.sample_fmt,
        bgm_samples.audio_data[0].as_mut().unwrap(),
        bgm_samples.nb_samples
            * ffi::av_get_bytes_per_sample(bgm_samples.sample_fmt),
        0,
    );
    if err < 0 {
        bail!("avcodec_fill_audio_frame failed with {}", err);
    }
}

let frame = unsafe { AVFrame::from_raw(raw_ptr) };

The frame from above code would cause an error in encoder:

[aac @ 0000021A08201AC0] Input contains (near) NaN/+-Inf

If I change the above code as below, to copy samples into an AVAudioFifo and then read samples from AVAudioFifo into AVFrame, it works! It means the bgm_samples contains valid samples. But somehow avcodec_fill_audio_frame does not copy the samples correctly.

Can someone help? thanks in advance.

let mut fifo: AVAudioFifo = AVAudioFifo::new(
    bgm_samples.sample_fmt,
    1,
    bgm_samples.nb_samples,
);

unsafe {
    fifo.write(bgm_samples.audio_data.as_ptr(), bgm_samples.nb_samples)?;
}

// create frame from samples
let mut frame = AVFrame::new();
frame.set_nb_samples(bgm_samples.nb_samples);
frame.set_ch_layout(AVChannelLayout::from_nb_channels(1).into_inner());
frame.set_format(bgm_samples.sample_fmt);
frame.set_sample_rate(22050);

frame
    .alloc_buffer()
    .context("Could not allocate frame buffer")?;

if unsafe { fifo.read(frame.data_mut().as_mut_ptr(), bgm_samples.nb_samples)? }
    < bgm_samples.nb_samples
{
    bail!("Could not read data from FIFO");
}

[ldm0]

Well, actually avcodec_fill_audio_frame or AVAudioFifo are both overkill here. You can do things like this:

https://github.com/larksuite/rsmpeg/blob/a3e43643bf001791dfcaa895cb0154429f4508a3/src/avutil/frame.rs#L241-L254

Note: AVSamples is almost the same as AVImage(They are both created by av_*_fill_arrays methods)

Copy the data, linesize, nb_channels, nb_samples and sample_fmt into your AVFrame instance (make AVFrame borrows the data in AVSamples, be aware of the lifetime though), and everything should be fine.

It would be nice if you could provide a minimal example, therefore reproducing your problem would be much simpler.

[wangjia184]

Thanks @idm0.

It works for mono channel sound.

        let mut frame = AVFrame::new();
        frame.set_nb_samples(bgm_samples.nb_samples);
        frame.set_ch_layout(AVChannelLayout::from_nb_channels(AUDIO_CHANNEL_NUM).into_inner());
        frame.set_format(bgm_samples.sample_fmt);
        frame.set_sample_rate(AUDIO_SAMPLE_RATE);

        frame
            .alloc_buffer()
            .context("Could not allocate frame buffer")?;

        unsafe {
            frame.deref_mut().linesize[0] = bgm_samples.linesize;
            assert_ne!(
                frame.deref_mut().extended_data,
                std::ptr::null_mut::<*mut u8>()
            );

            std::ptr::copy_nonoverlapping(
                bgm_samples.audio_data[0],
                *frame.deref_mut().extended_data,
                bgm_samples.linesize as usize,
            );
        }