There is something wrong when converting s16le -> f32le RawAudio

[anzaika]

Hi!

As always I want to say how grateful I am for all the work you're doing with Membrane. It's an absolutely fantastic framework.

I've been pondering with WebRTC for some time and what at first seemed to be a mistake on my side, starts to look like a strange behavior at the very least. Here's the code in question.

alias Membrane.RTC.Engine.Endpoint.WebRTC.TrackReceiver

def handle_pad_added({_, _, track_id} = pad, ctx, state) do
  structure = [
    bin_input(pad)
      |> child(:track_receiver, %TrackReceiver{
        track: state.track,
        initial_target_variant: :high
      })
      |> child(:depayloader, get_depayloader(state.track))
      |> child(:opus_decoder, %Membrane.Opus.Decoder{
        sample_rate: 16_000
      })
      |> child(:resample, %Converter{
        output_stream_format: %RawAudio{channels: 1, sample_format: :f32le, sample_rate: 16_000}
      })
      |> child({:wav_serializer, track_id}, Membrane.WAV.Serializer)
      |> child({:sink, track_id}, %Membrane.File.Sink{
        location: Path.join(state.output_dir_path, "out.wav")
      })
  ]
end

The source track is coming from the browser via WebRTC. Whenever I'm resampling to s16le or s24le or s32le the resulting out.wav is perfectly fine. But resampling to f32le produces a screeching and squeaking mess. ffmpeg sees nothing wrong with the resulting file and I'm not savvy enough with audio processing to figure out the cause.

Do you have any ideas about where this might be coming from?

[DominikWolek]

Hi @anzaika! I've spent some time debugging this issue. This was caused by the bug in WAV.Serializer, not by any of the elements in FFMpeg.SWResample plugin. I've fixed the bug, it's in review right now (to be released soon): https://github.com/membraneframework/membrane_wav_plugin/pull/25

[anzaika]

@DominikWolek wooooow, sounds awesome! Thank you for diving deep and finding the root cause, I really appreciate it.

Membrane team is the best!