Memory leak on mux_* functions

[GuillaumeOuint]

Describe the bug Memory leak on mux_* functions. Without drop_audio and mux_audio, it result in normal memory behavior

To Reproduce

radio = input.http("icecast stream")
default = blank()
default = drop_audio(default)
default = mux_audio(default,audio=mksafe(radio))
encoder = %ffmpeg(format="flv",%audio(codec="aac", b="128k", samplerate=44100, q=4),%video(codec="libx264",b="6600k", threads=4, "x264-params"="scenecut=0:open_gop=0:min-keyint=150:keyint=150", preset="ultrafast"))
output.url(url="rtmp://rtmp_server/live/stream",encoder,mksafe(default))

Expected behavior No memory leak

Version details

• OS: Ubuntu
• Version 2.1.0+git@72afe684 (docker image main tag, same behavior on 2.0.0)

Install method Docker

[toots]

Thanks for these timely report. I'll look at it very shortly.

[toots]

I can reproduce with this script:

radio = noise()
default = (blank():source(audio=none,video=yuva420p,midi=none))
default = mux_audio(default,audio=radio)
output.dummy(fallible=true, default)

[toots]

Well that was a tricky one wow! Ended up chasing a bug in the OCaml compiler! https://github.com/ocaml/ocaml/pull/10788 has all the details. Meanwhile, https://github.com/savonet/ocaml-mm/releases/tag/v0.7.2 has a workaround in place for this issue.

[copy]

@toots I think this should be reopened. I'm running into a very similar issue to the original, reproducible as follows:

stream = input.srt(port=8000)
fb = mux_audio(audio=blank(), single("fallback.png"))
stream = fallback(track_sensitive=false, [stream, fb])
output.sdl(fallible=true, stream)

Once the srt streams starts (using obs to stream to srt://localhost:8000), it leaks about 80MB per second, even with the fixed mm version (and liquidsoap 2.0.0):

% opam list --installed |grep mm
mm                    0.7.2       The mm library contains high-level to create and manipulate multimedia streams (audio, video, MIDI)

[copy]

Interestingly, it also seems to happen with two static images:

fb = mux_audio(audio=blank(), single("fallback.png"))
fb2 = mux_audio(audio=blank(), single("fallback2.png"))
stream = fallback([fb, fb2])
output.sdl(fallible=true, stream)

And the logs indicate that there is some catching up going on:

2021/11/30 19:34:34 >>> LOG START
2021/11/30 19:34:33 [main:3] Liquidsoap 2.0.0
2021/11/30 19:34:33 [main:3] Using: bytes=[distributed with OCaml 4.02 or above] pcre=7.5.0 sedlex=2.4 menhirLib=20211125 dtools=0.4.4 duppy=0.9.2 mm=0.7.2 dynlink=[distributed with Ocaml] ffmpeg-avutil=1.0.1 ffmpeg-avcodec=1.0.1 ffmpeg-avdevice=1.0.1 ffmpeg-av=1.0.1 ffmpeg-avfilter=1.0.1 ffmpeg-swresample=1.0.1 ffmpeg-swscale=1.0.1 alsa=0.3.0 camomile=1.0.2 pulseaudio=0.1.4 tsdl=v0.9.8 tsdl-image=0.3.0 srt.constants=0.2.0 srt.types=0.2.0 srt.stubs=0.2.0 srt.stubs.locked=0.2.0 srt=0.2.0
2021/11/30 19:34:33 [dynamic.loader:3] Could not find dynamic module for lame encoder.
2021/11/30 19:34:33 [dynamic.loader:3] Could not find dynamic module for fdkaac encoder.
2021/11/30 19:34:33 [clock:3] Using builtin (low-precision) implementation for latency control
2021/11/30 19:34:34 [frame:3] Using 44100Hz audio, 25Hz video, 44100Hz main.
2021/11/30 19:34:34 [frame:3] Video frame size set to: 1280x720
2021/11/30 19:34:34 [frame:3] Frame size must be a multiple of 1764 ticks = 1764 audio samples = 1 video samples.
2021/11/30 19:34:34 [frame:3] Targeting 'frame.duration': 0.04s = 1764 audio samples = 1764 ticks.
2021/11/30 19:34:34 [frame:3] Frames last 0.04s = 1764 audio samples = 1 video samples = 1764 ticks.
2021/11/30 19:34:34 [sandbox:3] Sandboxing disabled
2021/11/30 19:34:34 [video.converter:3] Using preferred video converter: ffmpeg.
2021/11/30 19:34:34 [audio.converter:3] Using samplerate converter: ffmpeg.
2021/11/30 19:34:34 [single_0:3] fallback.png is static, resolving once for all...
2021/11/30 19:34:34 [decoder:3] Method "SDL/IMAGE" accepted "fallback.png".
2021/11/30 19:34:34 [decoder:3] Method "SDL/IMAGE" accepted "fallback.png".
[swscaler @ 0x7fdf68457900] Warning: data is not aligned! This can lead to a speed loss
2021/11/30 19:34:34 [fallback.png:3] Prepared "fallback.png" (RID 0).
2021/11/30 19:34:34 [single_1:3] fallback2.png is static, resolving once for all...
2021/11/30 19:34:34 [decoder:3] Method "SDL/IMAGE" accepted "fallback2.png".
[swscaler @ 0x7fdf68457900] Warning: dstStride is not aligned!
         ->cannot do aligned memory accesses anymore
2021/11/30 19:34:34 [decoder:3] Method "SDL/IMAGE" accepted "fallback2.png".
2021/11/30 19:34:34 [fallback2.png:3] Prepared "fallback2.png" (RID 1).
2021/11/30 19:34:34 [clock.main:3] Streaming loop starts in auto-sync mode
2021/11/30 19:34:34 [clock.main:3] Delegating synchronisation to CPU clock
2021/11/30 19:34:34 [switch_0:3] Switch to mux_audio_0.
2021/11/30 19:34:39 [clock.main:2] We must catchup 0.84 seconds!
2021/11/30 19:34:40 [clock.main:2] We must catchup 1.00 seconds!
2021/11/30 19:34:41 [clock.main:2] We must catchup 1.15 seconds!
2021/11/30 19:34:42 [clock.main:2] We must catchup 1.30 seconds!
2021/11/30 19:34:43 [clock.main:2] We must catchup 1.45 seconds!
2021/11/30 19:34:44 [clock.main:2] We must catchup 1.61 seconds!
2021/11/30 19:34:45 [clock.main:2] We must catchup 1.74 seconds!
2021/11/30 19:34:46 [clock.main:2] We must catchup 1.86 seconds!
2021/11/30 19:34:47 [clock.main:2] We must catchup 1.99 seconds!
2021/11/30 19:34:48 [clock.main:2] We must catchup 2.11 seconds!
2021/11/30 19:34:49 [clock.main:2] We must catchup 2.24 seconds!
2021/11/30 19:34:50 [clock.main:2] We must catchup 2.37 seconds!

[toots]

Thanks. Reopening... 😭

[toots]

Does it also occur without the sdl output?

[copy]

@toots Yes, it also happens with output.url.

[toots]

Ok thks.

[toots]

Thanks, I can confirm it with a dummy output:

v = (single("/tmp/fallback.png"):source(audio=none,video=yuva420p,midi=none))
v2 = (single("/tmp/fallback2.png"):source(audio=none,video=yuva420p,midi=none))

fb = mux_audio(audio=blank(), v)
fb2 = mux_audio(audio=blank(), v2)
stream = fallback([fb, fb2])
output.dummy(stream)

At first glance, this looks like a case of bad internal buffering.

[toots]

Confirmed, it's a bad buffering on our end. Will see about fixing it and possibly getting a 2.0.2 started with the fix.

[toots]

This should be fixed now. Feel free to test, either in the main branch or in the v2.0.2-preview branch.

[copy]

Thanks! I can confirm that it's fixed in main.