search

savonet / liquidsoap

Liquidsoap is a statically typed scripting general-purpose language with dedicated operators and backend for all thing media, streaming, file generation, automation, HTTP backend and more.
http://liquidsoap.info
GNU General Public License v2.0
1.4k stars 130 forks source link

cant get stereo.rms, rms, stereo.peak and peak working anymore #4084

Closed liquidpeter closed 2 months ago

liquidpeter commented 2 months ago

Description

I wanted to upgrade my scripts from Liquidsoap 2.0.2 to 2.2.4-1+dev (Ubuntu 24.04 LTS). Unfortunately the rms and peak operations do not work as expected any more, for both mono and stereo channels.

Steps to reproduce

Install ubuntu 24.04 with liquidsoap from packages Remove comments on thread.run to show one of the 4 issues (rms mono, rms stereo, peak mono, peak stereo).

Test script

radio = sine();
radio = stereo(radio)

# rms.stereo: uncaught error: index out of bounds
def test_rms_stereo()
 r = rms.stereo(duration=1., radio)
 output.dummy(r)
 res = r.rms()
end
#thread.run(every=1., test_rms_stereo)

# rms(): -nan for sine()
def test_rms_mono()
 rl = stereo.left(radio)
 rr = stereo.right(radio)
 rmsl = rms(duration=1., rl)
 rmsr = rms(duration=1., rr)
 output.dummy(rmsl)
 output.dummy(rmsr)
 resl= rmsl.rms();
 resr= rmsr.rms();
 print ("#{resl} #{resr}\n")
end
#thread.run(every=1., test_rms_mono)

# peak.stereo: uncaught error: uncaught error: index out of bounds
def test_peak_stereo()
 r = peak.stereo(duration=1., radio)
 output.dummy(r)
 res = r.peak()
end
#thread.run(every=1., test_peak_stereo)

# peak: -nan for sine()
def test_peak_mono()
 rl = stereo.left(radio)
 rr = stereo.right(radio)
 pl = peak(duration=1., rl)
 pr = peak(duration=1., rr)
 output.dummy(pl)
 output.dummy(pr)
 resl= pl.peak()
 resr= pr.peak()
 print ("#{resl} #{resr}\n")
end
#thread.run(every=1., test_peak_mono)

output(radio)

Results:

detailed log for index out of bounds

2024/08/10 13:27:16 [runtime:1] Raised by primitive operation at Window_op.rms.stereo in file "src/core/operators/window_op.ml", line 135, characters 52-61
2024/08/10 13:27:16 [runtime:1] Called from Liquidsoap_lang__Evaluation.apply.f in file "src/lang/evaluation.ml", line 174, characters 8-12
2024/08/10 13:27:16 [runtime:1] Called from Liquidsoap_lang__Evaluation.eval_term in file "src/lang/evaluation.ml", line 347, characters 10-43
2024/08/10 13:27:16 [runtime:1] Called from Liquidsoap_lang__Evaluation.eval in file "src/lang/evaluation.ml", line 359, characters 10-38
2024/08/10 13:27:16 [runtime:1] Called from Liquidsoap_lang__Evaluation.eval_base_term in file "src/lang/evaluation.ml", line 278, characters 16-38
2024/08/10 13:27:16 [runtime:1] Called from Liquidsoap_lang__Evaluation.eval_term in file "src/lang/evaluation.ml", line 347, characters 10-43
2024/08/10 13:27:16 [runtime:1] Called from Liquidsoap_lang__Evaluation.eval in file "src/lang/evaluation.ml", line 359, characters 10-38
2024/08/10 13:27:16 [runtime:1] Called from Liquidsoap_lang__Evaluation.eval_term in file "src/lang/evaluation.ml", line 347, characters 10-43
2024/08/10 13:27:16 [runtime:1] Called from Liquidsoap_lang__Evaluation.eval in file "src/lang/evaluation.ml", line 359, characters 10-38
2024/08/10 13:27:16 [runtime:1] Called from Liquidsoap_lang__Evaluation.eval_term in file "src/lang/evaluation.ml", line 347, characters 10-43
2024/08/10 13:27:16 [runtime:1] Called from Liquidsoap_lang__Evaluation.eval in file "src/lang/evaluation.ml", line 359, characters 10-38
2024/08/10 13:27:16 [runtime:1] Called from Liquidsoap_lang__Evaluation.apply.f in file "src/lang/evaluation.ml", line 174, characters 8-12
2024/08/10 13:27:16 [runtime:1] Called from Liquidsoap_lang__Evaluation.eval_term in file "src/lang/evaluation.ml", line 347, characters 10-43
2024/08/10 13:27:16 [runtime:1] Called from Liquidsoap_lang__Evaluation.eval in file "src/lang/evaluation.ml", line 359, characters 10-38
2024/08/10 13:27:16 [runtime:1] Called from Liquidsoap_lang__Evaluation.eval_base_term.ans.(fun) in file "src/lang/evaluation.ml", line 337, characters 46-68
2024/08/10 13:27:16 [runtime:1] Called from Stdlib__List.map in file "list.ml", line 92, characters 20-23
2024/08/10 13:27:16 [runtime:1] Called from Liquidsoap_lang__Evaluation.eval_base_term.ans in file "src/lang/evaluation.ml", line 337, characters 18-72
2024/08/10 13:27:16 [runtime:1] Called from Liquidsoap_lang__Evaluation.eval_term in file "src/lang/evaluation.ml", line 347, characters 10-43
2024/08/10 13:27:16 [runtime:1] Called from Liquidsoap_lang__Evaluation.eval in file "src/lang/evaluation.ml", line 359, characters 10-38
2024/08/10 13:27:16 [runtime:1] Called from Liquidsoap_lang__Evaluation.eval_base_term.ans.(fun) in file "src/lang/evaluation.ml", line 337, characters 46-68
2024/08/10 13:27:16 [runtime:1] Called from Stdlib__List.map in file "list.ml", line 92, characters 20-23
2024/08/10 13:27:16 [runtime:1] Called from Liquidsoap_lang__Evaluation.eval_base_term.ans in file "src/lang/evaluation.ml", line 337, characters 18-72
2024/08/10 13:27:16 [runtime:1] Called from Liquidsoap_lang__Evaluation.eval_term in file "src/lang/evaluation.ml", line 347, characters 10-43
2024/08/10 13:27:16 [runtime:1] Called from Liquidsoap_lang__Evaluation.eval in file "src/lang/evaluation.ml", line 359, characters 10-38
2024/08/10 13:27:16 [runtime:1] Called from Liquidsoap_lang__Evaluation.eval_base_term in file "src/lang/evaluation.ml", line 332, characters 15-39
2024/08/10 13:27:16 [runtime:1] Called from Liquidsoap_lang__Evaluation.eval_term in file "src/lang/evaluation.ml", line 347, characters 10-43
2024/08/10 13:27:16 [runtime:1] Called from Liquidsoap_lang__Evaluation.eval in file "src/lang/evaluation.ml", line 359, characters 10-38
2024/08/10 13:27:16 [runtime:1] Called from Liquidsoap_lang__Evaluation.apply.f in file "src/lang/evaluation.ml", line 174, characters 8-12
2024/08/10 13:27:16 [runtime:1] Called from Stdlib__Fun.protect in file "fun.ml", line 33, characters 8-15
2024/08/10 13:27:16 [runtime:1] Re-raised at Stdlib__Fun.protect in file "fun.ml", line 38, characters 6-52
2024/08/10 13:27:16 [runtime:1] Called from Builtins_thread.(fun).f in file "src/core/builtins/builtins_thread.ml", line 60, characters 26-43
2024/08/10 13:27:16 [runtime:1] Re-raised at Liquidsoap_lang__Lang_core.raise_as_runtime in file "src/lang/lang_core.ml", line 416, characters 8-146
2024/08/10 13:27:16 [runtime:1] Called from Builtins_thread.(fun).f in file "src/core/builtins/builtins_thread.ml", line 63, characters 10-52
2024/08/10 13:27:16 [runtime:1] Called from Duppy.Task.t_of_task.(fun) in file "src/duppy.ml", line 153, characters 50-66
2024/08/10 13:27:16 [runtime:1] Called from Duppy.exec in file "src/duppy.ml", line 281, characters 16-23

Expected behavior

The defs rms, stereo.rms, peak and stereo.peak results should return valid values.

Liquidsoap version

Liquidsoap 2.2.4-1+dev
Copyright (c) 2003-2023 Savonet team
Liquidsoap is open-source software, released under GNU General Public License.
See <http://liquidsoap.info> for more information.

Liquidsoap build config

no more csLADSPA plugins

 * Liquidsoap version  : 2.2.4-1+dev

 * Compilation options
   - Release build       : false
   - Git SHA             : (none)
   - OCaml version       : 4.14.1
   - OS type             : Unix
   - Libs versions       : alsa=0.3.0 angstrom=0.16.0 ao=0.2.4 bigarray=[distributed with Ocaml] bigstringaf=0.9.1 bjack=0.1.6 bytes=[distributed with OCaml 4.02 or above] camlimages.all_formats=4.2.6 camlimages.core=5.0.4 camlimages.exif=5.0.4 camlimages.gif=5.0.4 camlimages.jpeg=5.0.4 camlimages.png=5.0.4 camlimages.tiff=5.0.4 camlimages.xpm=5.0.4 camlp-streams camomile.lib=2.0 cry=1.0.3 curl=0.9.2 dssi=0.1.5 dtools=0.4.5 dune-build-info=3.14.0 dune-private-libs.dune-section=3.14.0 dune-site=3.14.0 dune-site.private=3.14.0 duppy=0.9.4 faad=0.5.1 ffmpeg-av=1.1.11 ffmpeg-avcodec=1.1.11 ffmpeg-avdevice=1.1.11 ffmpeg-avfilter=1.1.11 ffmpeg-avutil=1.1.11 ffmpeg-swresample=1.1.11 ffmpeg-swscale=1.1.11 fileutils flac=0.5.0 flac.decoder=0.5.0 flac.ogg=0.5.0 frei0r=0.1.2 gd=0.5.1 gen=1.1 graphics=5.1.2 inotify=2.0-62-g5e58536 ladspa=0.2.2 lame=0.3.7 liquidsoap-lang=2.2.4-1 liquidsoap-lang.console=2.2.4-1 liquidsoap_alsa liquidsoap_ao liquidsoap_bjack liquidsoap_builtins liquidsoap_camlimages liquidsoap_core liquidsoap_dssi liquidsoap_faad liquidsoap_ffmpeg liquidsoap_flac liquidsoap_frei0r liquidsoap_gd liquidsoap_graphics liquidsoap_ladspa liquidsoap_lame liquidsoap_lo liquidsoap_mad liquidsoap_ogg liquidsoap_ogg_flac liquidsoap_optionals liquidsoap_opus liquidsoap_oss liquidsoap_portaudio liquidsoap_pulseaudio liquidsoap_runtime liquidsoap_samplerate liquidsoap_shine liquidsoap_soundtouch liquidsoap_speex liquidsoap_ssl liquidsoap_taglib liquidsoap_theora liquidsoap_vorbis liquidsoap_xmlplaylist lo=0.2.0 mad=0.5.2 magic-mime=1.3.1 menhirLib=20231231 metadata=0.3.0 mm=0.8.5 mm.audio=0.8.5 mm.base=0.8.5 mm.image=0.8.5 mm.midi=0.8.5 mm.video=0.8.5 ogg=0.7.4 ogg.decoder=0.7.4 opus=0.2.2 opus.decoder=0.2.2 pcre=7.5.0 portaudio=0.2.3 pulseaudio=0.1.5 re=1.11.0 samplerate=0.1.6 sedlex=3.2 seq=[distributed with OCaml 4.07 or above] shine=0.2.3 soundtouch=0.1.9 speex=0.4.1 speex.decoder=0.4.1 ssl=0.7.0 stdlib-shims=[distributed with OCaml 4.07 or above] str=[distributed with Ocaml] stringext=1.6.0 syslog=2.0.2 taglib=0.3.10 theora=0.4.0 theora.decoder=0.4.0 threads=[distributed with Ocaml] threads.posix=[internal] unix=[distributed with Ocaml] uri=4.4.0 vorbis=0.8.0 vorbis.decoder=0.8.0 xmlm=1.4.0 xmlplaylist=0.1.5
   - architecture        : amd64
   - host                : x86_64-pc-linux-gnu
   - target              : x86_64-pc-linux-gnu
   - system              : linux
   - ocamlopt_cflags     : -O2 -fno-strict-aliasing -fwrapv -pthread -fPIC -g -O2 -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection
   - native_c_compiler   : x86_64-linux-gnu-gcc -O2 -fno-strict-aliasing -fwrapv -pthread -fPIC -g -O2 -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection -D_FILE_OFFSET_BITS=64 -Wdate-time -D_FORTIFY_SOURCE=2
   - native_c_libraries  : -lm

 * Configured paths
   - mode              : default
   - standard library  : (set by dune-site)
   - scripted binaries : (set by dune-site)
   - rundir            : (set by dune-site)
   - logdir            : (set by dune-site)
   - camomile files    : (set by dune-site)

 * Supported input formats
   - MP3               : yes
   - AAC               : yes
   - Ffmpeg            : yes
   - Flac (native)     : yes
   - Flac (ogg)        : yes
   - Opus              : yes
   - Speex             : yes
   - Theora            : yes
   - Vorbis            : yes

 * Supported output formats
   - FDK-AAC           : no (requires fdkaac)
   - Ffmpeg            : yes
   - MP3               : yes
   - MP3 (fixed-point) : yes
   - Flac (native)     : yes
   - Flac (ogg)        : yes
   - Opus              : yes
   - Speex             : yes
   - Theora            : yes
   - Vorbis            : yes

 * Tags
   - Taglib (ID3 tags) : yes
   - Vorbis            : yes

 * Input / output
   - ALSA              : yes
   - AO                : yes
   - FFmpeg            : yes
   - GStreamer         : no (requires gstreamer)
   - JACK              : yes
   - OSS               : yes
   - Portaudio         : yes
   - Pulseaudio        : yes
   - SRT               : no (requires srt)

 * Audio manipulation
   - FFmpeg            : yes
   - LADSPA            : yes
   - Lilv              : no (requires lilv)
   - Samplerate        : yes
   - SoundTouch        : yes
   - StereoTool        : no (requires ctypes-foreign)

 * Video manipulation
   - camlimages        : yes
   - FFmpeg            : yes
   - frei0r            : yes
   - ImageLib          : no (requires imagelib)
   - SDL               : no (requires tsdl-image & tsdl-ttf)

 * MIDI manipulation
   - DSSI              : yes

 * Visualization
   - GD                : yes
   - Graphics          : yes
   - SDL               : no (requires tsdl-image & tsdl-ttf)

 * Additional libraries
   - FFmpeg filters    : yes
   - FFmpeg devices    : yes
   - inotify           : yes
   - irc               : no (requires irc-client-unix)
   - jemalloc          : no (requires jemalloc)
   - lastfm            : no (requires lastfm)
   - lo                : yes
   - memtrace          : no (requires memtrace)
   - mem_usage         : no (requires mem_usage)
   - osc               : no (requires osc-unix)
   - ssl               : yes
   - tls               : no (requires tls-liquidsoap)
   - posix-time2       : no (requires posix)
   - windows service   : no (requires winsvc)
   - YAML support      : no (requires yaml)
   - XML playlists     : yes

 * Monitoring
   - Prometheus        : no (requires prometheus)

Installation method

From distribution packages

Additional Info

As I prefer to use deb packages, it would be very helpful to get a fix / backport? package for Ubuntu 24.04 LTS (noble). Thanks in advance and best regards!

vitoyucepi commented 2 months ago

Hi @liquidpeter,

I have very little experience with rms, but your script seems a bit odd to me. Here's how I would refactor it:

radio = sine();

def source_rms(s)
  r = rms.stereo(s, duration=1.)
  rl = rms(stereo.left(radio), duration=1.)
  rr = rms(stereo.right(radio), duration=1.)
  output.dummy(r)
  output.dummy(rl)
  output.dummy(rr)

  {
    rms = r.rms,
    rms_left = rl.rms,
    rms_right = rr.rms
  }
end

r = source_rms(radio)
thread.run({log("#{r.rms()} #{r.rms_left()} #{r.rms_right()}")}, every=1.)

output.dummy(radio)
Alternative If the first one looks too complicated, here's another one. ```ruby radio = sine(); r = rms.stereo(radio, duration=1.) rl = rms(stereo.left(radio), duration=1.) rr = rms(stereo.right(radio), duration=1.) output.dummy(r) output.dummy(rl) output.dummy(rr) def log_rms() log("#{r.rms()} #{rl.rms()} #{rr.rms()}") end thread.run(log_rms, every=1.) output.dummy(radio) ```
liquidpeter commented 2 months ago

There are two “defs” that basically try to achieve the same result but cause different problems.

gAlleb commented 2 months ago

Thanks for mixing them, but I cannot see how this helps to analyze the issues.

I think what is happening here is that 2.0.2 and 2.2.4 are different. Lots of stuff has changed and syntax that you try to use in 2.2.4 just doesn't work.

Vito has provided you with 2 working options. How this is not helpful? Haven't you tried it out?

  1. Your original script indeed looks a bit off. res = r.rms() is unused and doesn't make sense. Why do you use? what's the point? To solve "def levels" shows the problem with stereo audio when using "rms.stereo". you have to remove this unused var.
def levels()
 r = rms.stereo(duration=1., radio)
 output.dummy(r)
end
thread.run(every=1., levels)

Still this function makes no sense at all. But now it doesn't show an error.

  1. To make your second function work you have to put majority of stuff outside the function:
 rl = stereo.left(radio)
 rr = stereo.right(radio)
 rmsl = rms(duration=1., rl)
 rmsr = rms(duration=1., rr)
 output.dummy(rmsl)
 output.dummy(rmsr)

def levels_wa()
 print ("#{rmsl.rms()} #{rmsr.rms()}");
end
thread.run(every=1., levels_wa)

and use #{rmsl.rms()} #{rmsr.rms()}.

#{resl} #{resr}\n can't just work. because resl = rms(duration=1., rl).rms(); which kinda makes no sense too and will not give you the desired result.

Now you've edited your post and pasted this

# peak.stereo: uncaught error: uncaught error: index out of bounds
def test_peak_stereo()
 r = peak.stereo(duration=1., radio)
 output.dummy(r)
 res = r.peak()
end
#thread.run(every=1., test_peak_stereo)

# peak: -nan for sine()
def test_peak_mono()
 rl = stereo.left(radio)
 rr = stereo.right(radio)
 pl = peak(duration=1., rl)
 pr = peak(duration=1., rr)
 output.dummy(pl)
 output.dummy(pr)
 resl= pl.peak()
 resr= pr.peak()
 print ("#{resl} #{resr}\n")
end
#thread.run(every=1., test_peak_mono)

It's all the same again. It's wrong use of functions. If you want it to work - use the right syntax given by @vitoyucepi 

radio = sine();
radio = stereo(radio)

r = rms.stereo(radio, duration=1.)
r1 = peak.stereo(duration=1., radio)

rl = rms(stereo.left(radio), duration=1.)
rr = rms(stereo.right(radio), duration=1.)

pl = peak(duration=1., stereo.left(radio))
pr = peak(duration=1., stereo.right(radio))

output.dummy(r)
output.dummy(r1)

output.dummy(rl)
output.dummy(rr)

output.dummy(pl)
output.dummy(pr)

def log_rms_peak()
  log("#{r.rms()} #{rl.rms()} #{rr.rms()}")
  log("#{r1.peak()} #{pl.peak()} #{pr.peak()}")
end

thread.run(log_rms_peak, every=1.)

output(radio)

or


radio = sine();

def source_rms(s)
  r = rms.stereo(s, duration=1.)
  rl = rms(stereo.left(radio), duration=1.)
  rr = rms(stereo.right(radio), duration=1.)
  output.dummy(r)
  output.dummy(rl)
  output.dummy(rr)

  {
    rms = r.rms,
    rms_left = rl.rms,
    rms_right = rr.rms
  }
end

def source_peak(s)
  p = peak.stereo(s, duration=1.)
  pl = peak(stereo.left(radio), duration=1.)
  pr = peak(stereo.right(radio), duration=1.)
  output.dummy(p)
  output.dummy(pl)
  output.dummy(pr)

  {
    peak = p.peak,
    peak_left = pl.peak,
    peak_right = pr.peak
  }
end

r = source_rms(radio)
p = source_peak(radio)

thread.run({log("#{r.rms()} #{r.rms_left()} #{r.rms_right()}\n#{p.peak()} #{p.peak_left()} #{p.peak_right()}")}, every=1.)

output(radio)
liquidpeter commented 2 months ago

Hi Stefan and Vito,

Thank you very much for the clarification, and Vito, and I apologize for my earlier response. I should have tested your code before providing feedback.

I mistakenly conflated the mono and stereo cases in the same function, which led to an oversight on my part. Specifically, I incorrectly used the stereo.rms, stereo.peak, and for mono rms, and peak functions. These functions are designed to inject the data getters rms() and peak() and should only be invoked once. However, my implementation called them repeatedly, causing the data getters to be re-initialized with every call, leading to the observed behavior where no data was collected and the output was incorrect. The correct approach is to have the thread run frequently and simply read the data from the previously initialized data getters. My mistake was unrelated to the upgrade from version 2.0.2 to 2.2.5 but rather a result of modifications I made to my working script. Now that I've corrected the implementation, everything is functioning as expected. Thank you again for your help!

I have still two questions open 1.) When I incorrectly called r = rms(stereo.left(radio), duration=1.) and x=rl.rms() for mono, it resulted in x=-nan. I think that is fine. For stereo, I would expect calling r = stereo.rms(radio, duration=1.) and x=r.rms frequently to return x=(-nan, -nan) instead of throwing the "out of index" error. (The error ist independent of doing anything with the rms values. ) While this is an edge case and not critical, it would improve consistency between mono and stereo behavior.

2.) When running Vito's script, it outputs the following:

2024/08/11 10:41:53 [lang:3] (0.707106781187, 0.707106781187) 0.353553390593 0.353553390593
2024/08/11 10:41:53 [lang:3] (0.999999746258, 0.999999746258) 0.499999873129 0.499999873129

In each line, values (1, 2) are from the stereo operators, while values 3 and 4 are from the mono operators. I expected the values to be identical for both mono and stereo channels, whether for peak or RMS calculations. However, it seems that the mono channel values are half as large as the stereo channel values. Could you explain why this discrepancy exists between mono and stereo calculations?

Thanks for your patience!

gAlleb commented 2 months ago

Hi @liquidpeter.

2024/08/11 10:41:53 [lang:3] (0.707106781187, 0.707106781187) 0.353553390593 0.353553390593 2024/08/11 10:41:53 [lang:3] (0.999999746258, 0.999999746258) 0.499999873129 0.499999873129 In each line, values (1, 2) are from the stereo operators, while values 3 and 4 are from the mono operators. I expected the values to be identical for both mono and stereo channels, whether for peak or RMS calculations. However, it seems that the mono channel values are half as large as the stereo channel values. Could you explain why this discrepancy exists between mono and stereo calculations?

Yeap, it is somehow misleading. Not sure about it at all. I've created an issue https://github.com/savonet/liquidsoap/issues/4090

toots commented 2 months ago

Thanks for the help y'all! I think that this issue should be moved to a discussion.