LS 2.3.0-rc2: Posix build from source tarball fails during doc generation

DigiBC commented 4 days ago

Description

Building Liquidsoap 2.3.0-rc2 on Linux (Debian 12) from source tarball with dune build and default settings works, but after changing the runtime lookup paths for Liquidsoap components to posix, it fails during doc generation.

The issue seems to be a result of commit c4746c8: Move doc generation to liquidsoap package, add pandoc and pandoc-include as doc deps.

However, it appears that the issue isn't related to pandoc and pandoc-include.

Steps to reproduce

export LIQUIDSOAP_BUILD_TARGET=posix
dune build

This will result in the following errors:

File "doc/dune.inc", lines 28-40, characters 0-362:
28 | (rule
29 |   (alias doc)
30 |   (package liquidsoap)
....
38 |       (setenv PAGER none
39 |         (run %{bin:liquidsoap} --no-external-plugins --list-extra-functions-md)))))
40 |       )
Command exited with code 1.
File "doc/dune.inc", lines 56-66, characters 0-228:
56 | (rule
57 |   (alias doc)
58 |   (package liquidsoap)
59 |   (deps
60 |     
61 |     (source_tree ../src/libs))
62 |   (target settings.md)
63 |   (action
64 |     (with-stdout-to settings.md
65 |       (setenv PAGER none
66 |         (run %{bin:liquidsoap} --list-settings)))))
Command exited with code 1.
File "doc/dune.inc", lines 14-26, characters 0-320:
14 | (rule
15 |   (alias doc)
16 |   (package liquidsoap)
....
24 |       (setenv PAGER none
25 |         (run %{bin:liquidsoap} --list-functions-md)))))
26 |       )
Command exited with code 1.
File "doc/dune.inc", lines 42-54, characters 0-353:
42 | (rule
43 |   (alias doc)
44 |   (package liquidsoap)
....
52 |       (setenv PAGER none
53 |         (run %{bin:liquidsoap} --list-deprecated-functions-md)))))
54 |       )
Command exited with code 1.
File "doc/dune.inc", lines 2-12, characters 0-234:
 2 | (rule
 3 |   (alias doc)
 4 |   (package liquidsoap)
 5 |   (deps
 6 |     
 7 |     (source_tree ../src/libs))
 8 |   (target protocols.md)
 9 |   (action
10 |     (with-stdout-to protocols.md
11 |       (setenv PAGER none
12 |         (run %{bin:liquidsoap} --list-protocols-md)))))
Command exited with code 1.

Expected behavior

Doc generation should work with different environment variables.

Liquidsoap version

Liquidsoap 2.3.0-rc2+dev
Copyright (c) 2003-2024 Savonet team
Liquidsoap is open-source software, released under GNU General Public License.
See <http://liquidsoap.info> for more information.

Liquidsoap build config

* Liquidsoap version  : 2.3.0-rc2+dev

 * Compilation options
   - Release build       : false
   - Git SHA             : (none)
   - OCaml version       : 4.14.2
   - OS type             : Unix
   - Libs versions       : alsa=82b575f angstrom=0.16.1 ao=9b685f4 asetmap=0.8.1 asn1-combinators=0.3.2 astring=0.8.5 base=v0.16.3 base.base_internalhash_types=v0.16.3 base.caml=v0.16.3 base.shadow_stdlib=v0.16.3 base64=3.5.1 bigarray=[distributed with Ocaml] bigarray-compat=1.1.0 bigstringaf=0.10.0 bjack=84cb2d5 bos=0.2.1 bytes=[distributed with OCaml 4.02 or above] ca-certs=v1.0.0 camlp-streams camomile.lib=2.0 cohttp=5.3.1 cohttp-lwt=5.3.0 cohttp-lwt-unix=5.3.0 conduit=7.1.0 conduit-lwt=7.1.0 conduit-lwt-unix=7.1.0 cry=e6f8a26 cstruct=6.2.0 ctypes=0.23.0 ctypes-foreign=0.23.0 ctypes.stubs=0.23.0 curl=0.9.2 digestif.c=1.2.0 domain-name=0.4.0 domain_shims dssi=9e75cbb dtools=0b431df dune-build-info=3.16.1 dune-private-libs.dune-section=3.16.1 dune-site=3.16.1 dune-site.private=3.16.1 duppy=e6c8533 eqaf=0.10 faad=95d7c39 fdkaac=59c1188 ffmpeg-av=1b72beb ffmpeg-avcodec=1b72beb ffmpeg-avdevice=1b72beb ffmpeg-avfilter=1b72beb ffmpeg-avutil=1b72beb ffmpeg-swresample=1b72beb ffmpeg-swscale=1b72beb fileutils=0.6.4 flac=107871f flac.decoder=107871f flac.ogg=107871f fmt=0.9.0 fpath=0.7.3 frei0r=d4846de gd=1.1 gen=1.1 gmap=0.3.0 inotify=v2.6 integers ipaddr=5.6.0 ipaddr-sexp=5.6.0 ipaddr.unix=5.6.0 irc-client irc-client-unix kdf.hkdf=1.0.0 kdf.pbkdf=1.0.0 ladspa=e8050d7 lame=d1f29e2 lastfm=b4d7e46 lilv=e863e12 liquidsoap-lang=2.3.0-rc2 liquidsoap-lang.console=2.3.0-rc2 liquidsoap_alsa liquidsoap_ao liquidsoap_bjack liquidsoap_builtins liquidsoap_core liquidsoap_dssi liquidsoap_faad liquidsoap_fdkaac liquidsoap_ffmpeg liquidsoap_flac liquidsoap_frei0r liquidsoap_gd liquidsoap_irc liquidsoap_ladspa liquidsoap_lame liquidsoap_lastfm liquidsoap_lilv liquidsoap_lo liquidsoap_mad liquidsoap_ndi liquidsoap_ogg liquidsoap_ogg_flac liquidsoap_optionals liquidsoap_opus liquidsoap_osc liquidsoap_oss liquidsoap_portaudio liquidsoap_posix_time liquidsoap_prometheus liquidsoap_pulseaudio liquidsoap_runtime liquidsoap_samplerate liquidsoap_sdl liquidsoap_sdl_log_level liquidsoap_shine liquidsoap_soundtouch liquidsoap_speex liquidsoap_sqlite liquidsoap_srt liquidsoap_ssl liquidsoap_stereotool liquidsoap_theora liquidsoap_tls liquidsoap_vorbis liquidsoap_xmlplaylist liquidsoap_yaml lo=0b43bdf logs=0.7.0 logs.fmt=0.7.0 logs.lwt=0.7.0 lwt lwt.unix macaddr=5.6.0 mad=51c568a magic-mime=1.3.1 mem_usage=9908046 memtrace=0.2.3 menhirLib=20240715 metadata=2d65927 mirage-crypto=1.1.0 mirage-crypto-ec=1.1.0 mirage-crypto-pk=1.1.0 mirage-crypto-rng=1.1.0 mirage-crypto-rng.unix=1.1.0 mm=6c440f5 mm.audio=6c440f5 mm.base=6c440f5 mm.image=6c440f5 mm.midi=6c440f5 mm.video=6c440f5 ndi ocplib-endian ocplib-endian.bigstring ogg=a708447 ogg.decoder=a708447 ohex opus=7d8fe5b opus.decoder=7d8fe5b osc osc-unix portaudio=cb52caa posix-base=2.0.2 posix-socket=2.0.2 posix-socket.constants=2.0.2 posix-socket.stubs=2.0.2 posix-socket.types=2.0.2 posix-time2=2.0.2 posix-time2.constants=2.0.2 posix-time2.stubs=2.0.2 posix-time2.types=2.0.2 posix-types=2.0.2 posix-types.constants=2.0.2 ppx_compare.runtime-lib=v0.16.0 ppx_hash.runtime-lib=v0.16.0 ppx_sexp_conv.runtime-lib=v0.16.0 prometheus=1.2 prometheus-app=1.2 ptime=1.2.0 ptime.clock=1.2.0 ptime.clock.os=1.2.0 pulseaudio=d64fbad re=1.12.0 result=1.5 rresult=0.7.0 samplerate=v0.1.6-8-g06a3a79 saturn_lockfree=0.4.1 sedlex=3.3 seq=[distributed with OCaml 4.07 or above] sexplib0=v0.16.0 shine=8071973 soundtouch=7baf637 speex=9edca47 speex.decoder=9edca47 sqlite3=5.2.0 srt=b880a73 srt.constants=b880a73 srt.stubs=b880a73 srt.stubs.locked=b880a73 srt.types=b880a73 ssl=0.7.0 stdlib-shims=0.3.0 stereotool str=[distributed with Ocaml] stringext=1.6.0 theora=a41ba52 theora.decoder=a41ba52 threads=[distributed with Ocaml] threads.posix=[internal] tls=1.0.4 tsdl=v1.1.0 tsdl-image=0.5 tsdl-ttf=0.6 unix=[distributed with Ocaml] unix-errno=0.6.2 unix-errno.errno_bindings=0.6.2 unix-errno.errno_types=0.6.2 unix-errno.errno_types_detected=0.6.2 unix-errno.unix=0.6.2 uri=4.4.0 uri-sexp=4.4.0 uri.services=4.4.0 vorbis=1060dac vorbis.decoder=1060dac x509=1.0.5 xmlm=1.4.0 xmlplaylist=31dba8c yaml=3.2.0 yaml.bindings=3.2.0 yaml.bindings.types=3.2.0 yaml.c=3.2.0 yaml.ffi=3.2.0 yaml.types=3.2.0 zarith=1.14
   - architecture        : amd64
   - host                : x86_64-pc-linux-gnu
   - target              : x86_64-pc-linux-gnu
   - system              : linux
   - ocamlopt_cflags     : -O2 -fno-strict-aliasing -fwrapv -pthread -fPIC
   - native_c_compiler   : gcc -O2 -fno-strict-aliasing -fwrapv -pthread -fPIC -D_FILE_OFFSET_BITS=64
   - native_c_libraries  : -lm

 * Configured paths
   - mode              : posix
   - standard library  : /usr/share/liquidsoap/libs
   - scripted binaries : /usr/share/liquidsoap/bin
   - rundir            : /var/run/liquidsoap
   - logdir            : /var/log/liquidsoap
   - user cache        : $HOME/.cache/liquidsoap (override with $LIQ_CACHE_USER_DIR)
   - system cache      : /var/cache/liquidsoap (override with $LIQ_CACHE_SYSTEM_DIR)
   - camomile files    : /usr/share/liquidsoap/camomile

 * Supported input formats
   - MP3               : yes
   - AAC               : yes
   - Ffmpeg            : yes
   - Flac (native)     : yes
   - Flac (ogg)        : yes
   - Opus              : yes
   - Speex             : yes
   - Theora            : yes
   - Vorbis            : yes
   - WAV/AIFF          : yes (native)

 * Supported output formats
   - FDK-AAC           : yes
   - FFmpeg            : yes
   - MP3               : yes
   - MP3 (fixed-point) : yes
   - Flac (native)     : yes
   - Flac (ogg)        : yes
   - Opus              : yes
   - Speex             : yes
   - Theora            : yes
   - Vorbis            : yes
   - WAV/AIFF          : yes (native)

 * Tags
   - AAC               : yes
   - FFmpeg            : yes
   - FLAC (native)     : yes
   - Flac (ogg)        : yes
   - Native decoder    : yes
   - Vorbis            : yes

 * Input / output
   - ALSA              : yes
   - AO                : yes
   - FFmpeg            : yes
   - JACK              : yes
   - NDI               : yes
   - OSS               : yes
   - Portaudio         : yes
   - Pulseaudio        : yes
   - SRT               : yes

 * Audio manipulation
   - FFmpeg            : yes
   - LADSPA            : yes
   - Lilv              : yes
   - Samplerate        : yes
   - SoundTouch        : yes
   - StereoTool        : yes

 * Video manipulation
   - camlimages        : no (requires camlimages)
   - FFmpeg            : yes
   - frei0r            : yes
   - ImageLib          : no (requires imagelib)
   - SDL               : yes

 * MIDI manipulation
   - DSSI              : yes

 * Visualization
   - GD                : yes
   - Graphics          : no (requires graphics)
   - SDL               : yes

 * Additional libraries
   - FFmpeg filters    : yes
   - FFmpeg devices    : yes
   - inotify           : yes
   - irc               : yes
   - jemalloc          : no (requires jemalloc)
   - lastfm            : yes
   - lo                : yes
   - memtrace          : yes
   - osc               : yes
   - ssl               : yes
   - sqlite3           : yes
   - tls               : yes
   - posix-time2       : yes
   - windows service   : no (requires winsvc)
   - YAML support      : yes
   - XML playlists     : yes

 * Monitoring
   - Prometheus        : yes

Installation method

From source/self-built

Additional Info

OS: Linux Mint Debian Edition 6 (Debian 12.8 AMD64)

DigiBC commented 4 days ago

Here are a few more details about the issue related to the generation of these 5 files:

protocols.md
reference-deprecated.md
reference-extras.md
reference.md
settings.md

Incomplete files with these names are created that contain the following error:

At /usr/share/liquidsoap/libs/thread.liq, line 34, char 28-32:
  thread.run.recurrent(fast=fast, delay=delay, on_error=on_error, f)

Error 6: Cannot apply that parameter because the function at /usr/share/liquidsoap/libs/thread.liq, line 34, char 2-68
has no argument labeled "fast"!

vitoyucepi commented 3 days ago

Hi @DigiBC, Could you provide the command sequence you are using to trigger this error? Here's how I install liquidsoap with a similar set of options.

Init script

OPAM_VERSION=2.3.0;
OCAML_VERSION=4.14.2;

sudo apt-get update;
sudo apt-get dist-upgrade;
sudo apt-get install --no-install-recommends -y curl ca-certificates patch unzip bubblewrap git bzip2 gcc make libc6-dev g++;
sudo curl -sS -L -o /usr/local/bin/opam "https://github.com/ocaml/opam/releases/download/$OPAM_VERSION/opam-$OPAM_VERSION-x86_64-linux";
sudo chmod +x /usr/local/bin/opam;
opam init --bare -n;
opam switch create "$OCAML_VERSION";
eval $(opam env --switch="$OCAML_VERSION");

Build liquidsoap

git clone https://github.com/savonet/liquidsoap.git;
cd liquidsoap;
opam pin --no-action --yes .
opam install --deps-only --confirm-level=unsafe-yes liquidsoap liquidsoap-lang liquidsoap-libs-extra liquidsoap-libs liquidsoap-core;
export LIQUIDSOAP_BUILD_TARGET=posix;
dune build --release;

Install other modules

opam install --deps-only --confirm-level=unsafe-yes liquidsoap sdl-liquidsoap liquidsoap-lang prometheus-liquidsoap liquidsoap-libs-extra liquidsoap-libs liquidsoap-mode tls-liquidsoap liquidsoap-core liquidsoap-js;
opam install --confirm-level=unsafe-yes mad faad ffmpeg flac ogg opus speex theora vorbis fdkaac lame shine alsa ao bjack portaudio pulseaudio srt ladspa lilv samplerate soundtouch frei0r dssi gd inotify irc-client-unix lastfm lo memtrace osc-unix ssl sqlite3 posix-time2 yaml xmlplaylist;
dune build --release;

DigiBC commented 3 days ago

Hi Vito,

Thanks for your reply.

My approach is a little different, because my intent is to build my own Debian (.deb) packages similar to the official released packages. I have been doing this for some time now without any problems.

I use a local OCAML installation with all required modules and the Liquidsoap source tarball for building.

The issue described here originally occurred during the dpkg-buildpackage process, but it was easy to reproduce with dune build.

All dependencies should be up to date except OPAM which is from the official Debian repository. In fact, it's version 2.1.2 might be a bit outdated.

So maybe I should update OPAM to the latest version 2.3.0 and see what happens...

EDIT Upgrading to OPAM 2.3.0 has no effect on the issue.

DigiBC commented 2 days ago

Finally, I discovered the cause of the issue: Fragments of a previous 2.3.0-rc1 installation had been left on my system. :disappointed: After cleaning the system, the Posix build also worked successfully (including the new release version).

So the issue can be closed.

I'm sorry... :pensive:

toots commented 2 days ago

Awesome, glad this was fixed!

savonet / liquidsoap