search

savonet / liquidsoap

Liquidsoap is a statically typed scripting general-purpose language with dedicated operators and backend for all thing media, streaming, file generation, automation, HTTP backend and more.
http://liquidsoap.info
GNU General Public License v2.0
1.4k stars 130 forks source link

2.0.0 fails to compile opus / ogg / vorbis inconsistant assumptions #1378

Closed Russsgithub closed 4 years ago

Russsgithub commented 4 years ago

Describe the bug Installing 2.0.0 via opam, liquidsoap fails with the following error :-

#=== ERROR while compiling liquidsoap.2.0.0 ===================================#
# context     2.0.5 | linux/x86_64 | ocaml-base-compiler.4.10.1 | pinned(git+file:///home/ubuntu/liquidsoap#master#af7b5cc2)
# path        ~/.opam/4.10.1/.opam-switch/build/liquidsoap.2.0.0
# command     /usr/bin/make
# exit-code   2
# env-file    ~/.opam/log/liquidsoap-122434-ab22a1.env
# output-file ~/.opam/log/liquidsoap-122434-ab22a1.out
### output ###
# [...]
# CC -c tools/unix_c.c
# CC -c io/oss_io_c.c
# OCAMLOPT -o liquidsoap
# File "_none_", line 1:
# Error: Files /home/ubuntu/.opam/4.10.1/lib/opus/opus.cmxa
#        and /home/ubuntu/.opam/4.10.1/lib/ogg/ogg.cmxa
#        make inconsistent assumptions over interface Ogg
# make[2]: *** [../Makefile.rules:106: liquidsoap] Error 2
# make[2]: Leaving directory '/home/ubuntu/.opam/4.10.1/.opam-switch/build/liquidsoap.2.0.0/src'
# make[1]: *** [../Makefile.rules:67: all-auto-ocaml-prog] Error 2
# make[1]: Leaving directory '/home/ubuntu/.opam/4.10.1/.opam-switch/build/liquidsoap.2.0.0/src'
# make: *** [Makefile.rules:27: all-subdirs] Error 2

To Reproduce Install from liquidsoap git using liquidsoap-full pinned dependancies

Expected behavior Clean install

Version details

Install method Opam

Thanks

toots commented 4 years ago

Hi! It looks like you need to recompile these package. I'd suggest the following:

opam reinstall -y ogg opus liquidsoap
Russsgithub commented 4 years ago

Hello, Thank for responding, I recompiled, but still get the same error. ogg and opus compile but liquidsoap fails. Also tried removing ogg and opus, (as [NOTE] While removing ogg.0.6.0: not removing non-empty directories:

Install succeeds without ogg and opus, but I'd like to stream using opus really.

Russsgithub commented 4 years ago

Also tried re downloading from liquidsoap-full, repinning and reinstalling.

Build fails with same error.

Package versions are as follows.

# Name                  # Installed # Synopsis
base-bigarray           base
base-bytes              base        Bytes library distributed with the OCaml compiler
base-threads            base
base-unix               base
camomile                1.0.2       A Unicode library
conf-libpcre            1           Virtual package relying on a libpcre system installation
conf-m4                 1           Virtual package relying on m4
conf-pkg-config         1.3         Virtual package relying on pkg-config installation
cry                     0.6.5       pinned to version 0.6.5 at git+file:///home/ubuntu/liquidsoap-fu
csexp                   1.3.2       Parsing and printing of S-expressions in Canonical form
dtools                  0.4.3       pinned to version 0.4.3 at git+file:///home/ubuntu/liquidsoap-fu
dune                    2.7.1       Fast, portable, and opinionated build system
dune-configurator       2.7.1       Helper library for gathering system configuration
duppy                   0.9.0       pinned to version 0.9.0 at git+file:///home/ubuntu/liquidsoap-fu
ffmpeg                  0.5.0       pinned to version 0.5.0 at git+file:///home/ubuntu/liquidsoap-fu
ffmpeg-av               0.5.0       pinned to version 0.5.0 at git+file:///home/ubuntu/liquidsoap-fu
ffmpeg-avcodec          0.5.0       pinned to version 0.5.0 at git+file:///home/ubuntu/liquidsoap-fu
ffmpeg-avdevice         0.5.0       pinned to version 0.5.0 at git+file:///home/ubuntu/liquidsoap-fu
ffmpeg-avfilter         0.5.0       pinned to version 0.5.0 at git+file:///home/ubuntu/liquidsoap-fu
ffmpeg-avutil           0.5.0       pinned to version 0.5.0 at git+file:///home/ubuntu/liquidsoap-fu
ffmpeg-swresample       0.5.0       pinned to version 0.5.0 at git+file:///home/ubuntu/liquidsoap-fu
ffmpeg-swscale          0.5.0       pinned to version 0.5.0 at git+file:///home/ubuntu/liquidsoap-fu
gen                     0.5.3       Iterators for OCaml, both restartable and consumable
ladspa                  0.2.0       pinned to version 0.2.0 at git+file:///home/ubuntu/liquidsoap-fu
lo                      0.1.2       pinned to version 0.1.2 at git+file:///home/ubuntu/liquidsoap-fu
mad                     0.5.0       pinned to version 0.5.0 at git+file:///home/ubuntu/liquidsoap-fu
menhir                  20200624    An LR(1) parser generator
menhirLib               20200624    Runtime support library for parsers generated by Menhir
menhirSdk               20200624    Compile-time library for auxiliary tools related to Menhir
mm                      0.6.0       pinned to version 0.6.0 at git+file:///home/ubuntu/liquidsoap-fu
ocaml                   4.10.0      The OCaml compiler (virtual package)
ocaml-base-compiler     4.10.0      Official release 4.10.0
ocaml-config            1           OCaml Switch Configuration
ocaml-migrate-parsetree 1.7.3       Convert OCaml parsetrees between different versions
ocamlbuild              0.14.0      OCamlbuild is a build system with builtin rules to easily build 
ocamlfind               1.8.1       A library manager for OCaml
ogg                     0.6.0       pinned to version 0.6.0 at git+file:///home/ubuntu/liquidsoap-fu
opam-depext             1.1.3       Query and install external dependencies of OPAM packages
opus                    0.1.3       pinned to version 0.1.3 at git+file:///home/ubuntu/liquidsoap-fu
pcre                    7.4.6       Bindings to the Perl Compatibility Regular Expressions library
ppx_derivers            1.2.1       Shared [@@deriving] plugin registry
ppx_tools_versioned     5.4.0       A variant of ppx_tools based on ocaml-migrate-parsetree
result                  1.5         Compatibility Result module
sedlex                  2.2         An OCaml lexer generator for Unicode
soundtouch              0.1.9       pinned to version 0.1.9 at git+file:///home/ubuntu/liquidsoap-fu
taglib                  0.3.7       pinned to version 0.3.7 at git+file:///home/ubuntu/liquidsoap-fu
uchar                   0.0.2       Compatibility library for OCaml's Uchar module

I downgraded switch to 4.10.0 since the original post.

Thanks again.

toots commented 4 years ago

I'm sorry that it's being so hard for you. I'd suggest to remove all your pinned repositories. I recently released most dependencies needed for liquidsoap either 1.4.3 or 2.0.0 (still in development) so you shouldn't need to pin either opus or ogg at this point..

toots commented 4 years ago

Worst case, you can just wipe out your whole ~/.opam folder and restart from scratch. No need to use liquidsoap-full and/or pin any dependency. Just type opam install liquidsoap opus ogg and you should be set :-)

Russsgithub commented 4 years ago

version 1.4.3 installs perfectly ...... brilliant.

I'm using lo for metadata transfers and have been having issues with it not working when liquidsoap is deamonized, which is why I've been trying to install 2.0.0 , I understand this version is in development.

version 2.0.0 install is still giving me problems, when I upgrade from 1.4.3 I get 

#=== ERROR while compiling liquidsoap.2.0.0 ===================================#
# context     2.0.5 | linux/x86_64 | ocaml-base-compiler.4.10.1 | pinned(git+file:///home/ubuntu/liquidsoap#master#fefd3dd8)
# path        ~/.opam/4.10.1/.opam-switch/build/liquidsoap.2.0.0
# command     /usr/bin/make
# exit-code   2
# env-file    ~/.opam/log/liquidsoap-37945-17d454.env
# output-file ~/.opam/log/liquidsoap-37945-17d454.out
### output ###
# [...]
# OCAMLOPT -c encoder_formats/fdkaac_format.ml
# OCAMLOPT -c encoder_formats/flac_format.ml
# OCAMLOPT -c encoder_formats/ffmpeg_format.ml
# File "encoder_formats/ffmpeg_format.ml", line 38, characters 17-38:
# 38 |   pixel_format : Avutil.Pixel_format.t;
#                       ^^^^^^^^^^^^^^^^^^^^^
# Error: Unbound module Avutil
# make[2]: *** [../Makefile.rules:125: encoder_formats/ffmpeg_format.cmx] Error 2
# make[2]: Leaving directory '/home/ubuntu/.opam/4.10.1/.opam-switch/build/liquidsoap.2.0.0/src'
# make[1]: *** [../Makefile.rules:67: all-auto-ocaml-prog] Error 2
# make[1]: Leaving directory '/home/ubuntu/.opam/4.10.1/.opam-switch/build/liquidsoap.2.0.0/src'
# make: *** [Makefile.rules:27: all-subdirs] Error 2

So I updated ocaml-ffmpeg to fix this and the ogg / opus inconsitant assumptions returns

#=== ERROR while compiling liquidsoap.2.0.0 ===================================#
# context     2.0.5 | linux/x86_64 | ocaml-base-compiler.4.10.1 | pinned(git+file:///home/ubuntu/liquidsoap/liquidsoap-full/liquidsoap#master#fefd3dd8)
# path        ~/.opam/4.10.1/.opam-switch/build/liquidsoap.2.0.0
# command     /usr/bin/make
# exit-code   2
# env-file    ~/.opam/log/liquidsoap-49666-fdd1f6.env
# output-file ~/.opam/log/liquidsoap-49666-fdd1f6.out
### output ###
# [...]
# CC -c tools/unix_c.c
# CC -c io/oss_io_c.c
# OCAMLOPT -o liquidsoap
# File "_none_", line 1:
# Error: Files /home/ubuntu/.opam/4.10.1/lib/opus/opus.cmxa
#        and /home/ubuntu/.opam/4.10.1/lib/ogg/ogg.cmxa
#        make inconsistent assumptions over interface Ogg
# make[2]: *** [../Makefile.rules:106: liquidsoap] Error 2
# make[2]: Leaving directory '/home/ubuntu/.opam/4.10.1/.opam-switch/build/liquidsoap.2.0.0/src'
# make[1]: *** [../Makefile.rules:67: all-auto-ocaml-prog] Error 2
# make[1]: Leaving directory '/home/ubuntu/.opam/4.10.1/.opam-switch/build/liquidsoap.2.0.0/src'
# make: *** [Makefile.rules:27: all-subdirs] Error 2

I'll try to copy across builtins_lo.ml from the master branch later and recompile liquidsoap 1.4.3.

Thanks

toots commented 4 years ago

This is very surprising and annoying. I'll work out a patch for 1.4.3 next but before, I'd like to give opam one last try. Here are instructions for a complete re-install from scratch:

% opam update
% opam pin list  | cut -d'.' -f 1 | xargs opam unpin -y
% git clone https://github.com/savonet/ocaml-ffmpeg.git
% cd ocaml-ffmpeg && opam pin -n -y . && cd ..
% git clone https://github.com/savonet/liquidsoap.git
% cd liquidsoap && opam pin -n -y . && cd ..
% opam depext -i liquidosoap ffmpeg lo
Russsgithub commented 4 years ago

I'm afraid not .....

When I run the commands I get

# Detecting depexts using vars: arch=x86_64, os=linux, os-distribution=ubuntu, os-family=debian
[ERROR] No solution for liquidosoap & ffmpeg & lo & ogg & opus: Sorry, no solution found: there seems to be a problem with your request.

Command failed: opam list --readonly --external  --resolve=liquidosoap,ffmpeg,lo,ogg,opus returned 20

If I install separately i.e

opam install ffmpeg
opam install lo
opam install opus
etc

All goes well untill

opam install liquidsoap

when the 

#=== ERROR while compiling liquidsoap.2.0.0 ===================================#
# context     2.0.5 | linux/x86_64 | ocaml-base-compiler.4.10.1 | pinned(git+file:///home/ubuntu/liquidsoap#master#e9134526)
# path        ~/.opam/4.10.1/.opam-switch/build/liquidsoap.2.0.0
# command     /usr/bin/make
# exit-code   2
# env-file    ~/.opam/log/liquidsoap-87860-1315d7.env
# output-file ~/.opam/log/liquidsoap-87860-1315d7.out
### output ###
# [...]
# CC -c tools/unix_c.c
# CC -c io/oss_io_c.c
# OCAMLOPT -o liquidsoap
# File "_none_", line 1:
# Error: Files /home/ubuntu/.opam/4.10.1/lib/opus/opus.cmxa
#        and /home/ubuntu/.opam/4.10.1/lib/ogg/ogg.cmxa
#        make inconsistent assumptions over interface Ogg
# make[2]: *** [../Makefile.rules:106: liquidsoap] Error 2
# make[2]: Leaving directory '/home/ubuntu/.opam/4.10.1/.opam-switch/build/liquidsoap.2.0.0/src'
# make[1]: *** [../Makefile.rules:67: all-auto-ocaml-prog] Error 2
# make[1]: Leaving directory '/home/ubuntu/.opam/4.10.1/.opam-switch/build/liquidsoap.2.0.0/src'
# make: *** [Makefile.rules:27: all-subdirs] Error 2

reappears.

Thanks

Russsgithub commented 4 years ago

Just noticed a typo in the last command liquidosoap (I do that a lot).

-- Retried in ubuntu which works, but if i add opus the make inconsistent assumptions over interface Ogg error returns.

I also ran the same install in arch with the same result.

toots commented 4 years ago

Frankly I'm quite confused as to what's causing your issue. It may be a problem in opam or in our packages or in your local setup. If you don't mind I'd like to get more details on the situation. Would you mind sending me the result of opam pin list ?

Russsgithub commented 4 years ago

I'm running a fresh AWS Ubuntu 20.04 install, opam from apt.

Pin list at the time was

ffmpeg.0.5.0                            git  git+file:///home/ubuntu/ocaml-ffmpeg#master
ffmpeg-av.0.5.0                         git  git+file:///home/ubuntu/ocaml-ffmpeg#master
ffmpeg-avcodec.0.5.0                    git  git+file:///home/ubuntu/ocaml-ffmpeg#master
ffmpeg-avdevice.0.5.0                   git  git+file:///home/ubuntu/ocaml-ffmpeg#master
ffmpeg-avfilter.0.5.0                   git  git+file:///home/ubuntu/ocaml-ffmpeg#master
ffmpeg-avutil.0.5.0                     git  git+file:///home/ubuntu/ocaml-ffmpeg#master
ffmpeg-swresample.0.5.0                 git  git+file:///home/ubuntu/ocaml-ffmpeg#master
ffmpeg-swscale.0.5.0                    git  git+file:///home/ubuntu/ocaml-ffmpeg#master
liquidsoap.2.0.0         (uninstalled)  git  git+file:///home/ubuntu/liquidsoap#master

Opam list


# Name                  # Installed # Synopsis
base-bigarray           base
base-bytes              base        Bytes library distributed with the OCaml compiler
base-threads            base
base-unix               base
camomile                1.0.2       A Unicode library
conf-libpcre            1           Virtual package relying on a libpcre system installation
conf-m4                 1           Virtual package relying on m4
conf-pkg-config         1.3         Virtual package relying on pkg-config installation
csexp                   1.3.2       Parsing and printing of S-expressions in Canonical form
dtools                  0.4.2       Library providing various helper functions to make daemons
dune                    2.7.1       Fast, portable, and opinionated build system
dune-configurator       2.7.1       Helper library for gathering system configuration
duppy                   0.9.0       Library providing monadic threads
ffmpeg                  0.5.0       pinned to version 0.5.0 at git+file:///home/ubuntu/ocaml-ffmpeg#
ffmpeg-av               0.5.0       pinned to version 0.5.0 at git+file:///home/ubuntu/ocaml-ffmpeg#
ffmpeg-avcodec          0.5.0       pinned to version 0.5.0 at git+file:///home/ubuntu/ocaml-ffmpeg#
ffmpeg-avdevice         0.5.0       pinned to version 0.5.0 at git+file:///home/ubuntu/ocaml-ffmpeg#
ffmpeg-avfilter         0.5.0       pinned to version 0.5.0 at git+file:///home/ubuntu/ocaml-ffmpeg#
ffmpeg-avutil           0.5.0       pinned to version 0.5.0 at git+file:///home/ubuntu/ocaml-ffmpeg#
ffmpeg-swresample       0.5.0       pinned to version 0.5.0 at git+file:///home/ubuntu/ocaml-ffmpeg#
ffmpeg-swscale          0.5.0       pinned to version 0.5.0 at git+file:///home/ubuntu/ocaml-ffmpeg#
gen                     0.5.3       Iterators for OCaml, both restartable and consumable
lo                      0.1.2       Bindings for the lo library which provides functions for communi
menhir                  20200624    An LR(1) parser generator
menhirLib               20200624    Runtime support library for parsers generated by Menhir
menhirSdk               20200624    Compile-time library for auxiliary tools related to Menhir
mm                      0.6.0       The mm library contains high-level to create and manipulate mult
ocaml                   4.10.1      The OCaml compiler (virtual package)
ocaml-base-compiler     4.10.1      Official release 4.10.1
ocaml-config            1           OCaml Switch Configuration
ocaml-migrate-parsetree 1.7.3       Convert OCaml parsetrees between different versions
ocamlbuild              0.14.0      OCamlbuild is a build system with builtin rules to easily build 
ocamlfind               1.8.1       A library manager for OCaml
ogg                     0.6.0       Interface for Ogg Bitstream Library, otherwise known as libogg
opam-depext             1.1.3       Query and install external dependencies of OPAM packages
opus                    0.1.3       Bindings for the opus library to decode audio files in opus form
pcre                    7.4.6       Bindings to the Perl Compatibility Regular Expressions library
ppx_derivers            1.2.1       Shared [@@deriving] plugin registry
ppx_tools_versioned     5.4.0       A variant of ppx_tools based on ocaml-migrate-parsetree
result                  1.5         Compatibility Result module
sedlex                  2.2         An OCaml lexer generator for Unicode
uchar                   0.0.2       Compatibility library for OCaml's Uchar module

As this is a clean install I have scrapped the above AWS instance and am now trying on a new instance of the same spec.

I'll let you know how it goes.

Russsgithub commented 4 years ago

Ok, so new instance , Ubuntu 20.04, install failed at aclocal (forgot to install all deps), installed aspcud autoconf automake build-essential ca-certificates curl debhelper devscripts fakeroot git openssh-client unzip gnupg dirmngr apt-transport-https Re-ran opam install liquidsoap ffmepg lo opus But hit the same error again make inconsistent assumptions over interface Ogg

I wish I could help pin down the issue.

Unless AWS are using a funky version of Ubuntu 20.04 I cant see that being an issue.

Opam .. I cant see a clean install having issues ( 1.4.3 with ffmpeg and opus installs - the issue has arisen on 3 different installs )

Liquidsoap packages - If you can't reproduce - ?

I'm happy to run 1.4.3 in screen for now as the script is still in development, so nothing to stress about from this end.

Thanks again.

toots commented 4 years ago

I think we got an hint of the issue here: https://github.com/ocaml/opam-repository/pull/17424 Gonna have a hard look at it later today.

toots commented 4 years ago

Thanks for your patience with this. It looks like the switch to dune wasn't as transparent as we were hoping. I've reverted to autoconf and published a new ogg.0.6.1 version which fixes your issue. You should be able to pin the latest ocaml-ogg if you need a quick solution.

Russsgithub commented 4 years ago

Still not installing I'm afraid, new error

<><> Carrying on to "opam depext -i ogg" ><><><><><><><><><><><><><><><><><><><>

# Detecting depexts using vars: arch=x86_64, os=linux, os-distribution=ubuntu, os-family=debian
[ERROR] No solution for ogg: Sorry, no solution found: there seems to be a problem with your request.

Command failed: opam list --readonly --external  --resolve=ogg returned 20

gives

[ogg.0.6.1] no changes from git+file:///home/russ/ocaml-ogg#master

The following dependencies couldn't be met:
  - ogg → conf-libogg
      unknown package

in Arch.

Thanks

toots commented 4 years ago

Ha yes, sorry I sliced up the depext into a separate package.. It should come through opam shortly.

toots commented 4 years ago

Actually, there is a full ogg.opam in the tagged release:

git clone https://github.com/savonet/ocaml-ogg.git
cd ocaml-ogg && git checkout v0.6.1 && opam install -y .

Should do it..

Russsgithub commented 4 years ago

Great, that did indeed do it.

Thank you.

toots commented 4 years ago

Awesome. Again, thank you for your patience.