Building on FreeBSD 11.3 (FreeNAS jail)

asmod3us commented 4 years ago

Hello,

I'm trying to build from sources on FreeBSD (jail with 11.3-RELEASE-p9 on FreeNAS 11.3) based on the provided script.

Currently running into a challenge with configure not finding libraries:


checking for zlib... no
configure: error: Package requirements (zlib) were not met:

No package 'zlib' found

Consider adjusting the PKG_CONFIG_PATH environment variable if you installed software in a non-standard prefix.

Alternatively, you may set the environment variables ZLIB_CFLAGS and ZLIB_LIBS to avoid the need to call pkg-config. See the pkg-config man page for more details.

'[' -f /usr/local/etc/forked-daapd.conf ']'
echo 'Backing up old config file to /usr/local/etc/forked-daapd.conf.bak' Backing up old config file to /usr/local/etc/forked-daapd.conf.bak
cp /usr/local/etc/forked-daapd.conf /usr/local/etc/forked-daapd.conf.bak
gmake install gmake: *** No rule to make target 'install'. Stop.
sed -i -- 's/\/var\/cache/\/usr\/local\/var\/cache/g' /usr/local/etc/forked-daapd.conf
echo daapd::::::forked-daapd:/nonexistent:/usr/sbin/nologin:
adduser -w no -D -f - adduser: ERROR: User exists! pw: login name `daapd' already exists adduser: ERROR: There was an error adding user (daapd).
chown -R daapd:daapd /usr/local/var/cache/forked-daapd
'[' '!' -f freebsd_start_11.3.sh ']'
echo 'Could not find FreeBSD startup script' Could not find FreeBSD startup script
exit

One challenge seems to be that pkg-config is not fully working, or some libraries do not ship .pc files. For zlib it seems to work:

PKG_CONFIG_PATH=/usr/local/lib:/usr/local/lib/pkgconfig:/usr/libdata/pkgconfig/:/usr/local/libdata/pkgconfig
export PKG_CONFIG_PATH
pkg-config zlib --cflags --libs
-lz

But for some reason they are not picked up by configure. Any ideas on that? Short of manually setting the ~50 THELIB_CFLAGS, THELIB_LIBS (based on configure --help, is there a better way to configure the build?

Setting ZLIB_LIBS, ZLIB_CFLAGS allowed me to move past that error and run into the next one with mxml. The pkg-config has this:
```
pkg-config mxml --cflags --libs
Variable 'DESTDIR' not defined in '/usr/local/libdata/pkgconfig/mxml.pc'
```

and configure runs into that error.

$ grep DESTDIR /usr/local/libdata/pkgconfig/mxml.pc
prefix=${DESTDIR}/usr/local
exec_prefix=${DESTDIR}/usr/local

i think DESTDIR should be empty, as mxml is installed in /usr/local, but export DESTDIR= does not work, configure runs into the same error.

I'm not very familiar with the FreeBSD ecosystem, is anyone else able to help?

whatdoineed2do commented 4 years ago

Try using the following to specify locations for all:

CFLAGS="-I/usr/local/include" LDFLAGS="-L/usr/local/lib -lz -llxmxml" ./configure

Adjust the CFLAGS to include extra include dir locations and LDFLAGS to incl the libs. Still not great but you get to spec everythign in 2 variables instead of the ~50 THELIB_CFLAGS / THELIB_LDFLAGS

asmod3us commented 4 years ago

Thanks, that's a great idea! I also had to remove /usr/local/libdata/pkgconfig from PKG_CONFIG_PATH.

What I use right now:

CFLAGS="-I/usr/local/include" LDFLAGS="-L/usr/local/lib -lz -lmxml -lconfuse" ./configure

On to resolve version dependency on libconfuse >= 3 now...

asmod3us commented 4 years ago

pkg-config insists on a package for libconfuse even with the correct linker flags already applied (see invocation of configure in last comment):

configure:16219: $PKG_CONFIG --exists --print-errors "libconfuse >= 3.0"
Package libconfuse was not found in the pkg-config search path.
Perhaps you should add the directory containing `libconfuse.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libconfuse' found
configure:16222: $? = 1
configure:16236: $PKG_CONFIG --exists --print-errors "libconfuse >= 3.0"
Package libconfuse was not found in the pkg-config search path.
Perhaps you should add the directory containing `libconfuse.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libconfuse' found
configure:16239: $? = 1
configure:16253: result: no
No package 'libconfuse' found
configure:16278: error: Package requirements (libconfuse >= 3.0) were not met:

libconfuse is installed:

[root@daapd ~/forked-daapd_build/forked-daapd]# ls -l /usr/local/lib/libconfuse.*
-rw-r--r--  1 root  wheel  81838 May 26 11:51 /usr/local/lib/libconfuse.a
lrwxr-xr-x  1 root  wheel     19 May 26 11:51 /usr/local/lib/libconfuse.so -> libconfuse.so.2.0.0
lrwxr-xr-x  1 root  wheel     19 May 26 11:51 /usr/local/lib/libconfuse.so.2 -> libconfuse.so.2.0.0
-rwxr-xr-x  1 root  wheel  57592 May 26 11:51 /usr/local/lib/libconfuse.so.2.0.0

The version installed via pkg or ports is 3.2.1_1.

Setting

CONFUSE_CFLAGS="-I/usr/local/include"
CONFUSE_LIBS="-L/usr/local/lib -lconfuse"

works but gets me back to mxml not being found.

configure:16328: checking for mxml
configure:16335: $PKG_CONFIG --exists --print-errors "mxml"
Package mxml was not found in the pkg-config search path.
Perhaps you should add the directory containing `mxml.pc'
to the PKG_CONFIG_PATH environment variable
No package 'mxml' found
configure:16338: $? = 1
configure:16352: $PKG_CONFIG --exists --print-errors "mxml"
Package mxml was not found in the pkg-config search path.
Perhaps you should add the directory containing `mxml.pc'
to the PKG_CONFIG_PATH environment variable
No package 'mxml' found
configure:16355: $? = 1
configure:16369: result: no
No package 'mxml' found
configure:16394: error: Package requirements (mxml) were not met:

Any ideas?

asmod3us commented 4 years ago

I've managed to build in the end, using the dedicated variables like PKG_LIBS. Any suggestion welcome on how to use pkg-config to build properly, i.e. without managing the lib dependencies manually.

Leaving the script here for reference. I took the existing one, removed user prompts and made some modifications to get it working in a jail.

#!/usr/bin/env bash

set -o errexit   # Exit on most errors
set -o errtrace  # Make sure any error trap is inherited
set -o nounset   # Disallow expansion of unset variables
set -o pipefail  # Use last non-zero exit code in a pipeline

DEPS="gmake autoconf automake libtool gettext gperf glib pkgconf wget git \
     libevent mxml libgcrypt libunistring libiconv libplist libinotify avahi \
     sqlite3 alsa-lib libsodium json-c libwebsockets gmake gperf iconv \
     libtool localbase pathfix pkgconfig sqlite libavahi-client libconfuse \
     libevent libgcrypt libinotify libunistring libasound libplist libcurl \
     git gawk avahi-libdns openjdk8-jre"

pkg install -y "$DEPS"

# this does not work in a jail. is there an alternative?
# sh -c 'echo "fdesc      /dev/fd fdescfs rw      0       0" >> /etc/fstab'
# sh -c 'echo "proc       /proc   procfs  rw      0       0" >> /etc/fstab'
# mount /dev/fd
# mount /proc

WORKDIR=$(mktemp -d -t daapd)
CONFIG=/usr/local/etc/forked-daapd.conf
pushd $WORKDIR

ENABLE64BIT="--enable-64bit"
PKG_CONFIG_PATH="${PKG_CONFIG_PATH}:/usr/local/ffmpeg/target/lib/pkgconfig"
export PKG_CONFIG_PATH

wget -c --no-check-certificate https://github.com/antlr/website-antlr3/raw/gh-pages/download/antlr-3.4-complete.jar
wget -c --no-check-certificate https://github.com/antlr/website-antlr3/raw/gh-pages/download/C/libantlr3c-3.4.tar.gz

install antlr-3.4-complete.jar /usr/local/share/java
printf "#!/bin/sh
export CLASSPATH
CLASSPATH=\$CLASSPATH:/usr/local/share/java/antlr-3.4-complete.jar:/usr/local/share/java
/usr/local/bin/java org.antlr.Tool \$*
" > antlr3
install -m 755 antlr3 /usr/local/bin

tar xzf libantlr3c-3.4.tar.gz
cd libantlr3c-3.4
./configure $ENABLE64BIT && gmake -j8 && gmake install
cd $WORKDIR

git clone --depth 1 https://github.com/ejurgensen/forked-daapd.git
cd forked-daapd

gmake clean
git clean -f
autoreconf -vi

export CC=cc
PKG_CONFIG_PATH="${PKG_CONFIG_PATH}:/usr/local/ffmpeg/target/lib/pkgconfig"
export PKG_CONFIG_PATH

GEN_CFLAGS="-I/usr/include"
LOCAL_CFLAGS="-I/usr/local/include"

ZLIB_CFLAGS=$GEN_CFLAGS
ZLIB_LIBS="-lz"
export ZLIB_CFLAGS ZLIB_LIBS

CONFUSE_CFLAGS=$LOCAL_CFLAGS
CONFUSE_LIBS="-L/usr/local/lib -lconfuse"
export CONFUSE_CFLAGS
export CONFUSE_LIBS

MINIXML_CFLAGS=$LOCAL_CFLAGS
MINIXML_LIBS="-L/usr/local/lib -lmxml"
export MINIXML_CFLAGS MINIXML_LIBS

SQLITE3_CFLAGS=$LOCAL_CFLAGS
SQLITE3_LIBS="-L/usr/local/lib -lsqlite3"
export SQLITE3_CFLAGS SQLITE3_LIBS

LIBEVENT_CFLAGS=$LOCAL_CFLAGS
LIBEVENT_LIBS="-L/usr/local/lib -levent"
export LIBEVENT_CFLAGS LIBEVENT_LIBS

JSON_C_CFLAGS="-I/usr/local/include/json-c"
JSON_C_LIBS="-L/usr/local/lib -ljson-c"
export JSON_C_CFLAGS JSON_C_LIBS

LIBAV_CFLAGS="-I/usr/local/ffmpeg/target/include"
LIBAV_LIBS="-L/usr/local/ffmpeg/target/lib -aac -lasound -lavcodec -lavdevice -lavfilter -lavformat -lavutil -lfdk-aac -lpostproc -lswresample -lswscale -lx264 -lx265 -L/usr/local/lib -drm -lX11 -lXau -lXdmcp -lao -lbz2 -lc++ -ldl -lexpat -lfontconfig -lfreetype -lfribidi -lgcc -lgcc_s -liconv -llzma -lm -lpthread -lrt -lva -lvdpau -lxcb -lz -pthread -render -shape -shm -xfixes"

AVAHI_CFLAGS=$LOCAL_CFLAGS
AVAHI_LIBS="-L/usr/local/lib -lavahi-common -lavahi-client"
export AVAHI_CFLAGS AVAHI_LIBS

LIBPLIST_CFLAGS=$LOCAL_CFLAGS
LIBPLIST_LIBS="-L/usr/local/lib -lplist"
export LIBPLIST_CFLAGS LIBPLIST_LIBS

LIBSODIUM_CFLAGS=$LOCAL_CFLAGS
LIBSODIUM_LIBS="-L/usr/local/lib -lsodium"
export LIBSODIUM_CFLAGS LIBSODIUM_LIBS

LIBWEBSOCKETS_CFLAGS=$LOCAL_CFLAGS
LIBWEBSOCKETS_LIBS="-L/usr/local/lib -lwebsockets"
export LIBWEBSOCKETS_CFLAGS LIBWEBSOCKETS_LIBS

CFLAGS="-I/usr/local/include"
LDFLAGS="-L/usr/local/lib -lz -lmxml -ldns_sd -L/usr/local/ffmpeg/target/lib -lasound -lavcodec -lavdevice -lavfilter -lavformat -lavutil -lfdk-aac -lpostproc -lswresample -lswscale -lx264 -lx265"
export CFLAGS LDFLAGS

./configure && gmake -j8

if [ -f $CONFIG ]; then
        echo "Backing up old config file to $CONFIG.bak"
        cp "$CONFIG" "$CONFIG.bak"
fi
gmake install
sed -i -- 's/\/var\/cache/\/usr\/local\/var\/cache/g' $CONFIG

# Setup user and startup scripts
echo "daapd::::::forked-daapd:/nonexistent:/usr/sbin/nologin:" | adduser -w no -D -f -
chown -R daapd:daapd /usr/local/var/cache/forked-daapd
if [ ! -f scripts/freebsd_start_10.1.sh ]; then
        echo "Could not find FreeBSD startup script"
        exit
fi
install -m 755 scripts/freebsd_start_10.1.sh /usr/local/etc/rc.d/forked-daapd

popd

if ! service forked-daapd enabled; then
    sh -c 'echo "forked_daapd_enable=\"YES\"" >> /etc/rc.conf'
fi

if ! service dbus enabled; then
    sh -c 'echo "dbus_enable=\"YES\"" >> /etc/rc.conf'
fi
service dbus start

if ! service avahi-daemon enabled; then
    sh -c 'echo "avahi_daemon_enable=\"YES\"" >> /etc/rc.conf'
fi
service avahi-daemon start
service forked-daapd restart

ddppddpp commented 4 years ago

Great stuff, @asmod3us I tried to run your script in a newly created jail on a FreeNAS-11.3-U3.1, however I keep getting an error:

PKG_CONFIG_PATH: unbound variable

Any instructions how to setup the jail before running the script? i.e. pkg install bash...

asmod3us commented 4 years ago

Yep, sorry about that - I think I fixed that last night. I'm keeping the latest version at https://github.com/asmod3us/jailman/blob/daapd/blueprints/forked_daapd/build-ffmpeg.sh and have submitted a pull request for jailmanager/jailman

ddppddpp commented 4 years ago

Thnx, Probably off-topic here, but I have no experience with jailman, so what should be the format of the itunes_media variable? should it be like in your readme example (/mnt/tank/media/music/itunes/) or dataset-style (tank/media/music/itunes)?

ddppddpp commented 4 years ago

The script results in a forked-daapd built, which is great, but there are still a few small things. I can't open issues on your jailman PR. Should I provide feedback here? i.e. on start forked-daapd tries to check for a database in the /config directory and fails, because the directory is owned by root:wheel and not the daapd user...

asmod3us commented 4 years ago

Let me provide a variant of the script here without jailman.

asmod3us commented 4 years ago

Thank you for the feedback on the jailman blueprint. I believe this is ironed out now, the database is in /config and should have proper permissions. The input to itunes_media should be a dataset, it might not work if it's a path (which happens to be my use case too). I'm still thinking about the best way to reflect that in jailman .

I have created a gist to build without jailman, just run build.sh in a fresh jail. LMK how that works for you.

owntone / owntone-server

Building on FreeBSD 11.3 (FreeNAS jail) #1018