ElementsProject / lightning

Core Lightning — Lightning Network implementation focusing on spec compliance and performance
Other
2.85k stars 902 forks source link

FreeBSD: Out of date installation instructions? #6301

Closed ghost closed 1 year ago

ghost commented 1 year ago

According to lightning/blob/master/doc/INSTALL.md#to-build-on-freebsd one should use the ports system for installing c-lightning on freebsd. However, as far as I can tell the version available on ports (FreshPorts) is very out of date (version 0.11.2).

Attempt 1: Follow the instructions on lightning/blob/master/doc/INSTALL.md#to-build-on-freebsd

Anyway, the current instructions (found in "/usr/ports/net-p2p/c-lightning/Makefile") are to modify a few lines near the top of the makefile, specifically:

PORTNAME=       c-lightning
#To build from an arbitrary git commit comment PORTVERSION and PORTREVISION (if present)
PORTVERSION=    0.11.2
DISTVERSIONPREFIX=      v
#and uncomment the following two lines (use for example -git-HEAD or -git-f8d8348c)
#PORTVERSION=   0
#PKGNAMESUFFIX= -git-f8d8348c
CATEGORIES=     net-p2p finance
…

Which I changed into:

PORTNAME=       c-lightning
#To build from an arbitrary git commit comment PORTVERSION and PORTREVISION (if present)
#PORTVERSION=   0.11.2
#DISTVERSIONPREFIX=     v
#and uncomment the following two lines (use for example -git-HEAD or -git-f8d8348c)
PORTVERSION=   0
PKGNAMESUFFIX=  v23.05.1
CATEGORIES=     net-p2p finance
…

This results in:

#cd /usr/ports/net-p2p/c-lightning && make install
===>  License MIT accepted by the user
===>   c-lightning- depends on file: /usr/local/sbin/pkg - found
=> ElementsProject-lightning-_GH0.tar.gz is not in /usr/ports/net-p2p/c-lightning/distinfo.
=> Either /usr/ports/net-p2p/c-lightning/distinfo is out of date, or
=> ElementsProject-lightning-_GH0.tar.gz is spelled incorrectly.
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/net-p2p/c-lightning
*** Error code 1

Stop.
make: stopped in /usr/ports/net-p2p/c-lightning

Attempt 2: Clone repo from GitHub and build myself

On a new host (jail to be precise) I ran the following script:

#!/usr/bin/env zsh

# install dependencies - found these by reading the makefile for the port

# build dependencies
pkg install -y bash py39-mako py39-mrkd
# lib dependencies
pkg install -y gmp libsodium sqlite3
# more dependencies
pkg install -y py39-pip py39-sqlite3
pkg install -y py39-cpe gettext gmake libtool

# add c-lightning user
# save this for later when things are actually working
#pw adduser -n c-lightning -u 735 -s '' -h -

# part of the instructions for building on ubuntu
# probably not needed since i found a pkg for py39-mako
# either way, ends in the message:
#   Requirement already satisfied: mako in /usr/local/lib/python3.9/site-packages (1.2.4)
pip install --upgrade pip
pip install mako

# clone repo from github
mkdir -p /usr/local/src
git clone https://github.com/ElementsProject/lightning.git /usr/local/src/cln
cd $_
git checkout v23.05

# attempt to configure and build
./configure
# use gmake rather than make since linux distros have a different flavour of make
gmake
gmake install

This ends on the "./configure" step with the following output:

…
checking for __builtin_cpu_supports()... yes
checking for closefrom() offered by system... yes
checking for F_CLOSEM defined for fctnl.... no
checking for close_range syscall available as __NR_close_range.... no
checking for F_MAXFD defined for fcntl.... no
checking for libsodium with IETF chacha20 variants... no
checking for sqlite3_expanded_sql... no
checking for sqlite3... no
checking for postgres... no
checking for compiler is GCC... no
checking for GCC version is 7 or above... no
checking for pwritev() defined... yes
Writing variables to config.vars.45220... yes
Writing header to ccan/config.h.45220... yes
checking for python3-mako... not found
checking for lowdown... not found
checking for sha256sum... found
Setting PREFIX... /usr/local
Setting CC... cc
Setting CONFIGURATOR_CC... cc
Setting CWARNFLAGS... -Wall -Wundef -Wmissing-prototypes -Wmissing-declarations -Wstrict-prototypes -Wold-style-definition -Werror
Setting CDEBUGFLAGS... -std=gnu11 -g -fstack-protector-strong
Setting COPTFLAGS... -Og
CSANFLAGS not found
SQLITE3_CFLAGS not found
Setting SQLITE3_LDLIBS... -lsqlite3
POSTGRES_INCLUDE not found
POSTGRES_LDLIBS not found
Setting VALGRIND... 0
Setting DEVELOPER... 0
Setting EXPERIMENTAL_FEATURES... 0
Setting COMPAT... 1
PYTEST not found
Setting STATIC... 0
Setting ASAN... 0
Setting UBSAN... 0
Setting TEST_NETWORK... regtest
Setting HAVE_PYTHON3_MAKO... 0
Setting HAVE_LOWDOWN... 0
Setting SHA256SUM... sha256sum
Setting FUZZING... 0
Setting RUST... 0
*** We need a database, but neither sqlite3 nor postgres found

I removed a bunch of rows at the top… the interesting rows in my opinion are the ones where it doesn't find sqlite3 and python3-mako… since I know they are there, it would appear it is looking in the wrong location. I assume it is due some differences in linux vs freebsd, but I'm not sure where start looking for answers. The freebsd port probably got the answer to it which leads me to attempt 3…

Attempt 3 - Build the old port version, then attempt to build using GitHub src

I attempted this and got a different set of errors. Sadly I'm running out of time to spend on this now, and I had been fiddling around too much with the build environment to be sure what state things actually were in. If no-one has posted any good ideas on this thread by Monday I will give it another try and make sure to document each step.

vincenzopalazzo commented 1 year ago

I think in this case, is you have a FreeBSD where to test, it is better to open a PR with the update changes.

I think they are old, and unfortunately I do not have a system where to test it

ghost commented 1 year ago

If I manage to get it to work I'll be sure to do that. I'll give it some time on Monday and see how it goes.

vincenzopalazzo commented 1 year ago

if you need help I am on core lightning discord, so please ping me

ghost commented 1 year ago

Turns out I only needed to add the SHA256 checksum for the tar.gz file for the requested version to make it work. It is compiling right now. Will post the exact instructions here either later tonight or tomorrow.

I'll look into how to update the port as well, however I don't have a clue regarding the procedures so not making any promises.

ghost commented 1 year ago

The build eventually failed… posting the output here without really looking into it right now. I'll investigate tomorrow.

The "MAKE_JOBS_UNSAFE=yes" is due to suggestion when it first failed. "make BATCH=" is so it will just accept all the default options when building all the dependencies.

cln# MAKE_JOBS_UNSAFE=yes
cln# make BATCH=
===>  Building for c-lightningv23.05.1-0
gmake[1]: Entering directory '/usr/ports/net-p2p/c-lightning/work/lightning-23.05.1'
CC: cc -DBINTOPKGLIBEXECDIR="../libexec/c-lightning" -Wall -Wundef -Wmissing-prototypes -Wmissing-declarations -Wstrict-prototypes -Wold-style-definition -Werror -std=gnu11 -g -fstack-protector-strong -Og -I ccan -I external/libwally-core/include/ -I external/libwally-core/src/secp256k1/include/ -I external/jsmn/ -I external/libbacktrace/ -I external/gheap/ -I external/x86_64-unknown-freebsd13.2/libbacktrace-build -I . -I/usr/local/include -I/usr/local/include      -DSHACHAIN_BITS=48 -DJSMN_PARENT_LINKS  -DCOMPAT_V052=1 -DCOMPAT_V060=1 -DCOMPAT_V061=1 -DCOMPAT_V062=1 -DCOMPAT_V070=1 -DCOMPAT_V072=1 -DCOMPAT_V073=1 -DCOMPAT_V080=1 -DCOMPAT_V081=1 -DCOMPAT_V082=1 -DCOMPAT_V090=1 -DCOMPAT_V0100=1 -DCOMPAT_V0121=1  -DBUILD_ELEMENTS=1 -c -o
LD: cc  -fstack-protector-strong    -Og  config.vars  -Lexternal/x86_64-unknown-freebsd13.2 -lwallycore -lsecp256k1 -ljsmn -lbacktrace -L/usr/local/include -lm -lgmp -L/usr/local/lib -lsqlite3  -lz  -lsodium -o
cc common/addr.c
cc common/bolt11.c
cc common/bolt11_json.c
cc common/bolt12.c
cc common/configdir.c
cc common/scb_wiregen.c
cc common/status_levels.c
cc common/invoice_path_id.c
cc common/json_filter.c
cc common/json_param.c
cc common/json_parse.c
cc common/json_parse_simple.c
cc common/json_stream.c
cc common/onion_encode.c
cc db/bindings.c
cc db/exec.c
cc wire/bolt12_wiregen.c
cc db/utils.c
cc db/db_postgres.c
cc db/db_sqlite3.c
cc cli/lightning-cli.c
cc devtools/bolt11-cli.c
cc wire/onion_printgen.c
cc wire/peer_printgen.c
cc wire/bolt12_printgen.c
cc wire/channel_type_printgen.c
cc devtools/print_wire.c
cc devtools/decodemsg.c
cc devtools/onion.c
cc devtools/dump-gossipstore.c
cc devtools/gossipwith.c
cc devtools/create-gossipstore.c
cc devtools/mkcommit.c
cc devtools/mkfunding.c
cc devtools/mkclose.c
cc devtools/mkgossip.c
cc devtools/mkencoded.c
cc devtools/mkquery.c
cc devtools/lightning-checkmessage.c
cc common/gossmap.c
cc common/fp16.c
cc common/dijkstra.c
cc common/route.c
cc devtools/clean_topo.c
cc devtools/topology.c
cc devtools/route.c
cc devtools/bolt12-cli.c
cc common/iso4217.c
cc devtools/encodeaddr.c
cc devtools/features.c
cc devtools/fp16.c
cc devtools/rune.c
cc devtools/checkchannels.c
cc tools/hsmtool.c
cc common/descriptor_checksum.c
cc tools/check-bolt.c
cc plugins/autoclean.c
cc plugins/libplugin.c
cc plugins/chanbackup.c
cc plugins/bcli.c
cc plugins/commando.c
cc plugins/fetchinvoice.c
cc plugins/funder.c
cc plugins/funder_policy.c
cc plugins/topology.c
cc plugins/keysend.c
cc plugins/libplugin-pay.c
cc common/blindedpay.c
cc plugins/offers.c
cc plugins/offers_offer.c
cc plugins/offers_invreq_hook.c
cc plugins/offers_inv_hook.c
cc plugins/pay.c
cc plugins/txprepare.c
cc plugins/spender/fundchannel.c
cc plugins/spender/main.c
cc plugins/spender/multifundchannel.c
cc plugins/spender/multiwithdraw.c
cc plugins/spender/openchannel.c
cc plugins/sql.c
cc plugins/bkpr/account.c
plugins/spender/openchannel.c:120:28: error: no member named 'sequence' in 'struct wally_psbt_input'
                    input->sequence,
                    ~~~~~  ^
plugins/spender/openchannel.c:183:37: error: no member named 'script' in 'struct wally_psbt_output'
                psbt_append_output(clone, output->script, amount_sat(output->amount));
                                          ~~~~~~  ^
plugins/spender/openchannel.c:183:64: error: no member named 'amount' in 'struct wally_psbt_output'
                psbt_append_output(clone, output->script, amount_sat(output->amount));
                                                                     ~~~~~~  ^
plugins/spender/openchannel.c:648:29: error: no member named 'script' in 'struct wally_psbt_output'
                                  mfc->psbt->outputs[j].script,
                                  ~~~~~~~~~~~~~~~~~~~~~ ^
plugins/spender/openchannel.c:649:29: error: no member named 'script_len' in 'struct wally_psbt_output'
                                  mfc->psbt->outputs[j].script_len))
                                  ~~~~~~~~~~~~~~~~~~~~~ ^
5 errors generated.
gmake[1]: *** [Makefile:292: plugins/spender/openchannel.o] Error 1
gmake[1]: *** Waiting for unfinished jobs....
cc plugins/bkpr/account_entry.c
gmake[1]: Leaving directory '/usr/ports/net-p2p/c-lightning/work/lightning-23.05.1'
===> Compilation failed unexpectedly.
Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to
the maintainer.
*** Error code 1

Stop.
make: stopped in /usr/ports/net-p2p/c-lightning
ghost commented 1 year ago

Patch for /usr/ports/net-p2p/c-lightning/Makefile

3,4c3,4
< PORTVERSION=  0.11.2
< DISTVERSIONPREFIX=    v
---
> #PORTVERSION= 0.11.2
> #DISTVERSIONPREFIX=   v
6,7c6,7
< #PORTVERSION= 0
< #PKGNAMESUFFIX=       -git-f8d8348c
---
> PORTVERSION=  0
> PKGNAMESUFFIX=        v23.05.1

Patch for /usr/ports/net-p2p/c-lightning/distinfo

13a14,16
> SHA256 (ElementsProject-lightning-0-v23.05.1_GH0.tar.gz) = 2b0697f1fc079acc2401c862089e8141e9b3289713a432c978332605ef3cc3c8
> SIZE (ElementsProject-lightning-0-v23.05.1_GH0.tar.gz) = 4509537
>
ghost commented 1 year ago

I'm currently redoing the build to verify the results from yesterday… the exact steps I took are as follows on a clean FreeBSD 13.2-RELEASE system (new jail). I'm expecting this to fail with a build error for lowdown, but I believe I fixed that yesterday by manually installing the lowdown pkg before building. This is just to verify everything so I don't lead anyone down on a false path.

Step 0: Prepare environment

(Not really relevant to this, but included for completeness)

# Switch from quarterly to latest pkg repo
sed "s/quarterly/latest/" /etc/pkg/FreeBSD.conf > /usr/local/etc/pkg/repos/FreeBSD.conf
env ASSUME_ALWAYS_YES=YES pkg update && pkg upgrade -y

# Install some base packages
pkg install -y ca_root_nss git vim zsh
# Switch from csh to zsh
chsh -s /usr/local/bin/zsh root

# Install ports
portsnap fetch
portsnap extract

Step 1: Patch

Apply the patches from the previous comment onto the port. This means updating the Makefile to use the git tag for the current release, as well as adding the SHA256 checksum for the 23.05.1 release tar.gz file.

Step 2: Follow the build instructions

Instructions found at: c-lightning docs INSTALL.md

cd /usr/ports/net-p2p/c-lightning

# Add BATCH=yes to accept all the default build options so I don't have to babysit the build process
# This is a kind of lengthy build since it has to build all the dependencies
make install BATCH=yes
ghost commented 1 year ago

The build process failed as I anticipated. Including the relevant log section here. However, I believe this fail is not related to c-lightning, but rather something with the lowdown port.

…
===>   Returning to build of c-lightningv23.05.1-0
===>   c-lightningv23.05.1-0 depends on shared library: libintl.so - found (/usr/local/lib/libintl.so)
===>  Configuring for c-lightningv23.05.1-0
CSANFLAGS =
Compiling ccan/tools/configurator/configurator...done
Making autoconf users comfortable... yes
checking for off_t is 32 bits... no
checking for __alignof__ support... yes
checking for asprintf() declaration... yes
checking for __attribute__((cold)) support... yes
checking for __attribute__((const)) support... yes
checking for __attribute__((deprecated)) support... no
checking for __attribute__((nonnull)) support... yes
checking for __attribute__((returns_nonnull)) support... yes
checking for __attribute__((sentinel)) support... yes
checking for __attribute__((pure)) support... yes
checking for __attribute__((may_alias)) support... yes
checking for __attribute__((noreturn)) support... yes
checking for __attribute__ format printf support... yes
checking for __attribute__((unused)) support... yes
checking for __attribute__((used)) support... yes
checking for backtrace() in <execinfo.h>... yes
checking for big endian... no
checking for <byteswap.h>... yes
checking for bswap64 in byteswap.h... yes
checking for __builtin_choose_expr support... yes
checking for __builtin_clz support... yes
checking for __builtin_clzl support... yes
checking for __builtin_clzll support... yes
checking for __builtin_ctz support... yes
checking for __builtin_ctzl support... yes
checking for __builtin_ctzll support... yes
checking for __builtin_constant_p support... yes
checking for __builtin_expect support... yes
checking for __builtin_ffs support... yes
checking for __builtin_ffsl support... yes
checking for __builtin_ffsll support... yes
checking for __builtin_popcount support... yes
checking for __builtin_popcountl support... yes
checking for __builtin_popcountll support... yes
checking for __builtin_types_compatible_p support... yes
checking for <intrinsics.h>... no
checking for struct timespec declaration... yes
checking for clock_gettime() declaration... yes
checking for compound literal support... yes
checking for fchdir support... yes
checking for <err.h>... yes
checking for for loop declaration support... yes
checking for flexible array member support... yes
checking for getpagesize() in <unistd.h>... yes
checking for isblank() in <ctype.h>... yes
checking for little endian... yes
checking for memmem in <string.h>... yes
checking for memrchr in <string.h>... yes
checking for mmap() declaration... yes
checking for /proc/self/maps exists... no
checking for qsort_r cmp takes trailing arg... no
checking for __attribute__((section)) and __start/__stop... yes
checking for stack grows upwards... no
checking for statement expression support... yes
checking for <sys/filio.h>... yes
checking for <sys/termios.h>... no
checking for <sys/unistd.h>... yes
checking for __typeof__ support... yes
checking for unaligned access to int... yes
checking for utime() declaration... yes
checking for __attribute__((warn_unused_result))... yes
checking for #pragma omp and -fopenmp support... yes
checking for <valgrind/memcheck.h>... no
checking for working <ucontext.h... yes
checking for passing pointers via makecontext()... no
checking for __builtin_cpu_supports()... yes
checking for closefrom() offered by system... yes
checking for F_CLOSEM defined for fctnl.... no
checking for close_range syscall available as __NR_close_range.... no
checking for F_MAXFD defined for fcntl.... no
checking for libsodium with IETF chacha20 variants... yes
checking for sqlite3_expanded_sql... yes
checking for sqlite3... yes
checking for postgres... no
checking for compiler is GCC... no
checking for GCC version is 7 or above... no
checking for pwritev() defined... yes
Writing variables to config.vars.97028... yes
Writing header to ccan/config.h.97028... yes
checking for python3-mako... found
checking for lowdown... not found
checking for sha256sum... found
Setting PREFIX... /usr/local
Setting CC... cc
Setting CONFIGURATOR_CC... cc
Setting CWARNFLAGS... -Wall -Wundef -Wmissing-prototypes -Wmissing-declarations -Wstrict-prototypes -Wold-style-definition -Werror
Setting CDEBUGFLAGS... -std=gnu11 -g -fstack-protector-strong
Setting COPTFLAGS... -Og
CSANFLAGS not found
Setting SQLITE3_CFLAGS... -I/usr/local/include
Setting SQLITE3_LDLIBS... -L/usr/local/lib -lsqlite3
POSTGRES_INCLUDE not found
POSTGRES_LDLIBS not found
Setting VALGRIND... 0
Setting DEVELOPER... 0
Setting EXPERIMENTAL_FEATURES... 0
Setting COMPAT... 1
Setting PYTEST... python3 -m pytest
Setting STATIC... 0
Setting ASAN... 0
Setting UBSAN... 0
Setting TEST_NETWORK... regtest
Setting HAVE_PYTHON3_MAKO... 1
Setting HAVE_LOWDOWN... 0
Setting SHA256SUM... sha256sum
Setting FUZZING... 0
Setting RUST... 0
===>  Building for c-lightningv23.05.1-0
gmake[1]: Entering directory '/usr/ports/net-p2p/c-lightning/work/lightning-23.05.1'
CC: cc -DBINTOPKGLIBEXECDIR="../libexec/c-lightning" -Wall -Wundef -Wmissing-prototypes -Wmissing-declarations -Wstrict-prototypes -Wold-style-definition -Werror -std=gnu11 -g -fstack-protector-strong -Og -I ccan -I external/libwally-core/include/ -I external/libwally-core/src/secp256k1/include/ -I external/jsmn/ -I external/libbacktrace/ -I external/gheap/ -I external/x86_64-unknown-freebsd13.2/libbacktrace-build -I . -I/usr/local/include -I/usr/local/include      -DSHACHAIN_BITS=48 -DJSMN_PARENT_LINKS  -DCOMPAT_V052=1 -DCOMPAT_V060=1 -DCOMPAT_V061=1 -DCOMPAT_V062=1 -DCOMPAT_V070=1 -DCOMPAT_V072=1 -DCOMPAT_V073=1 -DCOMPAT_V080=1 -DCOMPAT_V081=1 -DCOMPAT_V082=1 -DCOMPAT_V090=1 -DCOMPAT_V0100=1 -DCOMPAT_V0121=1  -DBUILD_ELEMENTS=1 -c -o
wiregen hsmd/hsmd_wiregen.h
LD: cc  -fstack-protector-strong    -Og  config.vars  -Lexternal/x86_64-unknown-freebsd13.2 -lwallycore -lsecp256k1 -ljsmn -lbacktrace -L/usr/local/include -lm -lgmp -L/usr/local/lib -lsqlite3  -lz  -lsodium -o
cc ccan/ccan/cdump/tools/cdump-enumstr.c
cc ccan/ccan/cdump/cdump.c
cc ccan/ccan/strmap/strmap.c
cc ccan/ccan/asort/asort.c
cc ccan/ccan/base64/base64.c
cc ccan/ccan/bitmap/bitmap.c
cc ccan/ccan/bitops/bitops.c
cc ccan/ccan/breakpoint/breakpoint.c
cc ccan/ccan/closefrom/closefrom.c
cc ccan/ccan/crc32c/crc32c.c
cc ccan/ccan/crypto/hmac_sha256/hmac_sha256.c
cc ccan/ccan/crypto/hkdf_sha256/hkdf_sha256.c
cc ccan/ccan/crypto/ripemd160/ripemd160.c
cc ccan/ccan/crypto/sha256/sha256.c
cc ccan/ccan/crypto/shachain/shachain.c -DSHACHAIN_BITS=48
cc ccan/ccan/crypto/siphash24/siphash24.c
cc ccan/ccan/err/err.c
cc ccan/ccan/fdpass/fdpass.c
cc ccan/ccan/htable/htable.c
cc ccan/ccan/ilog/ilog.c
cc ccan/ccan/io/io.c
cc ccan/ccan/intmap/intmap.c
cc ccan/ccan/io/poll.c
cc ccan/ccan/io/fdpass/fdpass.c
cc ccan/ccan/isaac/isaac.c
cc ccan/ccan/isaac/isaac64.c
cc ccan/ccan/json_escape/json_escape.c
cc ccan/ccan/json_out/json_out.c
cc ccan/ccan/list/list.c
cc ccan/ccan/mem/mem.c
cc ccan/ccan/membuf/membuf.c
cc ccan/ccan/noerr/noerr.c
cc ccan/ccan/opt/helpers.c
cc ccan/ccan/opt/parse.c
cc ccan/ccan/opt/usage.c
cc ccan/ccan/opt/opt.c
cc ccan/ccan/pipecmd/pipecmd.c
cc ccan/ccan/ptr_valid/ptr_valid.c
cc ccan/ccan/rbuf/rbuf.c
cc ccan/ccan/read_write_all/read_write_all.c
cc ccan/ccan/rune/coding.c
cc ccan/ccan/rune/rune.c
cc ccan/ccan/str/base32/base32.c
cc ccan/ccan/str/hex/hex.c
cc ccan/ccan/str/str.c
cc ccan/ccan/strset/strset.c
cc ccan/ccan/take/take.c
cc ccan/ccan/tal/grab_file/grab_file.c
cc ccan/ccan/tal/link/link.c
cc ccan/ccan/tal/path/path.c
cc ccan/ccan/tal/str/str.c
cc ccan/ccan/tal/tal.c
cc ccan/ccan/time/time.c
cc ccan/ccan/timer/timer.c
cc ccan/ccan/utf8/utf8.c
wiregen common/status_wiregen.h
wiregen common/peer_status_wiregen.h
wiregen common/scb_wiregen.h
wiregen wire/peer_wiregen.h
wiregen wire/onion_wiregen.h
wiregen wire/bolt12_wiregen.h
wiregen wire/channel_type_wiregen.h
printgen wire/bolt12_printgen.h
printgen wire/peer_printgen.h
printgen wire/onion_printgen.h
cc tools/headerversions.c
Version updated
printgen wire/channel_type_printgen.h
wiregen gossipd/gossipd_wiregen.h
wiregen gossipd/gossipd_peerd_wiregen.h
wiregen gossipd/gossip_store_wiregen.h
wiregen openingd/openingd_wiregen.h
wiregen openingd/dualopend_wiregen.h
wiregen channeld/channeld_wiregen.h
wiregen closingd/closingd_wiregen.h
wiregen onchaind/onchaind_wiregen.h
tools/generate-wire.py --page header tools/test/test_gen.h test_type < tools/test/test_cases > tools/test/test_gen.h
wiregen connectd/connectd_wiregen.h
wiregen connectd/connectd_gossipd_wiregen.h
tools/generate-wire.py -P --page header tools/test/print_gen.h test_type < tools/test/test_cases > tools/test/print_gen.h
GEN plugins/list_of_builtin_plugins_gen.h
wiregen hsmd/hsmd_wiregen.c
wiregen common/status_wiregen.c
wiregen wire/peer_wiregen.c
wiregen wire/channel_type_wiregen.c
wiregen wire/onion_wiregen.c
wiregen gossipd/gossipd_wiregen.c
wiregen gossipd/gossipd_peerd_wiregen.c
wiregen gossipd/gossip_store_wiregen.c
wiregen connectd/connectd_gossipd_wiregen.c
wiregen openingd/openingd_wiregen.c
wiregen common/peer_status_wiregen.c
wiregen channeld/channeld_wiregen.c
wiregen openingd/dualopend_wiregen.c
wiregen onchaind/onchaind_wiregen.c
wiregen closingd/closingd_wiregen.c
wiregen connectd/connectd_wiregen.c
xgettext wallet/statements_gettextgen.po
wiregen common/scb_wiregen.c
wiregen wire/bolt12_wiregen.c
printgen wire/onion_printgen.c
printgen wire/peer_printgen.c
printgen wire/bolt12_printgen.c
GEN plugins/sql-schema_gen.h
printgen wire/channel_type_printgen.c
tools/generate-wire.py --page impl tools/test/test_gen.h test_type < tools/test/test_cases > tools/test/test_gen.c
xgettext plugins/bkpr/statements_gettextgen.po
echo '#include "test_gen.h"' > tools/test/print_gen.c
cd external/lowdown && CC="cc" ./configure PREFIX=`pwd`/../../external/x86_64-unknown-freebsd13.2/lowdown-build/
tools/generate-wire.py -P --page impl tools/test/print_gen.h test_type < tools/test/test_cases >> tools/test/print_gen.c
/bin/sh: ./configure: not found
gmake[1]: *** [external/Makefile:118: external/x86_64-unknown-freebsd13.2/lowdown-build/bin/lowdown] Error 127
gmake[1]: *** Waiting for unfinished jobs....
gmake[1]: Leaving directory '/usr/ports/net-p2p/c-lightning/work/lightning-23.05.1'
===> Compilation failed unexpectedly.
Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to
the maintainer.
*** Error code 1

Stop.
make: stopped in /usr/ports/net-p2p/c-lightning
ghost commented 1 year ago

Step 3: Build once more

pkg install -y lowdown
make clean
make BATCH=yes

This fails with the following output (most of the output not included):

…
cc common/onionreply.c
cc common/permute_tx.c
cc common/psbt_open.c
cc common/pseudorand.c
common/psbt_open.c:390:26: error: no member named 'index' in 'struct wally_psbt_input'
                assert(psbt->inputs[i].index < input->utxo->num_outputs);
                       ~~~~~~~~~~~~~~~ ^
/usr/include/assert.h:56:21: note: expanded from macro 'assert'
#define assert(e)       ((e) ? (void)0 : __assert(__func__, __FILE__, \
                          ^
common/psbt_open.c:393:43: error: no member named 'index' in 'struct wally_psbt_input'
                                &input->utxo->outputs[psbt->inputs[i].index]);
                                                      ~~~~~~~~~~~~~~~ ^
common/psbt_open.c:394:19: error: implicit declaration of function 'wally_map_get_integer' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
                redeem_script = wally_map_get_integer(&psbt->inputs[i].psbt_fields, /* PSBT_IN_REDEEM_SCRIPT */ 0x04);
                                ^
common/psbt_open.c:394:58: error: no member named 'psbt_fields' in 'struct wally_psbt_input'
                redeem_script = wally_map_get_integer(&psbt->inputs[i].psbt_fields, /* PSBT_IN_REDEEM_SCRIPT */ 0x04);
                                                       ~~~~~~~~~~~~~~~ ^
4 errors generated.
gmake[1]: *** [Makefile:292: common/psbt_open.o] Error 1
rm external/x86_64-unknown-freebsd13.2/libwally-core-build/src/secp256k1/libsecp256k1.la
gmake[1]: Leaving directory '/usr/ports/net-p2p/c-lightning/work/lightning-23.05.1'
===> Compilation failed unexpectedly.
Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to
the maintainer.
*** Error code 1

Stop.
make: stopped in /usr/ports/net-p2p/c-lightning
ghost commented 1 year ago

I'm not able to replicate the exact errors from yesterday, but both failures mention wally_psbt_input. I gave the liberally-core repo a quick glance, and freebsd is not a supported platform. Right now I'm not sure how to proceed, will give it another go tomorrow.

Ending errors yesterday:

…
plugins/spender/openchannel.c:120:28: error: no member named 'sequence' in 'struct wally_psbt_input'
                    input->sequence
…

Ending errors today:

…
common/psbt_open.c:390:26: error: no member named 'index' in 'struct wally_psbt_input'
                assert(psbt->inputs[i].index < input->utxo->num_outputs);
…
vincenzopalazzo commented 1 year ago

I was able to compile with the following commands

pkg install git python py39-pip gmake libtool gmp sqlite3 \
            postgresql13-client gettext autotools
https://github.com/ElementsProject/lightning.git
pip install --upgrade pip
pip3 install mako
./configure
gmake -j$(nproc)
gmake install
grubles commented 1 year ago

@vincenzopalazzo You're missing autotools, which includes autoreconf, and a ./configure but otherwise those instructions work for me on FBSD 13.2-RELEASE.

vincenzopalazzo commented 1 year ago

Updated my comment, thanks @grubles

ghost commented 1 year ago

Excellent… then there is something funky going on with my environment. Thanks for the help, and sorry for wasting your time.

I'll post the solution once I figure it out.

ghost commented 1 year ago

I apologize for going awol, but I've now verified that it builds on my system as well.

The exact difference that made it fail on my end is still elusive. Now that I got working instructions I don't feel like fine combing my own script to find the key diff, however I ran at least one test build without the postgres pkg, which failed. So evidently just including the sqlite3 pkg isn't enough. I also suspect I was missing the autotool pkg, but that part I haven't verified.

Finally, I also contacted the freebsd port maintainer, and he responded that the build issues most likely are due to "old/incompatible git submodule" ( ports makefile diffs ) and he will investigate further once he is back from some traveling.

vincenzopalazzo commented 1 year ago

Nice, but I would like to remove it from the docs because we are not able to double-check if this FreeBSD port is tested by someone or maintained

vasild commented 1 year ago

I have updated the FreeBSD port to 23.05.1:

https://cgit.freebsd.org/ports/commit/?id=ac28bd03143a3fdee07377cfd66c3b81b1084eba

Sorry for not being more active :disappointed: