release_0.8.5 build fails on OpenBSD 7.1

[grubles]

Seems libwally-core release_0.8.5 fails to build when building the Core Lightning project.

Arch: amd64 OS: OpenBSD 7.1 CLN: at commit ad3cbed7c28b2d4e9327885f6c3763f8f4acef41

configure: WARNING: cannot find 'swig' program. You should look at http://www.swig.org
In file included from ../../../../libwally-core/src/secp256k1/src/secp256k1.c:814:
In file included from ../../../../libwally-core/src/secp256k1/src/modules/extrakeys/main_impl.h:12:
../../../../libwally-core/src/secp256k1/src/modules/extrakeys/hsort_impl.h:26:55: error: too many arguments provided to function-like macro invocation
static SECP256K1_INLINE void swap64(unsigned char *a, size_t i, size_t j, size_t stride) {
                                                      ^
/usr/include/sys/endian.h:72:9: note: macro 'swap64' defined here
#define swap64(x) __swap64(x)
        ^
In file included from ../../../../libwally-core/src/secp256k1/src/secp256k1.c:814:
In file included from ../../../../libwally-core/src/secp256k1/src/modules/extrakeys/main_impl.h:12:
../../../../libwally-core/src/secp256k1/src/modules/extrakeys/hsort_impl.h:26:8: error: 'inline' can only appear on functions
static SECP256K1_INLINE void swap64(unsigned char *a, size_t i, size_t j, size_t stride) {
       ^
../../../../libwally-core/src/secp256k1/src/modules/extrakeys/../../../include/secp256k1.h:130:28: note: expanded from macro 'SECP256K1_INLINE'
#  define SECP256K1_INLINE inline
                           ^
In file included from ../../../../libwally-core/src/secp256k1/src/secp256k1.c:814:
In file included from ../../../../libwally-core/src/secp256k1/src/modules/extrakeys/main_impl.h:12:
../../../../libwally-core/src/secp256k1/src/modules/extrakeys/hsort_impl.h:26:30: error: variable has incomplete type 'void'
static SECP256K1_INLINE void swap64(unsigned char *a, size_t i, size_t j, size_t stride) {
                             ^
../../../../libwally-core/src/secp256k1/src/modules/extrakeys/hsort_impl.h:26:36: error: expected ';' after top level declarator
static SECP256K1_INLINE void swap64(unsigned char *a, size_t i, size_t j, size_t stride) {
                                   ^
                                   ;
../../../../libwally-core/src/secp256k1/src/modules/extrakeys/hsort_impl.h:36:35: error: too many arguments provided to function-like macro invocation
        swap64(a + (stride - 64), i, j, 64);
                                  ^
/usr/include/sys/endian.h:72:9: note: macro 'swap64' defined here
#define swap64(x) __swap64(x)
        ^
In file included from ../../../../libwally-core/src/secp256k1/src/secp256k1.c:814:
In file included from ../../../../libwally-core/src/secp256k1/src/modules/extrakeys/main_impl.h:12:
../../../../libwally-core/src/secp256k1/src/modules/extrakeys/hsort_impl.h:39:15: error: too many arguments provided to function-like macro invocation
    swap64(a, i, j, stride);
              ^
/usr/include/sys/endian.h:72:9: note: macro 'swap64' defined here
#define swap64(x) __swap64(x)
        ^
In file included from ../../../../libwally-core/src/secp256k1/src/secp256k1.c:814:
../../../../libwally-core/src/secp256k1/src/modules/extrakeys/main_impl.h:163:71: error: expected expression
    return secp256k1_xonly_pubkey_cmp(((secp256k1_xonly_sort_cmp_data*)cmp_data)->ctx,
                                                                      ^
../../../../libwally-core/src/secp256k1/src/modules/extrakeys/main_impl.h:163:41: error: use of undeclared identifier 'secp256k1_xonly_sort_cmp_data'; did you mean 'secp256k1_xonly_sort_cmp'?
    return secp256k1_xonly_pubkey_cmp(((secp256k1_xonly_sort_cmp_data*)cmp_data)->ctx,
                                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                        secp256k1_xonly_sort_cmp
../../../../libwally-core/src/secp256k1/src/modules/extrakeys/main_impl.h:162:12: note: 'secp256k1_xonly_sort_cmp' declared here
static int secp256k1_xonly_sort_cmp(const void* pk1, const void* pk2, void *cmp_data) {
           ^
../../../../libwally-core/src/secp256k1/src/modules/extrakeys/main_impl.h:162:77: warning: unused parameter 'cmp_data' [-Wunused-parameter]
static int secp256k1_xonly_sort_cmp(const void* pk1, const void* pk2, void *cmp_data) {
                                                                            ^
../../../../libwally-core/src/secp256k1/src/modules/extrakeys/main_impl.h:169:5: error: use of undeclared identifier 'secp256k1_xonly_sort_cmp_data'; did you mean 'secp256k1_xonly_sort_cmp'?
    secp256k1_xonly_sort_cmp_data cmp_data;
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    secp256k1_xonly_sort_cmp
../../../../libwally-core/src/secp256k1/src/modules/extrakeys/main_impl.h:162:12: note: 'secp256k1_xonly_sort_cmp' declared here
static int secp256k1_xonly_sort_cmp(const void* pk1, const void* pk2, void *cmp_data) {
           ^
../../../../libwally-core/src/secp256k1/src/modules/extrakeys/main_impl.h:169:34: error: expected ';' after expression
    secp256k1_xonly_sort_cmp_data cmp_data;
                                 ^
                                 ;
../../../../libwally-core/src/secp256k1/src/modules/extrakeys/main_impl.h:169:5: warning: expression result unused [-Wunused-value]
    secp256k1_xonly_sort_cmp_data cmp_data;
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../../../libwally-core/src/secp256k1/src/modules/extrakeys/main_impl.h:169:35: error: use of undeclared identifier 'cmp_data'
    secp256k1_xonly_sort_cmp_data cmp_data;
                                  ^
../../../../libwally-core/src/secp256k1/src/modules/extrakeys/main_impl.h:173:5: error: use of undeclared identifier 'cmp_data'
    cmp_data.ctx = ctx;
    ^
../../../../libwally-core/src/secp256k1/src/modules/extrakeys/main_impl.h:174:86: error: use of undeclared identifier 'cmp_data'
    secp256k1_hsort(pubkeys, n_pubkeys, sizeof(*pubkeys), secp256k1_xonly_sort_cmp, &cmp_data);
                                                                                     ^
../../../../libwally-core/src/secp256k1/src/modules/extrakeys/main_impl.h:268:11: warning: implicit declaration of function 'secp256k1_extrakeys_ge_even_y' is invalid in C99 [-Wimplicit-function-declaration]
    tmp = secp256k1_extrakeys_ge_even_y(&pk);
          ^
../../../../libwally-core/src/secp256k1/src/modules/extrakeys/main_impl.h:272:5: warning: implicit declaration of function 'secp256k1_xonly_pubkey_save' is invalid in C99 [-Wimplicit-function-declaration]
    secp256k1_xonly_pubkey_save(pubkey, &pk);
    ^
../../../../libwally-core/src/secp256k1/src/modules/extrakeys/main_impl.h:290:16: warning: implicit declaration of function 'secp256k1_extrakeys_ge_even_y' is invalid in C99 [-Wimplicit-function-declaration]
    y_parity = secp256k1_extrakeys_ge_even_y(&pk);
               ^
In file included from ../../../../libwally-core/src/secp256k1/src/secp256k1.c:818:
../../../../libwally-core/src/secp256k1/src/modules/schnorrsig/main_impl.h:244:10: warning: implicit declaration of function 'secp256k1_xonly_pubkey_load' is invalid in C99 [-Wimplicit-function-declaration]
    if (!secp256k1_xonly_pubkey_load(ctx, &pk, pubkey)) {
         ^
6 warnings and 13 errors generated.
gmake[5]: *** [Makefile:1336: src/libsecp256k1_la-secp256k1.lo] Error 1
gmake[4]: *** [Makefile:1036: all] Error 2
gmake[3]: *** [Makefile:1712: all-recursive] Error 1
gmake[2]: *** [Makefile:924: all] Error 2
gmake[1]: *** [Makefile:435: all-recursive] Error 1
gmake: *** [external/Makefile:77: external/amd64-unknown-openbsd7.1/libwally-core-build/src/libwallycore.la] Error 2

[jgriffiths]

@grubles

Can you confirm the libsecp256k1 patch here: https://pastebin.com/yDGxA1wK fixes this? If so I will submit to secp upstream.

[jgriffiths]

secp PR: https://github.com/ElementsProject/secp256k1-zkp/pull/194

[grubles]

Looks like secp256k1-zkp with that patch builds and passes gmake check.

openbsd71$ gmake check
gmake  check-am
gmake[1]: Entering directory '/home/user/secp256k1-zkp'
gmake  check-TESTS
gmake[2]: Entering directory '/home/user/secp256k1-zkp'
gmake[3]: Entering directory '/home/user/secp256k1-zkp'
PASS: tests
PASS: exhaustive_tests
============================================================================
Testsuite summary for libsecp256k1 0.1.0-pre
============================================================================
# TOTAL: 2
# PASS:  2
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================
gmake[3]: Leaving directory '/home/user/secp256k1-zkp'
gmake[2]: Leaving directory '/home/user/secp256k1-zkp'
gmake[1]: Leaving directory '/home/user/secp256k1-zkp'

[jgriffiths]

@grubles does it fail just building secp without that patch? That would be very strange.

[grubles]

Building secp256k1-zkp master (without the patch) succeeds.

@jgriffiths Is there a specific commit libwally uses? I could try building that too.

[jgriffiths]

@jgriffiths Is there a specific commit libwally uses? I could try building that too.

No need thanks, the problem is with including the secp sources for building, discussed in detail on the linked secp-zkp PR.

If you patch the c-lightning wally with the below does it build?

diff --git a/src/wrap_js/src/combined.c b/src/wrap_js/src/combined.c
index 6a7d90b8..c69595b5 100644
--- a/src/wrap_js/src/combined.c
+++ b/src/wrap_js/src/combined.c
@@ -35,6 +35,7 @@
 #undef PACKAGE_VERSION
 #undef VERSION
 #undef SECP256K1_BUILD
+#undef swap64
 #include "src/secp256k1/src/secp256k1.c"
 #include "src/secp256k1/src/precomputed_ecmult_gen.c"
 #include "src/secp256k1/src/precomputed_ecmult.c"

[grubles]

I get the same build error using that patch.

[jgriffiths]

Hmm, same error when building wally, or wally builds and then the same error happens in libhsm?

if you can get a full verbose log of the failure building wally (make V=1) and pastebin it somewhere, we might be able to workaround this faster then waiting for secp to merge the more reliable fix.

[grubles]

Ahh ok. Before I was just running gmake from the CLN repo directory and running into that build error. This time I've tried building from within /external/libwally-core and it succeeded with the combined.c patch!

[jgriffiths]

OK, you can do the same in c-ln in contrib/libhsmd_python/setup.py temporarily to build.

libsecp has just taken the patch to prevent the name clash, so I will update it in wally master shortly and that should fix the builds without needing the #undef.

[jgriffiths]

@grubles master is updated, please re-open if you still have issues, thanks