Gerbil-git fails to build on OpenBSD

[zmyrgel]

The git checkout of gerbil fails to build on OpenBSD. The build crashes on crypto compilation:

... compile os/socket
... compile os/kqueue
... compile os/signal
... compile os/signal-handler
... compile os/pid
... compile net/bio/file
... compile net/bio
... compile xml/ssax
... compile xml
... compile crypto/libcrypto
libcrypto__0.c:3615:10: fatal error: 'libcrypto-rfc5114.c' file not found
#include "libcrypto-rfc5114.c"
         ^~~~~~~~~~~~~~~~~~~~~
1 error generated.
*** ERROR IN ##main -- target compilation or link failed while compiling "/home/tmy/git/gerbil/lib/std/crypto/libcrypto__0.scm"
*** ERROR IN #:loop38665, "misc/concurrent-plan.ss"@90.7 -- Failure to build (gxc: "crypto/libcrypto" (submodules: ((static-include: "crypto/libcrypto-rfc5114.c"))) "-cc-options" "" "-ld-options" " -lcrypto" "-e" "(include \"~~lib/_gambit#.scm\")") "Compilation error; gsc exit with nonzero status"
*** ERROR; build failed
tmy@asteroid gerbil $

Same issue is with 0.17 version.

[zmyrgel]

Anu link to the patch? I don't recall seeing one in ports.

Timo

On Fri, Jul 1, 2022, at 21:39, Jaime Fournier wrote:

There is a patch in ports that fixes this, it just has not been pushed upstream.

— Reply to this email directly, view it on GitHub https://github.com/vyzo/gerbil/issues/669#issuecomment-1172618332, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAI72WSY2MGE67CVRNFUWTVR43OTANCNFSM5O4P7G2A. You are receiving this because you authored the thread.Message ID: @.***>

[vyzo]

Can we pull thrm intree pls?

[zmyrgel]

Unless I'm mistaken those patches are already committed to gerbil. They are still in ports tree as gerbil port can't be updated as compilation of 0.17 or newer does not pass. Also, the patches do not fix the compilation issue which this issue describes.

Timo

On Sat, Jul 2, 2022, at 00:32, vyzo wrote:

Can we pull thrm intree pls?

— Reply to this email directly, view it on GitHub https://github.com/vyzo/gerbil/issues/669#issuecomment-1172727024, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAI72RMKL2OJWJS7S6V27LVR5PXDANCNFSM5O4P7G2A. You are receiving this because you authored the thread.Message ID: @.***>

[vyzo]

can we fix the situation in tree so that we work on openbsd out of the box?

[zmyrgel]

Can you look to see if the file exists in your checkout? Are you using a newer version of Gambit?

Yes, the file exists in my checkout and the release tarball. Gambit seems to be version 4.9.4.

What are your build and configure flags to Gambit/Gerbil?

The actual configure flags probably aren't that relevant but can be looked up from ports Makefiles: https://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/lang/gambit/Makefile?rev=1.45&content-type=text/x-cvsweb-markup https://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/lang/gerbil/Makefile?rev=1.16&content-type=text/x-cvsweb-markup

This builds ok on NetBSD, which I primary use.

I didn't see gerbil in pkgsrc at all so I assume this is some local build and I assume you are making it compiling against openssl and not libressl?

Anyhow, the root issue to me seems to be Gerbil build system error. It uses the static-include in build-spec.ss to include the libcrypto-rfc5114.c but when actually building libcrypto it does not find it even though it is in the file system. This used to work in 0.16 so something changed in Gerbil build system since that which seems to have caused this.

[vyzo]

uh oh, @fare ping

[zmyrgel]

Are you using libressl?

I'm mostly interested in getting OpenBSD ports version updated which means using the system compiler and libraries.

Timo

On Sat, Jul 2, 2022, at 14:34, Jaime Fournier wrote:

It builds for me on Openbsd using gcc and gmake.

— Reply to this email directly, view it on GitHub https://github.com/vyzo/gerbil/issues/669#issuecomment-1172883602, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAI72XFJ2T7CD6SZ52PS2TVSASNDANCNFSM5O4P7G2A. You are receiving this because you authored the thread.Message ID: @.***>

[zmyrgel]

My build script https://gist.github.com/ober/69f258935010f961fbb3a98fe273fa05 The build log https://gist.github.com/ober/5251268cd081acdf55746dca99eeb7e4

Ok, I still get same "file not found" error when compiling gerbil with gcc or clang. Though I'm using the ports version of gambit. I tried to change the ports version of gambit to use ports-gcc but then gerbil build crashes in different error:

tmy@asteroid gerbil-0.17 $ ./configure --enable-libyaml                                                                                                                                       
configure: creating ./config.status
config.status: creating Makefile
tmy@asteroid gerbil-0.17 $ cd src
tmy@asteroid src $ mg build.sh                                                                                                                                                                
tmy@asteroid src $ CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib GAMBCOMP_VERBOSE=1 sh build.sh                             
[*] Building Gerbil
[*] Compiling gxi shim
[*] Building gerbil stage0
>>> preparing /home/tmy/tmp/gerbil-0.17/bootstrap
>>> compiling runtime
building gerbil/runtime in /home/tmy/tmp/gerbil-0.17/bootstrap/lib
... compile gx-gambc.scm
cc: error: unknown argument: '-mpc64'
cc: warning: optimization flag '-fschedule-insns2' is not supported [-Wignored-optimization-argument]
*** ERROR IN ##main -- target compilation or link failed while compiling "gx-gambc.scm"
*** ERROR IN ##thread-deadlock-action! -- Deadlock detected
>>> preparing bootstrap
>>> compiling gerbil core
... compile /home/tmy/tmp/gerbil-0.17/bootstrap/lib/gerbil/compiler/base__0.scm
cc: error: unknown argument: '-mpc64'
cc: warning: optimization flag '-fschedule-insns2' is not supported [-Wignored-optimization-argument]
*** ERROR IN ##main -- target compilation or link failed while compiling "/home/tmy/tmp/gerbil-0.17/bootstrap/lib/gerbil/compiler/base__0.scm"
*** ERROR IN ##thread-deadlock-action! -- Deadlock detected
*** ERROR; build failed

[zmyrgel]

Ah, the previous error was my mistake in gambit port Makefile, I changed the COMPILER but forgot to change CC parameter. I guess this resulted the compilation to use mix of system gcc (4.2.1) and ports-gcc (8.4.1). Fixed it and I compiled the gambit port with following diff:

diff 3c1d0e6f7c1a7453566310164ea13533318a7bd7 /usr/ports
blob - 38a5b472c77244c78da2a7604d10b69eb2d120f7
file + lang/gambit/Makefile
--- lang/gambit/Makefile
+++ lang/gambit/Makefile
@@ -3,7 +3,7 @@ BROKEN-riscv64= gsi and gsi-script segfault at startup
 COMMENT=       complete, efficient and reliable implementation of Scheme

 V=             4.9.4
-REVISION=      0
+REVISION=      1
 DISTNAME=      gambit-v${V:S/./_/g}
 PKGNAME=       gambit-$V
 API_V=         ${V:R:S/./0/}00${V:E}
@@ -23,7 +23,7 @@ EXTRACT_SUFX= .tgz

 WANTLIB +=     c crypto m ssl util

-COMPILER=      base-clang ports-gcc
+COMPILER=      ports-gcc
 COMPILER_LANGS=        c

 CONFIGURE_STYLE=gnu
@@ -34,14 +34,19 @@ CONFIGURE_ARGS+=--disable-cplusplus \
                --libdir=${PREFIX}/lib/gambit/ \
                --enable-c-opt="${CFLAGS}" \
                --enable-openssl \
-               --enable-poll
+               --enable-poll \
+               --enable-march='' \
+               --enable-dynamic-clib \
+               --enable-single-host

-CONFIGURE_ENV= CC=${CC} CXX=${CXX} CFLAGS="${CFLAGS}" CXXFLAGS="${CXXFLAGS}"
+CONFIGURE_ENV= CC=egcc CXX=eg++ CFLAGS="${CFLAGS}" CXXFLAGS="${CXXFLAGS}"

 WRKDIST=       ${WRKDIR}/gambit-$V

 MAKE_FILE=     ${WRKSRC}/makefile

+WRKDIST=       ${WRKDIR}/${PKGNAME}
+
 # On sparc64, gcc crashes with the default stack limit:
 # "cc: Internal error: Illegal instruction" on _gambc.c
 do-build:

With above the resulting gsc compiles code without spewing out the "unknown argument" error so I'm assuming it is working as expected.

Though with above the Gerbil compile still fails with "file not found" error when compiling libcrypto:

... compile net/bio/file
... compile net/bio
... compile xml/ssax
... compile xml
... compile crypto/libcrypto
libcrypto__0.c:3615:10: fatal error: libcrypto-rfc5114.c: No such file or directory
 #include "libcrypto-rfc5114.c"
          ^~~~~~~~~~~~~~~~~~~~~
compilation terminated.
*** ERROR IN ##main -- target compilation or link failed while compiling "/home/tmy/git/gerbil/lib/std/crypto/libcrypto__0.scm"
*** ERROR IN #:loop38665, "misc/concurrent-plan.ss"@90.7 -- Failure to build (gxc: "crypto/libcrypto" (submodules: ((static-include: "crypto/libcrypto-rfc5114.c"))) "-cc-options" "" "-ld-options" " -lcrypto" "-e" "(include \"~~lib/_gambit#.scm\")") "Compilation error; gsc exit with nonzero status"
*** ERROR; build failed

I was using git checkout of gerbil and compiled with command: CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib GAMBCOMP_VERBOSE=1 CC=egcc sh build.sh

[zmyrgel]

Now if you can just please test the --with-openssl=/usr/local

On where should that be given? Gambit has '--enable-openssl' but it does not take any argument. Gerbil does not seem to have that either.

I'm not using GERBIL_BUILD_CORES on my builds. I'm building gerbil in the src dir:

tmy@asteroid src $ CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib GAMBCOMP_VERBOSE=1 CC=egcc ./build.sh stdlib  
[*] Building gerbil stdlib
... compile build-config
... compile xml
... compile crypto/libcrypto
libcrypto__0.c:3615:10: fatal error: libcrypto-rfc5114.c: No such file or directory
 #include "libcrypto-rfc5114.c"
          ^~~~~~~~~~~~~~~~~~~~~
compilation terminated.
*** ERROR IN ##main -- target compilation or link failed while compiling "/home/tmy/git/gerbil/lib/std/crypto/libcrypto__0.scm"
*** ERROR IN #:loop3833, "misc/concurrent-plan.ss"@90.7 -- Failure to build (gxc: "crypto/libcrypto" (submodules: ((static-include: "crypto/libcrypto-rfc5114.c"))) "-cc-options" "" "-ld-options" " -lcrypto" "-e" "(include \"~~lib/_gambit#.scm\")") "Compilation error; gsc exit with nonzero status"
*** ERROR; build failed

[zmyrgel]

I've been building gerbil against ported gambit or custom compilation of gambit but I'm hitting the same "file not found" error on all cases.

I used the pretty much same build script as you, I changed paths and tweaked gerbil build as follows:

build-gerbil() {
    if [[ ! -d ${GERBIL_SRC} ]]; then
        git clone https://github.com/vyzo/gerbil ${GERBIL_SRC}
    fi
    #alias gcc=/usr/local/bin/egcc
    #ln -s /usr/local/bin/egcc /usr/local/bin/gcc||true
    cd ${GERBIL_SRC}/src

    git fetch -a && git clean -xfd && git reset --hard
    git checkout ${GERBIL_VERSION}
    sed -i 's/gcc/egcc/g' ${GERBIL_SRC}/src/build.sh
    ./configure \
        --prefix=${GERBIL_HOME} \
        --enable-libxml \
        --enable-libyaml
    ./build.sh
    ./install
}

With it I still get same error:

... compile xml/ssax
... compile xml
... compile crypto/libcrypto
libcrypto__0.c: In function 'ffi_create_HMAC_CTX':
libcrypto__0.c:3537:46: error: invalid application of 'sizeof' to incomplete type 'HMAC_CTX' {aka 'struct hmac_ctx_st'}
   HMAC_CTX *ctx = (HMAC_CTX*)malloc (sizeof (HMAC_CTX));
                                              ^~~~~~~~
libcrypto__0.c:3539:5: warning: implicit declaration of function 'HMAC_CTX_init'; did you mean 'HMAC_CTX_new'? [-Wimplicit-function-declaration]
     HMAC_CTX_init (ctx);
     ^~~~~~~~~~~~~
     HMAC_CTX_new
libcrypto__0.c: In function 'ffi_release_HMAC_CTX':
libcrypto__0.c:3550:3: warning: implicit declaration of function 'HMAC_CTX_cleanup'; did you mean 'HMAC_CTX_get_md'? [-Wimplicit-function-declaration]
   HMAC_CTX_cleanup ((HMAC_CTX*) ptr);
   ^~~~~~~~~~~~~~~~
   HMAC_CTX_get_md
libcrypto__0.c: In function 'ffi_DH_pub_key':
libcrypto__0.c:3601:11: error: dereferencing pointer to incomplete type 'DH' {aka 'struct dh_st'}
  return dh->pub_key;
           ^~
libcrypto__0.c: At top level:
libcrypto__0.c:3615:10: fatal error: libcrypto-rfc5114.c: No such file or directory
 #include "libcrypto-rfc5114.c"
          ^~~~~~~~~~~~~~~~~~~~~
compilation terminated.
*** ERROR IN ##main -- target compilation or link failed while compiling "/home/tmy/tmp/gerbil-src/lib/std/crypto/libcrypto__0.scm"
*** ERROR IN #:loop38665, "misc/concurrent-plan.ss"@90.7 -- Failure to build (gxc: "crypto/libcrypto" (submodules: ((static-include: "crypto/libcrypto-rfc5114.c"))) "-cc-options" "" "-ld-options" " -lcrypto" "-e" "(include \"~~lib/_gambit#.scm\")") "Compilation error; gsc exit with nonzero status"
*** ERROR; build failed
 $

Though I'd guess the HMAC errors are due to changes in libressl which shouldn't be there if the build was using openssl instead of libressl.

[chenrui333]

Any update on this?

[vyzo]

The problem is with make, should be fixed in #734.