search

RealTimeLogic / BAS

Embedded Web Server Library with Integrated Scripting Engine
https://realtimelogic.com/products/barracuda-application-server/
GNU General Public License v2.0
58 stars 14 forks source link

BAS segfaults when build on a musl libc based system #10

Closed mterron closed 1 year ago

mterron commented 1 year ago

Trying to bootstrap BAS/mako on a musl libc based system (musl.libc.org) successfully compiles but the binary segfaults immediately.

~ # ./LinuxBuild.sh
Setting default compiler
Using compiler gcc
~/BAS-Resources/build ~/BAS
Building mako.zip
../../../lua-protobuf not found; Not Including lua-protobuf and Sparkplug
../../../LPeg not found; Not Including LPeg
~/BAS
Compiling using gcc; this may take some time........
Done
You may now run ./mako
~ # ./mako

Mako Server. Version 3.9
BAS lib 5453. Build date: Jun  6 2023
Copyright (c) Real Time Logic.

Segmentation fault
~ # ./mako -h

Mako Server. Version 3.9
BAS lib 5453. Build date: Jun  6 2023
Copyright (c) Real Time Logic.

Usage: mako [options]

Options:
 -l[name]:[priority]:app  - Load one or multiple applications
 -c configfile            - Load configuration file
 -? -h                    - print this help message
 -d                       - Run in daemon mode by detatching from the console
 -s                       - Run in daemon mode without detatching from the console
 -u username              - Username to run as
~ #
surfskidude commented 1 year ago

So I went ahead and compiled Mako using the ready-to-use distribution from https://sabotage-linux.github.io/

I was able to compile the code "as is" using the following commands:

git clone https://github.com/RealTimeLogic/BAS.git cd /tmp/ wget http://www.sqlite.org/2022/sqlite-amalgamation-3400000.zip unzip sqlite-amalgamation-3400000.zip cd cd BAS/src/ cp /tmp/sqlite-amalgamation-3400000/* . cd .. nano mako.mk Change CFLAGS to:

  CFLAGS += -fmerge-all-constants -O3 -Os -Wall -std=gnu99 -DUSE_FORKPTY=0

Comment out building mako.zip (distribution is missing unzip and is unable to run bash scripts)

mako.zip:
# Cannot build mako.zip (missing tools)
#       cd ../BAS-Resources/build&&./mako.sh
#       cp ../BAS-Resources/build/mako.zip .

Mako can now be built:

make -f mako.mk

Since we are missing mako.zip, fetch it from any of the pre-compiled versions.

cd /tmp/ wget makoserver.net/download/mako.linux-x32.tar.gz tar xvzf mako.linux-x32.tar.gz cp mako.zip /root/BAS/

We can now run mako

~/BAS # ./mako

Mako Server. Version 3.9 BAS lib 5453. Build date: Jun 6 2023 Copyright (c) Real Time Logic.

Mounting /root/BAS/mako.zip Server listening on IPv6 port 80 Server listening on IPv4 port 80 Loading certificate MakoServer SharkSSL server listening on IPv6 port 443 SharkSSL server listening on IPv4 port 443

mterron commented 1 year ago

Does not work on Alpine (arguably the best known musl based distro) after installing all dependencies though.

Here's a repro Dockerfile

FROM alpine
RUN echo '::::: DOWNLOADING DEPENDENCIES :::::' \
    && apk add libc-dev gcc git zip bash make file \
    && git clone https://github.com/RealTimeLogic/BAS.git \
    && git clone https://github.com/RealTimeLogic/BAS-Resources.git \
    && chmod +x BAS-Resources/build/*.sh \
    && cd /tmp/ \
    && wget http://www.sqlite.org/2022/sqlite-amalgamation-3400000.zip \
    && unzip sqlite-amalgamation-3400000.zip \
    && cp /tmp/sqlite-amalgamation-3400000/* /BAS/src/

RUN echo '::::: STARTING BUILD :::::' \
    && cd /BAS \
    && sed -i '30s/.*/CFLAGS += -fmerge-all-constants -O3 -Os -Wall -std=gnu99 -DUSE_FORKPTY=0/' mako.mk \
    # Will use prebuilt mako.zip
    && touch mako.zip \
    && make -f mako.mk \
    && file mako \
    && ldd mako 

RUN echo '::::: ADDING mako.zip :::::' \
    && cd /tmp/ \
    && wget makoserver.net/download/mako.linux-x32.tar.gz \
    && tar xvzf mako.linux-x32.tar.gz \
    && cp mako.zip /BAS/

  RUN /BAS/mako

The output is:

/ # id
uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel),11(floppy),20(dialout),26(tape),27(video)
/ # docker build -progress=plain .
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 1.02kB done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load metadata for docker.io/library/alpine:latest
#3 DONE 0.0s

#4 [1/4] FROM docker.io/library/alpine
#4 CACHED

#5 [2/4] RUN echo '::::: DOWNLOADING DEPENDENCIES :::::'        && apk add libc-dev gcc git zip bash make file    && git clone https://github.com/RealTimeLogic/BAS.git   && git clone https://github.com/RealTimeLogic/BAS-Resources.git   && chmod +x BAS-Resources/build/*.sh    && cd /tmp/     && wget http://www.sqlite.org/2022/sqlite-amalgamation-3400000.zip        && unzip sqlite-amalgamation-3400000.zip        && cp /tmp/sqlite-amalgamation-3400000/* /BAS/src/
#5 0.418 ::::: DOWNLOADING DEPENDENCIES :::::
#5 0.426 fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64/APKINDEX.tar.gz
#5 1.456 fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/community/x86_64/APKINDEX.tar.gz
#5 2.796 (1/31) Installing ncurses-terminfo-base (6.4_p20230506-r0)
#5 2.861 (2/31) Installing libncursesw (6.4_p20230506-r0)
#5 2.989 (3/31) Installing readline (8.2.1-r1)
#5 3.089 (4/31) Installing bash (5.2.15-r5)
#5 3.371 Executing bash-5.2.15-r5.post-install
#5 3.376 (5/31) Installing libmagic (5.44-r3)
#5 3.693 (6/31) Installing file (5.44-r3)
#5 3.786 (7/31) Installing libgcc (12.2.1_git20220924-r10)
#5 3.867 (8/31) Installing libstdc++ (12.2.1_git20220924-r10)
#5 4.559 (9/31) Installing zstd-libs (1.5.5-r4)
#5 4.939 (10/31) Installing binutils (2.40-r7)
#5 11.40 (11/31) Installing libgomp (12.2.1_git20220924-r10)
#5 11.84 (12/31) Installing libatomic (12.2.1_git20220924-r10)
#5 11.91 (13/31) Installing gmp (6.2.1-r3)
#5 12.52 (14/31) Installing isl26 (0.26-r1)
#5 14.42 (15/31) Installing mpfr4 (4.2.0-r3)
#5 15.17 (16/31) Installing mpc1 (1.3.1-r1)
#5 15.34 (17/31) Installing gcc (12.2.1_git20220924-r10)
#5 111.9 (18/31) Installing ca-certificates (20230506-r0)
#5 112.2 (19/31) Installing brotli-libs (1.0.9-r14)
#5 113.1 (20/31) Installing libunistring (1.1-r1)
#5 113.9 (21/31) Installing libidn2 (2.3.4-r1)
#5 114.1 (22/31) Installing nghttp2-libs (1.53.0-r0)
#5 114.2 (23/31) Installing libcurl (8.1.2-r0)
#5 114.7 (24/31) Installing libexpat (2.5.0-r1)
#5 114.8 (25/31) Installing pcre2 (10.42-r1)
#5 115.1 (26/31) Installing git (2.40.1-r0)
#5 120.1 (27/31) Installing musl-dev (1.2.4-r0)
#5 126.6 (28/31) Installing libc-dev (0.7.2-r5)
#5 126.7 (29/31) Installing make (4.4.1-r1)
#5 127.1 (30/31) Installing unzip (6.0-r14)
#5 127.4 (31/31) Installing zip (3.0-r12)
#5 127.8 Executing busybox-1.36.0-r9.trigger
#5 127.8 Executing ca-certificates-20230506-r0.trigger
#5 127.9 OK: 172 MiB in 46 packages
#5 128.0 Cloning into 'BAS'...
#5 146.9 Cloning into 'BAS-Resources'...
#5 149.3 Connecting to www.sqlite.org (45.33.6.223:80)
#5 149.6 saving to 'sqlite-amalgamation-3400000.zip'
#5 150.4 sqlite-amalgamation-   7% |**                              |  185k  0:00:12 ETA
#5 151.5 sqlite-amalgamation-  29% |*********                       |  737k  0:00:04 ETA
#5 152.4 sqlite-amalgamation-  51% |****************                | 1303k  0:00:02 ETA
#5 153.3 sqlite-amalgamation-  75% |************************        | 1923k  0:00:01 ETA
#5 154.2 sqlite-amalgamation- 100% |********************************| 2534k  0:00:00 ETA
#5 154.2 'sqlite-amalgamation-3400000.zip' saved
#5 154.2 Archive:  sqlite-amalgamation-3400000.zip
#5 154.2    creating: sqlite-amalgamation-3400000/
#5 154.2   inflating: sqlite-amalgamation-3400000/sqlite3.c
#5 154.3   inflating: sqlite-amalgamation-3400000/shell.c
#5 154.3   inflating: sqlite-amalgamation-3400000/sqlite3.h
#5 154.3   inflating: sqlite-amalgamation-3400000/sqlite3ext.h
#5 DONE 154.6s

#6 [3/4] RUN echo '::::: STARTING BUILD :::::'  && cd /BAS      && sed -i '30s/.*/CFLAGS += -fmerge-all-constants -O3 -Os -Wall -std=gnu99 -DUSE_FORKPTY=0/' mako.mk    && touch mako.zip  && make -f mako.mk      && file mako    && ldd mako
#6 0.535 ::::: STARTING BUILD :::::
#6 0.539 Including SQLite
#6 0.539 Excluding LPEG
#6 0.539 Excluding Lua Protobuf
#6 0.539 Excluding Lua MyCustomBindings example
#6 0.542 cc -fmerge-all-constants -O3 -Os -Wall -std=gnu99 -DUSE_FORKPTY=0 -DMAKO -DUSE_EMBEDDED_ZIP=0 -DLUA_USE_LINUX -DBA_FILESIZE64 -DUSE_LUAINTF=1 -DUSE_DBGMON=1 -DUSE_OPCUA=1 -Iinc -Iinc/arch/Posix -Iinc/arch/NET/Posix -c -o BAS.o src/BAS.c
#6 61.95 cc -fmerge-all-constants -O3 -Os -Wall -std=gnu99 -DUSE_FORKPTY=0 -DMAKO -DUSE_EMBEDDED_ZIP=0 -DLUA_USE_LINUX -DBA_FILESIZE64 -DUSE_LUAINTF=1 -DUSE_DBGMON=1 -DUSE_OPCUA=1 -Iinc -Iinc/arch/Posix -Iinc/arch/NET/Posix -c -o ThreadLib.o src/arch/Posix/ThreadLib.c
#6 62.09 cc -fmerge-all-constants -O3 -Os -Wall -std=gnu99 -DUSE_FORKPTY=0 -DMAKO -DUSE_EMBEDDED_ZIP=0 -DLUA_USE_LINUX -DBA_FILESIZE64 -DUSE_LUAINTF=1 -DUSE_DBGMON=1 -DUSE_OPCUA=1 -Iinc -Iinc/arch/Posix -Iinc/arch/NET/Posix -c -o SoDisp.o src/arch/NET/generic/SoDisp.c
#6 62.41 cc -fmerge-all-constants -O3 -Os -Wall -std=gnu99 -DUSE_FORKPTY=0 -DMAKO -DUSE_EMBEDDED_ZIP=0 -DLUA_USE_LINUX -DBA_FILESIZE64 -DUSE_LUAINTF=1 -DUSE_DBGMON=1 -DUSE_OPCUA=1 -Iinc -Iinc/arch/Posix -Iinc/arch/NET/Posix -c -o BaFile.o src/DiskIo/posix/BaFile.c
#6 62.80 cc -fmerge-all-constants -O3 -Os -Wall -std=gnu99 -DUSE_FORKPTY=0 -DMAKO -DUSE_EMBEDDED_ZIP=0 -DLUA_USE_LINUX -DBA_FILESIZE64 -DUSE_LUAINTF=1 -DUSE_DBGMON=1 -DUSE_OPCUA=1 -Iinc -Iinc/arch/Posix -Iinc/arch/NET/Posix -c -o MakoMain.o examples/MakoServer/src/MakoMain.c
#6 63.49 cc -fmerge-all-constants -O3 -Os -Wall -std=gnu99 -DUSE_FORKPTY=0 -DMAKO -DUSE_EMBEDDED_ZIP=0 -DLUA_USE_LINUX -DBA_FILESIZE64 -DUSE_LUAINTF=1 -DUSE_DBGMON=1 -DUSE_OPCUA=1 -Iinc -Iinc/arch/Posix -Iinc/arch/NET/Posix -c -o ls_sqlite3.o src/ls_sqlite3.c
#6 64.26 cc -fmerge-all-constants -O3 -Os -Wall -std=gnu99 -DUSE_FORKPTY=0 -DMAKO -DUSE_EMBEDDED_ZIP=0 -DLUA_USE_LINUX -DBA_FILESIZE64 -DUSE_LUAINTF=1 -DUSE_DBGMON=1 -DUSE_OPCUA=1 -Iinc -Iinc/arch/Posix -Iinc/arch/NET/Posix -c -o luasql.o src/luasql.c
#6 64.37 cc -fmerge-all-constants -O3 -Os -Wall -std=gnu99 -DUSE_FORKPTY=0 -DMAKO -DUSE_EMBEDDED_ZIP=0 -DLUA_USE_LINUX -DBA_FILESIZE64 -DUSE_LUAINTF=1 -DUSE_DBGMON=1 -DUSE_OPCUA=1 -Iinc -Iinc/arch/Posix -Iinc/arch/NET/Posix -c -o sqlite3.o src/sqlite3.c
#6 132.0 cc -o mako BAS.o ThreadLib.o SoDisp.o BaFile.o MakoMain.o ls_sqlite3.o luasql.o sqlite3.o -lpthread -lm -ldl
#6 132.1 mako: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-x86_64.so.1, with debug_info, not stripped
#6 132.1        /lib/ld-musl-x86_64.so.1 (0x7f69278f3000)
#6 132.1        libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1 (0x7f69278f3000)
#6 DONE 132.2s

#7 [4/4] RUN echo '::::: ADDING mako.zip :::::'         && cd /tmp/     && wget makoserver.net/download/mako.linux-x32.tar.gz   && tar xvzf mako.linux-x32.tar.gz       && cp mako.zip /BAS/
#7 0.522 ::::: ADDING mako.zip :::::
#7 0.578 Connecting to makoserver.net (198.23.228.254:80)
#7 0.994 saving to 'mako.linux-x32.tar.gz'
#7 1.608 mako.linux-x32.tar.g   2% |                                | 49686  0:00:38 ETA
#7 2.616 mako.linux-x32.tar.g  12% |****                            |  246k  0:00:13 ETA
#7 3.656 mako.linux-x32.tar.g  23% |*******                         |  449k  0:00:09 ETA
#7 4.503 mako.linux-x32.tar.g  30% |*********                       |  583k  0:00:09 ETA
#7 5.521 mako.linux-x32.tar.g  40% |*************                   |  777k  0:00:07 ETA
#7 6.532 mako.linux-x32.tar.g  49% |***************                 |  949k  0:00:06 ETA
#7 7.589 mako.linux-x32.tar.g  58% |******************              | 1117k  0:00:04 ETA
#7 8.513 mako.linux-x32.tar.g  67% |*********************           | 1283k  0:00:03 ETA
#7 9.549 mako.linux-x32.tar.g  76% |************************        | 1464k  0:00:02 ETA
#7 10.49 mako.linux-x32.tar.g  85% |***************************     | 1631k  0:00:01 ETA
#7 11.54 mako.linux-x32.tar.g  93% |*****************************   | 1784k  0:00:00 ETA
#7 12.23 mako.linux-x32.tar.g 100% |********************************| 1906k  0:00:00 ETA
#7 12.23 'mako.linux-x32.tar.gz' saved
#7 12.24 LICENSE.txt
#7 12.33 mako
#7 12.33 mako-nosql
#7 12.33 mako.zip
#7 12.33 README.txt
#7 12.33 rundemo.sh
#7 12.33 tutorial/
#7 12.33 tutorial/README.txt
#7 12.33 tutorial/DownloadTutorials.zip
#7 DONE 12.4s

#8 [5/5] RUN /BAS/mako
#8 0.469
#8 0.469 Mako Server. Version 3.9
#8 0.469 BAS lib 5453. Build date: Jun  7 2023
#8 0.469 Copyright (c) Real Time Logic.
#8 0.469
#8 ERROR: process "/bin/sh -c /BAS/mako" did not complete successfully: exit code: 139
------
 > [5/5] RUN /BAS/mako:
#8 0.469
#8 0.469 Mako Server. Version 3.9
#8 0.469 BAS lib 5453. Build date: Jun  7 2023
#8 0.469 Copyright (c) Real Time Logic.
#8 0.469
------
Dockerfile:28
--------------------
  26 |          && echo DONE mako.zip
  27 |
  28 | >>>  RUN /BAS/mako
  29 |
--------------------
ERROR: failed to solve: process "/bin/sh -c /BAS/mako" did not complete successfully: exit code: 139

Still segfaults. This is the file hash I get for the binary I built: 

# sha256sum mako
fe2530b4308b15a47810d9b5b4f81e28d6a63bb2466e14ab9ef60e2c85e707a1  mako
surfskidude commented 1 year ago

I installed Alpine, and I got the same exception as you. I recompiled with debug info and ran mako in gdb. After some investigation, the error appears to be in pthread_timedjoin_np (bug in the library).

(gdb) info threads
  Id   Target Id         Frame
  1    LWP 3372 "mako"   0x0000555555596b00 in sem_post@plt ()
* 2    LWP 3373 "mako"   Thread_threadStart (self=0x555555896220 <timer>) at src/arch/Posix/ThreadLib.c:184
(gdb) thread 1
[Switching to thread 1 (LWP 3372)]
#0  0x0000555555596b00 in sem_post@plt ()
(gdb) bt
#0  0x0000555555596b00 in sem_post@plt ()
#1  0x00005555556aa783 in Thread_start (o=0x555555896220 <timer>) at src/arch/Posix/ThreadLib.c:244
#2  0x00005555555f0e57 in BaTimer_constructor (o=0x555555896220 <timer>, slc90e66bridge=0x555555896000 <mutex>, stage2unmap=10000, gpio2direction=50,
    gpio1resources=ThreadPrioNormal, unmapaliases=0x0) at src/BAS.c:106765
#3  0x00005555556b07fe in createServer (argc=1, argv=0x7fffffffec48, execpath=0x7fffffffe9f8) at examples/MakoServer/src/MakoMain.c:1011
#4  0x00005555556b1538 in runMako (isWinService=0, argc=1, argv=0x7fffffffec48, envp=0x7fffffffec58) at examples/MakoServer/src/MakoMain.c:1527
#5  0x00005555556b1c1f in main (argc=1, argv=0x7fffffffec48, envp=0x7fffffffec58) at examples/MakoServer/src/MakoMain.c:1708
(gdb) thread 2
[Switching to thread 2 (LWP 3373)]
#0  Thread_threadStart (self=0x555555896220 <timer>) at src/arch/Posix/ThreadLib.c:184
184        Thread* thread = (Thread*)self;
(gdb) bt
#0  Thread_threadStart (self=0x555555896220 <timer>) at src/arch/Posix/ThreadLib.c:184
#1  0x00007ffff7fb9c0b in ?? () from /lib/ld-musl-x86_64.so.1
#2  0x0000000000000000 in ?? ()
(gdb) s

Thread 1 "mako" received signal SIGSEGV, Segmentation fault.
[Switching to LWP 3372]
0x00007ffff7fba47a in pthread_timedjoin_np () from /lib/ld-musl-x86_64.so.1

(gdb) bt
#0  0x00007ffff7fba47a in pthread_timedjoin_np () from /lib/ld-musl-x86_64.so.1
#1  0x00007ffff7ffdb48 in ?? () from /lib/ld-musl-x86_64.so.1
#2  0x00000001556aa724 in ?? ()
#3  0x00007ffff7f4eb38 in ?? ()
#4  0x00007fffffffe900 in ?? ()
#5  0x00005555556b1bf5 in runMako (isWinService=32767, argc=-134246888, argv=0x1e, envp=0x7fffffa100000000) at examples/MakoServer/src/MakoMain.c:1700
mterron commented 1 year ago

This is what Rich Felker (musl author) has to say about a similar issue:

This it an intentional trap for undefined behavior when the caller attempts to join a detached thread or detach a thread that was not joinable (already detached or already being joined by another thread).

So apparently this is a bug in Mako. In particular, it seems https://github.com/RealTimeLogic/BAS/blob/d7075920019acff07728cef8bdcf36ec10d280d0/src/arch/Posix/ThreadLib.c#L245 is trying to detach a thread that was created in a detached state by Thread_constructor https://github.com/RealTimeLogic/BAS/blob/d7075920019acff07728cef8bdcf36ec10d280d0/src/arch/Posix/ThreadLib.c#L228 This is UB and it's what's causing the Segmentation Fault by tripping musl UB guard.

Commenting that line solves the immediate issue, but is not sufficient. cc complains about the unused tid variable:

cc -fmerge-all-constants -O3 -Os -Wall -std=gnu99 -DUSE_FORKPTY=0 -DMAKO -DUSE_EMBEDDED_ZIP=0 -DLUA_USE_LINUX -DBA_FILESIZE64 -DUSE_LUAINTF=1 -DUSE_DBGMON=1 -DUSE_OPCUA=1 -Iinc -Iinc/arch/Posix -Iinc/arch/NET/Posix -c -o ThreadLib.o src/arch/Posix/ThreadLib.c
src/arch/Posix/ThreadLib.c: In function 'Thread_start':
src/arch/Posix/ThreadLib.c:243:14: warning: unused variable 'tid' [-Wunused-variable]
  243 |    pthread_t tid = o->tid;
      |              ^~~
cc -o mako BAS.o ThreadLib.o SoDisp.o BaFile.o MakoMain.o ls_sqlite3.o luasql.o sqlite3.o -lpthread -lm -ldl

Solving this is quite simple, it requires removal of https://github.com/RealTimeLogic/BAS/blob/d7075920019acff07728cef8bdcf36ec10d280d0/src/arch/Posix/ThreadLib.c#L243.

Can someone from RealTimeLogic please review this and ensure this is a bug and not expected behaviour? Happy to provide a patch.

mterron commented 1 year ago

Fixed by https://github.com/RealTimeLogic/BAS/pull/9

surfskidude commented 1 year ago

Thank you for reporting this. The thread is set in a detached state when it is created; thus, the additional detach was wrong. BTW, we cannot directly use your pull request, so can you cancel your pull request? I have updated the repo with working code.