MacOS 12.4 Bad system call: 12

[elamje]

This is an amazing project! I'm having trouble getting it running on my Mac. I was worried zip or Gatekeeper were corrupting the file somehow, but now I am thinking I just may not understand how to run it on MacOS. Please see output below.

OS: macOS Monterey 12.4 (Gatekeeper Disabled), Intel

curl https://redbean.dev/redbean-latest.com.dbg >redbean.com.dbg

file redbean.com.dbg 
redbean.com.dbg: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, with debug_info, not stripped

chmod +x redbean.com.dbg

./redbean.com.dbg 
bash: ./redbean.com.dbg: cannot execute binary file

./redbean.com.dbg --assimilate
bash: ./redbean.com.dbg: cannot execute binary file

alternatively

curl https://redbean.dev/redbean-latest.com >redbean.com

file redbean.com
redbean.com: DOS/MBR boot sector

chmod +x redbean.com

./redbean.com
Bad system call: 12

./redbean.com --assimilate

file redbean.com
redbean.com: Mach-O 64-bit executable x86_64

./redbean.com
Bad system call: 12

I hope this isn't a waste of your time, I didn't see anyone else from the 2.0 release thread on HN have this issue, though there were some similar problems on the original Show HN that didn't help me.

[jart]

This is a high priority item. I monitor the XNU syscalls.master file and I haven't seen any changes that would indicate a breakage. I'm running MacOS 11.6.5 right now and I don't have access to a v12.4 machine.

Could you please try running ./redbean.com --strace and ./redbean.com --ftrace and let me know what it says?

It would also help if someone could open the executable in a debugger, and let me know what the value of RAX register is when it falts on the SYSCALL instruction. Basically, I need to know which system call Apple isn't allowing.

Alternatively, you can grant me SSH access to an apple machine running this version. My SSH keys are in https://github.com/jart.keys

[elamje]

Of course:

./redbean.com --strace
SYS      0          1'113'297 bell system five system call support 383 magnums loaded on xnu's not unix!
SYS  28691          1'576'089 mprotect(0x700000000000, 4'096, 0) → 0
SYS  28691          1'968'404 mmap(0x700000000000, 262'144, PROT_READ|PROT_WRITE, MAP_STACK|MAP_ANONYMOUS, -1, 0) → 0x700000000000 (262'144 bytes total)
SYS  28691          2'239'454 mmap(0, 65'536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) → 0x100080000000 (327'680 bytes total)
SYS  28691          2'643'336 close(3) → 0
SYS  28691          2'942'506 getcwd(0x5ee720, 1'024) → "/Users/johnelamparq/Downloads/red"
SYS  28691          2'946'003 getenv("MAKEFLAGS") → NULL
SYS  28691          2'950'308 getenv("TERM") → "xterm-256color"
SYS  28691          3'979'373 openat(AT_FDCWD, "/Users/johnelamparq/Downloads/red/redbean.com", 0, 0) → 3
SYS  28691          3'988'286 getfiledescriptorsize(3) → 1'999'386
SYS  28691          3'998'485 mmap(0, 1'999'386, PROT_READ, MAP_SHARED, 3, 0) → 0x100080100000 (2'359'296 bytes total)
SYS  28691          4'468'443 munmap(0x100080100000, 1'703'936) → 0 (655'360 bytes total)
SYS  28691          4'476'056 __zipos_get("/Users/johnelamparq/Downloads/red/redbean.com")
SYS  28691          4'479'700 close(3) → 0
SYS  28691          4'481'944 openat(AT_FDCWD, "/zip/.args", 0, 0) → -1 ENOENT/2/No such file or directory
SYS  28691          4'489'392 mmap(0, 262'144, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) → 0x100080040000 (917'504 bytes total)
SYS  28691          4'493'059 sigaltstack({.ss_sp=0x100080040000, .ss_flags=0, .ss_size=262'144}, [{.ss_sp=0x100080040000, .ss_flags=0, .ss_size=262'144}]) → 0
SYS  28691          4'501'687 sigaction(SIGQUIT, {.sa_handler=0x52c737, .sa_flags=0x51, .sa_mask=~{}}, [{.sa_handler=0, .sa_flags=0, .sa_mask={}}]) → 0
SYS  28691          4'505'552 sigaction(SIGFPE, {.sa_handler=0x52c742, .sa_flags=0x51, .sa_mask=~{}}, [{.sa_handler=0, .sa_flags=0, .sa_mask={}}]) → 0
SYS  28691          4'508'872 sigaction(SIGILL, {.sa_handler=0x52c74d, .sa_flags=0x51, .sa_mask=~{}}, [{.sa_handler=0, .sa_flags=0, .sa_mask={}}]) → 0
SYS  28691          4'512'138 sigaction(SIGSEGV, {.sa_handler=0x52c758, .sa_flags=0x51, .sa_mask=~{}}, [{.sa_handler=0, .sa_flags=0, .sa_mask={}}]) → 0
SYS  28691          4'515'270 sigaction(SIGTRAP, {.sa_handler=0x52c763, .sa_flags=0x51, .sa_mask=~{}}, [{.sa_handler=0, .sa_flags=0, .sa_mask={}}]) → 0
SYS  28691          4'518'659 sigaction(SIGABRT, {.sa_handler=0x52c76e, .sa_flags=0x51, .sa_mask=~{}}, [{.sa_handler=0, .sa_flags=0, .sa_mask={}}]) → 0
SYS  28691          4'521'901 sigaction(SIGBUS, {.sa_handler=0x52c779, .sa_flags=0x51, .sa_mask=~{}}, [{.sa_handler=0, .sa_flags=0, .sa_mask={}}]) → 0
SYS  28691          4'893'796 mmap(0, 262'144, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) → 0x100080080000 (1'179'648 bytes total)
SYS  28691          6'359'021 inflate([u"SYMT☺   ↨         ♦       ♦       @     "...], 262'144, u"lⁿuÿV╒× ⁿ»↓bå£Ç↓RAѶ♣♀JQR☺Ñ♦ò♫âûÉ♫üí╒♥ét"..., 103'922) → 0
SYS  28691          6'368'102 GetSymbolTableFromZip() → 0x100080080000
Bad system call: 12

./redbean.com --ftrace
FUN 28703    16'912'675      64 &__gdtoa_Binit
FUN 28703    16'926'824      80   &atexit
FUN 28703    16'929'448     128     &__cxa_atexit
FUN 28703    16'931'186      96   &__gdtoa_i2b
FUN 28703    16'932'650     144     &__gdtoa_Balloc
FUN 28703    16'937'972     224       &dlmalloc
FUN 28703    16'940'324      80   &atexit
FUN 28703    16'941'528     128     &__cxa_atexit
FUN 28703    16'956'907      80   &atexit
FUN 28703    16'958'787     128     &__cxa_atexit
FUN 28703    16'968'202      64 &main
FUN 28703    16'972'810     112   &LoadZipArgs
FUN 28703    16'976'856     192     &xslurp
FUN 28703    16'980'730     288       &open
FUN 28703    16'982'655     464         &openat
FUN 28703    16'985'990     480           &__zipos_parseuri
FUN 28703    16'988'184     512           &__zipos_open
FUN 28703    16'989'449     592             &__zipos_get
FUN 28703    16'994'594     624             &__zipos_find
FUN 28703    16'998'354     640               &GetZipCdirOffset
FUN 28703    17'000'669     640               &GetZipCdirRecords
FUN 28703    17'003'412     528             &enoent
FUN 28703    17'007'339     112   &ShowCrashReports
FUN 28703    17'011'359     224     &mmap
FUN 28703    17'015'392     240       &rounddown2pow
FUN 28703    17'019'529     240       &sys_mmap
FUN 28703    17'026'840     320       &FinishMemory.isra.0
FUN 28703    17'032'761     448     &sigaltstack
FUN 28703    17'034'605     464       &sigaltstack2bsd
FUN 28703    17'036'773     464       &sigaltstack2linux
FUN 28703    17'038'393     160     &__cxa_atexit
FUN 28703    17'039'865     192     &InstallCrashHandlers
FUN 28703    17'041'263     208       &sigfillset
FUN 28703    17'042'663     208       &sigdelset
FUN 28703    17'044'203     496       &sigaction
FUN 28703    17'045'545     608         &__sigaction
FUN 28703    17'046'867     624           &sigaction_cosmo2native
FUN 28703    17'049'922     640           &sigaction_native2cosmo
FUN 28703    17'051'435     496       &sigaction
FUN 28703    17'052'669     608         &__sigaction
FUN 28703    17'053'927     624           &sigaction_cosmo2native
FUN 28703    17'055'915     640           &sigaction_native2cosmo
FUN 28703    17'057'234     496       &sigaction
FUN 28703    17'058'510     608         &__sigaction
FUN 28703    17'060'027     624           &sigaction_cosmo2native
FUN 28703    17'062'001     640           &sigaction_native2cosmo
FUN 28703    17'063'252     496       &sigaction
FUN 28703    17'064'440     608         &__sigaction
FUN 28703    17'065'638     624           &sigaction_cosmo2native
FUN 28703    17'067'457     640           &sigaction_native2cosmo
FUN 28703    17'068'702     496       &sigaction
FUN 28703    17'069'943     608         &__sigaction
FUN 28703    17'071'620     624           &sigaction_cosmo2native
FUN 28703    17'073'442     640           &sigaction_native2cosmo
FUN 28703    17'074'725     496       &sigaction
FUN 28703    17'075'920     608         &__sigaction
FUN 28703    17'077'119     624           &sigaction_cosmo2native
FUN 28703    17'078'934     640           &sigaction_native2cosmo
FUN 28703    17'080'284     496       &sigaction
FUN 28703    17'081'846     608         &__sigaction
FUN 28703    17'083'038     624           &sigaction_cosmo2native
FUN 28703    17'084'854     640           &sigaction_native2cosmo
FUN 28703    17'088'191     128     &GetSymbolTable
FUN 28703    17'092'238     128   &RedBean
FUN 28703    17'096'365     160     &nowl_setup
FUN 28703    17'099'004     192       &__clock_gettime_get
FUN 28703    17'101'788     224       &RefreshTime
FUN 28703    17'103'251     304         &MeasureNanosPerCycle
FUN 28703    17'104'632     368           &GetTimeSample
Bad system call: 12

[jart]

It looks like the issue might be with sched_yield. I've pushed a commit that I believe will fix this. Could you please try downloading https://redbean.dev/redbean-issue426.com and letting me know if that fixes the issue? Alternatively, you could build this repo with the latest change from HEAD. Thanks!

[sszachow]

Hi @jart , for me it fixed the problem:

curl https://redbean.dev/redbean-issue426.com >redbean.com
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1987k  100 1987k    0     0  1082k      0  0:00:01  0:00:01 --:--:-- 1086k

chmod +x redbean.com

./redbean.com -v
I2022-06-17T04:10:31.391340:tool/net/redbean.c:6923:redbean:5236] (srvr) listen http://127.0.0.1:8080
I2022-06-17T04:10:31+000054:tool/net/redbean.c:6923:redbean:5236] (srvr) listen http://192.168.0.152:8080
I2022-06-17T04:10:31+000026:tool/net/redbean.c:6923:redbean:5236] (srvr) listen http://10.70.31.188:8080
V2022-06-17T04:10:31+000133:tool/net/redbean.c:1857:redbean:5236] (ssl) could not find non-CA SSL certificate key pair with -addext keyUsage=digitalSignature -addext extendedKeyUsage=serverAuth
V2022-06-17T04:10:31+000006:tool/net/redbean.c:1860:redbean:5236] (ssl) could not find CA key signing key pair with -addext keyUsage=keyCertSign
V2022-06-17T04:10:31+000002:tool/net/redbean.c:1862:redbean:5236] (ssl) generating self-signed ssl certificates
V2022-06-17T04:10:31+001326:tool/net/redbean.c:610:redbean:5236] (ssl) using EC certificate "CN=localhost" for HTTPS server
V2022-06-17T04:10:31+057612:tool/net/redbean.c:610:redbean:5236] (ssl) using RSA certificate "CN=localhost" for HTTPS server

[jart]

That's great news! Thank you everyone for your help. I'll push a 2.0.2 release in a few hours.

[jart]

redbean-2.0.2 now available at https://redbean.dev/2.0.html

[elamje]

@jart Wanted to double confirm that it's working for me as well. Thank you for your diligence with this!

[tpoindex]

I was experiencing the same issue, 2.0.3 fixes the normal start up case, but I still get this with '-h'. Problem persists with using --assimilate also.

$ ./redbean.com203
I2022-06-17T10:25:57.898531:tool/net/redbean.c:6923:redbean:17471] (srvr) listen http://127.0.0.1:8080
I2022-06-17T10:25:57+000052:tool/net/redbean.c:6923:redbean:17471] (srvr) listen http://192.168.0.3:8080
I2022-06-17T10:25:57+000048:tool/net/redbean.c:6923:redbean:17471] (srvr) listen http://10.89.215.210:8080
>:
I2022-06-17T10:26:01.071838:tool/net/redbean.c:6945:redbean:17471] (srvr) received SIGINT
I2022-06-17T10:26:01+000164:tool/net/redbean.c:7305:redbean:17471] (srvr) shutdown complete
$
$
$ ./redbean.com203 -h
Bad system call: 12
$

[pkulchenko]

@tpoindex, can you post the output for the failing command with added --ftrace flag starting from &RedBean call?

[tpoindex]

Here's the --ftrace starting at &RedBean. I can see the man page being written, and I deleted most of that text as well It's still very large, 40k lines.

redbean203-h--ftrace.gz .

[jart]

@tpoindex I read through your ftrace log and I don't see the bad system call error being raised. The program proceeds all the way to _Exit() so it looks kosher. Could you share an ftrace log that reproduces the error?

[tpoindex]

@jart I'm not sure which ftrace you'd like me to run, the previous log was from using redbean -h --ftrace.

I also ran with --strace, and have a snippet from that. Just before the 'Bad system call', inside __paginate(): (and now running version 2.0.4, downloaded as redbean.com204)

$ redbean.com204 -h --strace
. . .
SYS  37530          8'973'346 faccessat(AT_FDCWD, "/usr/local/bin/less", 01, 0) → -1 ENOENT/2/No such file or directory
SYS  37530          9'020'591 faccessat(AT_FDCWD, "/usr/bin/less", 01, 0) → 0 ENOENT/2/No such file or directory
SYS  37530          9'025'563 commandv("less", 0x70000003f6f0, 1'024) → "/usr/bin/less"
SYS  37530          9'176'256 openat(AT_FDCWD, "/tmp/redbean.com204-paginate-37530.txt", O_WRONLY|O_CREAT|O_TRUNC, 0644) → 4
SYS  37530          9'371'972 write(4, u"SYNOPSIS◙◙  http://redbean.com  [-⁇BVabdfghjkmsu"..., 160'800) → 160'800
SYS  37530          9'479'396 close(4) → 0
SYS  37530          9'484'618 vfork()
Bad system call: 12

I also note that redirecting the -h output to a file or pipe, or unsetting TERM works just fine, so the issue seems to be in vfork():

$ redbean.com204 -h --strace | cat
. . .
S  37682          7'843'659 getenv("TERM") → "xterm-256color"
SYS  37682          7'848'547 isatty(0) → true
SYS  37682          7'851'859 isatty(1) → false ENODEV/19/No such device
SYS  37682          7'961'536 write(1, u"SYNOPSIS◙◙  [redbean.com](http://www.google.com/url?q=http%3A%2F%2Fredbean.com&sa=D&sntz=1&usg=AOvVaw1Ax59xRwSMEdZemrYezJc5) [-⁇BVabdfghjkmsu"..., 160'800) → 160'800
SYS  37682          7'965'103 exit(0)
SYS  37682          7'967'959 __cxa_finalize(&FreeGmt, 0x100080098bf0)
SYS  37682          7'972'378 __cxa_finalize(&FreeLocaltime, 0x100080090010)
SYS  37682          7'974'234 __cxa_finalize(&FreeSigAltStack, 0x100080000000)
SYS  37682          7'985'006 sigaction(SIGQUIT, {.sa_handler=0x534cfa, .sa_flags=0x50, .sa_mask=~{}}, [{.sa_handler=0x534cfa, .sa_flags=0x51, .sa_mask={}}]) → 0
SYS  37682          7'990'112 sigaction(SIGFPE, {.sa_handler=0x534d05, .sa_flags=0x50, .sa_mask=~{}}, [{.sa_handler=0x534d05, .sa_flags=0x51, .sa_mask={}}]) → 0
SYS  37682          7'994'653 sigaction(SIGILL, {.sa_handler=0x534d10, .sa_flags=0x50, .sa_mask=~{}}, [{.sa_handler=0x534d10, .sa_flags=0x51, .sa_mask={}}]) → 0
SYS  37682          7'999'011 sigaction(SIGSEGV, {.sa_handler=0x534d1b, .sa_flags=0x50, .sa_mask=~{}}, [{.sa_handler=0x534d1b, .sa_flags=0x51, .sa_mask={}}]) → 0
SYS  37682          8'003'322 sigaction(SIGTRAP, {.sa_handler=0x534d26, .sa_flags=0x50, .sa_mask=~{}}, [{.sa_handler=0x534d26, .sa_flags=0x51, .sa_mask={}}]) → 0
SYS  37682          8'007'675 sigaction(SIGABRT, {.sa_handler=0x534d31, .sa_flags=0x50, .sa_mask=~{}}, [{.sa_handler=0x534d31, .sa_flags=0x51, .sa_mask={}}]) → 0
SYS  37682          8'011'847 sigaction(SIGBUS, {.sa_handler=0x534d3c, .sa_flags=0x50, .sa_mask=~{}}, [{.sa_handler=0x534d3c, .sa_flags=0x51, .sa_mask={}}]) → 0
SYS  37682          8'015'392 sigaltstack({.ss_sp=0, .ss_flags=0x4, .ss_size=0}, [NULL]) → 0
SYS  37682          8'019'811 munmap(0x100080000000, 262'144) → 0 (3'538'944 bytes total)
SYS  37682          8'021'553 __cxa_finalize(&fflush_unlocked, 0)
SYS  37682          8'024'874 __cxa_finalize(&linenoiseAtExit, 0)
SYS  37682          8'035'354 __cxa_finalize(&__gdtoa_Bclear, 0)
SYS  37682          8'038'965 _Exit(0)

I'd argue that redbean shouldn't be forking 'less' on usage anyway, but that's a different issue; perhaps I'm just too old-fashion in Unix ways.

[jart]

Are you telling me they removed vfork? Because that's bordering on nefarious but it's not in POSIX anymore and based on my benchmarks it never went any faster on XNU than fork() did, so it doesn't surprise me. I'll make a change shortly addressing the issue.

[jart]

It's worth noting that we identified an issue with older versions of Darwin Kernel running on VMs in this commit https://github.com/jart/cosmopolitan/commit/e4258015b7cb313aff2e0cde9b9038ffa8f2c598 There will be a new minor release of redbean (2.0.5) available at redbean.dev later today.

[tpoindex]

@jart Redbean 2.0.7 now shows -h just fine, forks 'less' successfully., thanks!

Btw, my Mac is running 12.4 Monterey, uname -a reports 'Darwin Kernel Version 21.5.0, Tue Apr 26 21:08:22 PDT 2022'

[comex]

vfork was removed from the kernel, but the libc wrapper has a workaround that implements it as fork. This is not the first time the xnu syscall ABI has made incompatible changes; Apple considers it unstable and only provides a stability guarantee for the libc wrappers.

[jart]

Part of what makes Cosmopolitan so interesting is we're able to build 12kb binaries that run on seven operating systems. If we were required to always use dynamic shared objects on Mac OS then it would destroy much of what we've accomplished.

I believe Apple is being unfair. The binary interface for UNIX SYSCALLs is something Apple inherited from the System V codebase (e.g. 1 for exit, 3 for read, etc.). They shouldn't consider something like that an internal API, when it's shared by so many operating systems.

THEREFORE please be advised there is some risk you may need to rebuild your binaries if Apple breaks the system call interface again. I have no commercial interest in distributing Cosmopolitan binaries. I'm just an open source developer trying to help out. I'd rather be focusing on improving the product than worrying about what appears to be a disagreement between two megacorps.

When Apple broke gettimeofday() they effectively smashed all of Google's toys. So far there's no indication something that drastic will happen again. swtch() and vfork() are things I could have predicted since the first is totally non-essential Mach and the latter was removed from POSIX a while ago. I've also added to redbean automated error reporting facilities now which trap SIGSYS and inform us in RBX of the precise system call that broke the moment it happens so it can be fixed quickly and easily. I believe that over time we will converge on a stable subset of SYSCALLs that Apple won't unintentionally break.

Just please be advised there may be some bumps along the way.