`Illegal instruction (core dumped)` on Raspberry Pi 4 (ARMv8-A)

depthbomb commented 1 week ago

What version of Bun is running?

1.1.16+bf7b327f6

What platform is your computer?

Linux 6.2.0-1018-raspi aarch64 aarch64

What steps can reproduce the bug?

Run most commands

What is the expected behavior?

The commands execute normally as they have in version 1.1.15.

What do you see instead?

The output Illegal instruction (core dumped).

Additional information

After upgrading to Bun 1.1.16 on my Raspberry Pi 4, almost all commands other than bun and bun init cause a core dump immediately upon execution. This happens after both trying to reinstall while Bun is already installed and installing fresh, as well as after a device reboot.

Along with a core dump happening when running any commands, it also happens post-install of Bun itself.

6oikrk

I haven't tested all commands but notable ones that cause this behavior in my case have been:

bun i
bun upgrade
running scripts in package.json with bun <script>

Jarred-Sumner commented 1 week ago

are you sure you're running on arm64 and not arm32?

depthbomb commented 1 week ago

I believe I am but admittedly I'm not as familiar with Linux as I should be.

Below is the output of lscup

Architecture:            aarch64
  CPU op-mode(s):        32-bit, 64-bit
  Byte Order:            Little Endian
CPU(s):                  4
  On-line CPU(s) list:   0-3
Vendor ID:               ARM
  Model name:            Cortex-A72
    Model:               3
    Thread(s) per core:  1
    Core(s) per cluster: 4
    Socket(s):           -
    Cluster(s):          1
    Stepping:            r0p3
    CPU(s) scaling MHz:  87%
    CPU max MHz:         1500.0000
    CPU min MHz:         600.0000
    BogoMIPS:            108.00
    Flags:               fp asimd evtstrm crc32 cpuid
Caches (sum of all):
  L1d:                   128 KiB (4 instances)
  L1i:                   192 KiB (4 instances)
  L2:                    1 MiB (1 instance)
Vulnerabilities:
  Gather data sampling:  Not affected
  Itlb multihit:         Not affected
  L1tf:                  Not affected
  Mds:                   Not affected
  Meltdown:              Not affected
  Mmio stale data:       Not affected
  Retbleed:              Not affected
  Spec rstack overflow:  Not affected
  Spec store bypass:     Vulnerable
  Spectre v1:            Mitigation; __user pointer sanitization
  Spectre v2:            Vulnerable
  Srbds:                 Not affected
  Tsx async abort:       Not affected

eslym commented 1 week ago

same here

(lldb) target create "bun"
run install
Current executable set to 'bun' (aarch64).
(lldb) run install
Process 3799310 launched: '/home/engshun/.bun/bin/bun' (aarch64)
Process 3799310 stopped
* thread #1, name = 'bun', stop reason = signal SIGILL: illegal instruction
    frame #0: 0x00000055582094a0 bun`src.http.HTTPThread.init at atomic.zig:48:39
(lldb) bt all
* thread #1, name = 'bun', stop reason = signal SIGILL: illegal instruction
  * frame #0: 0x00000055582094a0 bun`src.http.HTTPThread.init at atomic.zig:48:39
    frame #1: 0x0000005558209494 bun`src.http.HTTPThread.init at http.zig:773
    frame #2: 0x0000005558322788 bun`src.cli.install_command.InstallCommand.exec [inlined] initWithCLI__anon_62747(ctx=0x000000555addc900, cli=src.install.install.PackageManager.CommandLineArguments @ 0x00000000216ce020) at install.zig:8246:33
    frame #3: 0x0000005558322784 bun`src.cli.install_command.InstallCommand.exec [inlined] init__anon_62404(ctx=0x000000555addc900) at install.zig:8237
    frame #4: 0x0000005558322680 bun`src.cli.install_command.InstallCommand.exec at install.zig:11204
    frame #5: 0x0000005558322680 bun`src.cli.install_command.InstallCommand.exec(ctx=<unavailable>) at install_command.zig:7
    frame #6: 0x000000555812c6cc bun`src.cli.Cli.start at cli.zig:1494:40
    frame #7: 0x000000555812b7f4 bun`src.cli.Cli.start at cli.zig:62
    frame #8: 0x000000555812ad04 bun`start.main [inlined] main at main.zig:50:22
    frame #9: 0x000000555812acc4 bun`start.main [inlined] callMain at start.zig:514
    frame #10: 0x000000555812acc4 bun`start.main at start.zig:482
    frame #11: 0x000000555812abe4 bun`start.main(c_argc=2, c_argv=0x0000007ffffff4f8, c_envp=0x0000007ffffff510) at start.zig:497
    frame #12: 0x0000007ff7e78e18 libc.so.6`__libc_start_main(main=(bun`start.main at start.zig:485), argc=2, argv=0x0000007ffffff4f8, init=<unavailable>, fini=<unavailable>, rtld_fini=<unavailable>, stack_end=<unavailable>) at libc-start.c:308:16
    frame #13: 0x000000555812aa34 bun`_start + 52
(lldb)

raspberry pi 4 rasbian OS 64 bit

p/s: i have a friend help test with his raspberry pi 4, can confirmed this will happens on pi4

vgf89 commented 1 week ago

Happens on my Pi 3 64-bit too. If I try to run bun serve or bun upgrade, it just returns "Illegal Instruction". Also got the same illegal instruction message during install. Had to downgrade

nzapponi commented 1 week ago

Same here. Running on a Raspberry Pi 4 with Ubuntu Server 64 bit.

jakobbak commented 1 week ago

same here

Linux xxxxxxxx 6.6.20+rpt-rpi-v8 #1 SMP PREEMPT Debian 1:6.6.20-1+rpt1 (2024-03-07) aarch64 GNU/Linux

lyorb commented 1 week ago

Same issue for me with :

Architecture:                    aarch64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
CPU(s):                          4
On-line CPU(s) list:             0-3
Thread(s) per core:              1
Core(s) per socket:              4
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       ARM
Model:                           4
Model name:                      Cortex-A53
Stepping:                        r0p4
CPU max MHz:                     1800.0000
CPU min MHz:                     1200.0000
BogoMIPS:                        16.00
NUMA node0 CPU(s):               0-3
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Not affected
Vulnerability Spectre v1:        Mitigation; __user pointer sanitization
Vulnerability Spectre v2:        Not affected
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid

The only way is to downgrade to version 1.1.15 @Jarred-Sumner

Jarred-Sumner commented 1 week ago

After some reading, it looks like Raspberry Pi 4 (ARMv8-A) is old enough to not support atomic instructions. It was added in ARMv8.1-A. That also explains why our test suite ran successfully on Linux arm64 and why I'm unable to reproduce this on our AWS machines that use Linux aarch64.

I'm not sure why this started in Bun v1.1.16. I think we can set -mcpu to the ARM Cortex A72 on Linux and that would address this. Another option is introducing a linux-baseline build when targeting linux arm64. That is what we do to support older CPUs on x64.

@eslym this stacktrace was helpful, thank you:

* thread #1, name = 'bun', stop reason = signal SIGILL: illegal instruction
    frame #0: 0x00000055582094a0 bun`src.http.HTTPThread.init at atomic.zig:48:39
(lldb) bt all
* thread #1, name = 'bun', stop reason = signal SIGILL: illegal instruction
  * frame #0: 0x00000055582094a0 bun`src.http.HTTPThread.init at atomic.zig:48:39
    frame #1: 0x0000005558209494 bun`src.http.HTTPThread.init at http.zig:773
    frame #2: 0x0000005558322788 bun`src.cli.install_command.InstallCommand.exec [inlined] initWithCLI__anon_62747(ctx=0x000000555addc900, cli=src.install.install.PackageManager.CommandLineArguments @ 0x00000000216ce020) at install.zig:8246:33
    frame #3: 0x0000005558322784 bun`src.cli.install_command.InstallCommand.exec [inlined] init__anon_62404(ctx=0x000000555addc900) at install.zig:8237
    frame #4: 0x0000005558322680 bun`src.cli.install_command.InstallCommand.exec at install.zig:11204
    frame #5: 0x0000005558322680 bun`src.cli.install_command.InstallCommand.exec(ctx=<unavailable>) at install_command.zig:7
    frame #6: 0x000000555812c6cc bun`src.cli.Cli.start at cli.zig:1494:40
    frame #7: 0x000000555812b7f4 bun`src.cli.Cli.start at cli.zig:62
    frame #8: 0x000000555812ad04 bun`start.main [inlined] main at main.zig:50:22
    frame #9: 0x000000555812acc4 bun`start.main [inlined] callMain at start.zig:514
    frame #10: 0x000000555812acc4 bun`start.main at start.zig:482
    frame #11: 0x000000555812abe4 bun`start.main(c_argc=2, c_argv=0x0000007ffffff4f8, c_envp=0x0000007ffffff510) at start.zig:497
    frame #12: 0x0000007ff7e78e18 libc.so.6`__libc_start_main(main=(bun`start.main at start.zig:485), argc=2, argv=0x0000007ffffff4f8, init=<unavailable>, fini=<unavailable>, rtld_fini=<unavailable>, stack_end=<unavailable>) at libc-start.c:308:16
    frame #13: 0x000000555812aa34 bun`_start + 52

nzapponi commented 6 days ago

I'd be up for having a baseline build for arm64.

What is the impact (if any) of not using atomic instructions?

Interesting how it only appeared as an issue in 1.1.16...

zoto-ff commented 2 days ago

same on Raspberry Pi 4 Model B Rev 1.4

Architecture:             aarch64
  CPU op-mode(s):         32-bit, 64-bit
  Byte Order:             Little Endian
CPU(s):                   4
  On-line CPU(s) list:    0-3
Vendor ID:                ARM
  Model name:             Cortex-A72
    Model:                3
    Thread(s) per core:   1
    Core(s) per cluster:  4
    Socket(s):            -
    Cluster(s):           1
    Stepping:             r0p3
    CPU(s) scaling MHz:   67%
    CPU max MHz:          1500.0000
    CPU min MHz:          600.0000
    BogoMIPS:             108.00
    Flags:                fp asimd evtstrm crc32 cpuid
Caches (sum of all):
  L1d:                    128 KiB (4 instances)
  L1i:                    192 KiB (4 instances)
  L2:                     1 MiB (1 instance)
Vulnerabilities:
  Gather data sampling:   Not affected
  Itlb multihit:          Not affected
  L1tf:                   Not affected
  Mds:                    Not affected
  Meltdown:               Not affected
  Mmio stale data:        Not affected
  Reg file data sampling: Not affected
  Retbleed:               Not affected
  Spec rstack overflow:   Not affected
  Spec store bypass:      Vulnerable
  Spectre v1:             Mitigation; __user pointer sanitization
  Spectre v2:             Vulnerable
  Srbds:                  Not affected
  Tsx async abort:        Not affected

oven-sh / bun