Segmentation fault when executing the jitted ebpf program

Description

Segmentation fault when executting the jitted ebpf program. But the same program works well when using intepreter mode.

Cargo.toml

[package]
name = "rbpf-poc"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
rbpf = { git = "https://github.com/qmonnet/rbpf" }

Rust program

fn main() {
    let prog = std::fs::read("prime.bpf.bin").unwrap();
    let mut vm = rbpf::EbpfVmRaw::new(Some(&prog)).unwrap();
    vm.jit_compile().unwrap();
    let mem = &mut [0x00];
    unsafe { vm.execute_program_jit(mem) }.unwrap();
}

ebpf program

prime.bpf.bin.zip

It was zipped, since github does not support uploading *.bin files.

The ebpf program was compiled from the following C source using clang 14.0.6, then the .text segment was extracted using llvm-objcopy

int main() {
  long cnt = 0;
  for (int i = 1; i < 1e4; i++) {
    int ok = 1;
    for (int j = 2; j * j <= i && ok; j++) {
      if (i % j == 0)
        ok = 0;
    }
    cnt += ok;
  }
  return cnt;
}

How to trigger

root@mnfe-pve:~/rbpf-poc# cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.07s
     Running `target/debug/rbpf-poc`
Segmentation fault

Expected behavior

Program successfully exited with no exception

Environment

stable-x86_64-unknown-linux-gnu (default)
rustc 1.71.0 (8ede3aae2 2023-07-12)

root@mnfe-pve:~/bpf-benchmark# uname -a
Linux mnfe-pve 6.2.16-6-pve #1 SMP PREEMPT_DYNAMIC PMX 6.2.16-7 (2023-08-01T11:23Z) x86_64 GNU/Linux
root@mnfe-pve:~/bpf-benchmark# lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 12 (bookworm)
Release:        12
Codename:       bookworm
root@mnfe-pve:~/bpf-benchmark#

Thanks for the report!

I can reproduce. We're having an issue with the division. Here's how clang translated your modulo operation:

mov64 r7, r5       // r7 = i
div64 r7, r0       // r7 = r7 / j           -> This instruction causes the crash
mul64 r7, r0       // r7 = r7 * j
mov64 r6, r5       // r6 = i
sub64 r6, r7       // r6 = i - i/j*j

The generated eBPF code looks correct. Playing with the code from the JIT compiler, we're having an issue with this line: removing the generation of the instructions to handle divisions-by-zero in this if {} make the instruction pass.

Looking further, the offset we use for the emit_direct_jcc(mem, 0x85, 7) is incorrect in your case. Why then does it work for all the existing tests? This is because some instruction may convert to 2 or 3 bytes, depending on the register we use. With r7, it's 3 bytes; whereas the existing tests usually use the lower registers. Given that the offset is incorrect, we don't jump correctly, causing the segfault.

Addressing the offset (based on the same check done in the instructions that emits a variable number of bytes) fixes the issue. I'm preparing a PR.

qmonnet / rbpf

Segmentation fault when executing jitted program #88