Wrong file-descriptor for __x64_sys_connect

[Jeftah]

I try to find out the file-descriptor from the systemcall connect.

For this I have created an environment with cargo generate. I selected kprobe as program type and ___x64_sysconnect as endpoint.

As file-descriptor I always get the same number e.g. 3468467552 , no matter if this is 3 or 4.

My source code looks like this:

#[kprobe(name = "test")]
pub fn test(ctx: ProbeContext) -> u32 {
    match try_test(ctx) {
        Ok(ret) => ret,
        Err(ret) => ret,
    }
}

fn try_test(ctx: ProbeContext) -> Result<u32, u32> {
    info!(&ctx, "function __x64_sys_connect called");

    let fd_unsafe_pt: *const u32 = ctx.arg(0).ok_or(1u32)?; 
    let fd: u32 = unsafe {
        bpf_probe_read_kernel(&(*fd_unsafe_pt) as *const u32).map_err(|_| 1u32)?
    };

    info!(&ctx, "fd : {}", fd);

    Ok(0)
}

What could be the reason for this? What error have I made?

Versions:

• aya-bpf v0.1.0 (https://github.com/aya-rs/aya?branch=main#3e9d85f8)
• Kernel: 5.15.69

[alessandrod]

Should be

    let fd: u32 = ctx.arg(0).ok_or(1u32)?; 

The register contains the fd itself, not a pointer to the fd.

[Jeftah]

Thank you for the quick reply.

I have now used your code example, but found that I still do not get a correct number. It is for example 1117863768 but should be 3.

[alessandrod]

Can you show your updated code and also the way you attach it from userspace?

[Jeftah]

Of course, thank you!

Here is my userspace program:

use aya::programs::KProbe;
use aya::{include_bytes_aligned, Bpf};
use aya_log::BpfLogger;
use log::{info, warn};
use tokio::signal;

#[tokio::main]
async fn main() -> Result<(), anyhow::Error> {
    env_logger::init();

    // This will include your eBPF object file as raw bytes at compile-time and load it at
    // runtime. This approach is recommended for most real-world use cases. If you would
    // like to specify the eBPF program at runtime rather than at compile-time, you can
    // reach for `Bpf::load_file` instead.
    #[cfg(debug_assertions)]
    let mut bpf = Bpf::load(include_bytes_aligned!(
        "../../target/bpfel-unknown-none/debug/test"
    ))?;
    #[cfg(not(debug_assertions))]
    let mut bpf = Bpf::load(include_bytes_aligned!(
        "../../target/bpfel-unknown-none/release/test"
    ))?;
    if let Err(e) = BpfLogger::init(&mut bpf) {
        // This can happen if you remove all log statements from your eBPF program.
        warn!("failed to initialize eBPF logger: {}", e);
    }
    let program: &mut KProbe = bpf.program_mut("test").unwrap().try_into()?;
    program.load()?;
    program.attach("__x64_sys_connect", 0)?;

    info!("Waiting for Ctrl-C...");
    signal::ctrl_c().await?;
    info!("Exiting...");

    Ok(())
}

And here the ebpf program:

#![no_std]
#![no_main]

use aya_bpf::{macros::kprobe, programs::ProbeContext,};
use aya_log_ebpf::info;

#[kprobe(name = "test")]
pub fn test(ctx: ProbeContext) -> u32 {
    match try_test(ctx) {
        Ok(ret) => ret,
        Err(ret) => ret,
    }
}

fn try_test(ctx: ProbeContext) -> Result<u32, u32> {
    info!(&ctx, "function __x64_sys_connect called");

    let fd: u32 = ctx.arg(0).ok_or(1u32)?; 

    info!(&ctx, "fd : {}", fd);

    Ok(0)
}

#[panic_handler]
fn panic(_info: &core::panic::PanicInfo) -> ! {
    unsafe { core::hint::unreachable_unchecked() }
}

[alessandrod]

try this:

use aya_bpf::PtRegs;

let regs = PtRegs::new(ctx.arg(0).ok_or(1u32)?);
let fd = regs.arg(0)?.ok_or(2)?;

[Jeftah]

Thank you very much.

Because of compiler errors I had to adjust the source code a bit:

fn try_test(ctx: ProbeContext) -> Result<u32, u32> {
    info!(&ctx, "function __x64_sys_connect called");

    let regs = PtRegs::new(ctx.arg(0).ok_or(1u32)?);
    let fd: u32 = regs.arg(0).ok_or(2u32)?;

    info!(&ctx, "fd : {}", fd);

    Ok(0)
}

Unfortunately, I can no longer load the program:

...
266: (85) call bpf_perf_event_output#25
 R0_w=map_value(id=0,off=0,ks=4,vs=8192,imm=0) R1_w=ctx(id=0,off=0,imm=0) R2_w=map_ptr(id=0,off=0,ks=4,vs=4,imm=0) R3_w=inv4294967295 R4_w=map_value(id=0,off=0,ks=4,vs=8192,imm=0) R5_w=inv184 R6_w=ctx(id=0,off=0,imm=0) R7_w=invP0 R8_w=inv115 R10=fp0 fp-8=????mmmm
last_idx 266 first_idx 0
regs=20 stack=0 before 265: (b7) r5 = 184
267: R0=inv(id=0) R6=ctx(id=0,off=0,imm=0) R7=invP0 R8=inv115 R10=fp0 fp-8=????mmmm
267: (bf) r3 = r6
268: R0=inv(id=0) R3_w=ctx(id=0,off=0,imm=0) R6=ctx(id=0,off=0,imm=0) R7=invP0 R8=inv115 R10=fp0 fp-8=????mmmm
268: (07) r3 += 112
269: R0=inv(id=0) R3_w=ctx(id=0,off=112,imm=0) R6=ctx(id=0,off=0,imm=0) R7=invP0 R8=inv115 R10=fp0 fp-8=????mmmm
269: (bf) r1 = r10
270: R0=inv(id=0) R1_w=fp0 R3_w=ctx(id=0,off=112,imm=0) R6=ctx(id=0,off=0,imm=0) R7=invP0 R8=inv115 R10=fp0 fp-8=????mmmm
270: (07) r1 += -8
271: R0=inv(id=0) R1_w=fp-8 R3_w=ctx(id=0,off=112,imm=0) R6=ctx(id=0,off=0,imm=0) R7=invP0 R8=inv115 R10=fp0 fp-8=????mmmm
271: (b7) r2 = 8
272: R0=inv(id=0) R1_w=fp-8 R2_w=inv8 R3_w=ctx(id=0,off=112,imm=0) R6=ctx(id=0,off=0,imm=0) R7=invP0 R8=inv115 R10=fp0 fp-8=????mmmm
272: (85) call bpf_probe_read#4
last_idx 272 first_idx 267
regs=4 stack=0 before 271: (b7) r2 = 8
273: R0_w=inv(id=0) R6=ctx(id=0,off=0,imm=0) R7=invP0 R8=inv115 R10=fp0 fp-8=mmmmmmmm
273: (15) if r0 == 0x0 goto pc+1
 R0_w=inv(id=0) R6=ctx(id=0,off=0,imm=0) R7=invP0 R8=inv115 R10=fp0 fp-8=mmmmmmmm
274: R0_w=inv(id=0) R6=ctx(id=0,off=0,imm=0) R7=invP0 R8=inv115 R10=fp0 fp-8=mmmmmmmm
274: (05) goto pc+255
530: R0=inv(id=0) R6=ctx(id=0,off=0,imm=0) R7=invP0 R8=inv115 R10=fp0 fp-8=mmmmmmmm
530: (b7) r0 = 0
531: R0_w=inv0 R6=ctx(id=0,off=0,imm=0) R7=invP0 R8=inv115 R10=fp0 fp-8=mmmmmmmm
531: (95) exit
275: R0_w=inv0 R6=ctx(id=0,off=0,imm=0) R7=invP0 R8=inv115 R10=fp0 fp-8=mmmmmmmm
275: (79) r1 = *(u64 *)(r10 -8)
276: R0_w=inv0 R1_w=inv(id=0) R6=ctx(id=0,off=0,imm=0) R7=invP0 R8=inv115 R10=fp0 fp-8=mmmmmmmm
276: (79) r8 = *(u64 *)(r1 +112)
R1 invalid mem access 'inv'
verification time 7384 usec
stack depth 8
processed 274 insns (limit 1000000) max_states_per_insn 0 total_states 2 peak_states 2 mark_read 1

Caused by:
    Permission denied (os error 13)
Failed to run `sudo -E target/release/test`

[alessandrod]

try

fn try_test(ctx: ProbeContext) -> Result<u32, u32> {
    info!(&ctx, "function __x64_sys_connect called");

    let regs: *mut pt_regs = ctx.arg(0).ok_or(1u32)?;
    let fd: u32 = unsafe { bpf_probe_read(&(*regs).rdi) };

    info!(&ctx, "fd : {}", fd);

    Ok(0)
}

(but also, you should really use a tracepoint for this and not a kprobe!)

[Jeftah]

Unfortunately I did not get the source code to run. I get the following compiler message:

error[E0308]: mismatched types
  --> src/main.rs:20:28
   |
20 |     let fd: u32 = unsafe { bpf_probe_read(&(*regs).rdi) };
   |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `u32`, found enum `Result`
   |
   = note: expected type `u32`
              found enum `Result<u64, i64>`

Thank you for the tip that I should use tracepoint, I will try that now.

[Jeftah]

Thanks again for the tip to use tracepoint.

For people who are interested: I have created a new project with the generator. My function now looks like this:

fn try_test_tracepoint(ctx: TracePointContext) -> Result<i64, i64> {
    info!(&ctx, "function __x64_sys_connect called");

    //from: sudo cat /sys/kernel/debug/tracing/events/syscalls/sys_enter_connect/format
    const FD_OFFSET: usize = 16;

    let fd: u32 = unsafe {ctx.read_at(FD_OFFSET)?};
    info!(&ctx, "fd : {}", fd);

    Ok(0)
}

What I don't understand is why it doesn't work properly with kprobe. I played around a bit more and found that the syscalls _x64_sysrecvmsg and _x64_syssendmsg also return wrong values. With the syscall ___x64_syswrite I get fd as written in https://github.com/aya-rs/aya/issues/503#issue-1559894358.