accessing bitfield for sk_protocol

[brendangregg]

I'm not sure bitfields work yet. sk_protocol is a bitfield. Trying to test it:

int kretprobe__inet_csk_accept(struct pt_regs *ctx)
{
        struct sock *newsk = (struct sock *)ctx->ax;
        u32 pid = bpf_get_current_pid_tgid();

        if (newsk == NULL)
                return 0;

        // check this is TCP
        if (newsk->sk_protocol != IPPROTO_TCP)
                return 0;
[...]

gets:

bpf: Permission denied
0: (79) r6 = *(u64 *)(r1 +80)
1: (85) call 14
2: (15) if r6 == 0x0 goto pc+71
 R0=inv R6=inv R10=fp
3: (61) r1 = *(u32 *)(r6 +328)
R6 invalid mem access 'inv'

I also tried:

        u8 protocol = 0;
        bpf_probe_read(&protocol, sizeof(protocol), &newsk->sk_protocol);

which gets:

<bcc-memory-buffer>:16:46: error: address of bit-field requested
        bpf_probe_read(&protocol, sizeof(protocol), &newsk->sk_protocol);
                                                    ^~~~~~~~~~~~~~~~~~~
1 error generated.
Traceback (most recent call last):
  File "./tcpaccept", line 101, in <module>
    b = BPF(text=bpf_text)
  File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 349, in __init__
    raise Exception("Failed to compile BPF module %s" % src_file)
Exception: Failed to compile BPF module 

[4ast]

In general bitfields are quite difficult for bpf_probe_read. Eventually we can add some sophistication to bcc to get offsetof of nearby fields, do the endian math and apply the mask. For now the easiest is to call bpf_probe_read manually like bpf_probe_read(&protocol, 1, ((long)&newsk->sk_wmem_queued) - 3); the above will work because sk_protocol bitfield is 8 bits and nicely aligned.

On Tue, Oct 13, 2015 at 3:16 PM, Brendan Gregg notifications@github.com wrote:

I'm not sure bitfields work yet. sk_protocol is a bitfield. Trying to test it:

int kretprobe__inet_csk_accept(struct pt_regs ctx) { struct sock newsk = (struct sock *)ctx->ax; u32 pid = bpf_get_current_pid_tgid();

    if (newsk == NULL)
            return 0;

    // check this is TCP
    if (newsk->sk_protocol != IPPROTO_TCP)
            return 0;

[...]

gets:

bpf: Permission denied 0: (79) r6 = (u64 )(r1 +80) 1: (85) call 14 2: (15) if r6 == 0x0 goto pc+71 R0=inv R6=inv R10=fp 3: (61) r1 = (u32 )(r6 +328) R6 invalid mem access 'inv'

I also tried:

    u8 protocol = 0;
    bpf_probe_read(&protocol, sizeof(protocol), &newsk->sk_protocol);

which gets:

:16:46: error: address of bit-field requested bpf_probe_read(&protocol, sizeof(protocol), &newsk->sk_protocol); ^~~~~~~~~~~~~~~~~~~ 1 error generated. Traceback (most recent call last): File "./tcpaccept", line 101, in b = BPF(text=bpf_text) File "/usr/lib/python2.7/dist-packages/bcc/**init**.py", line 349, in **init** raise Exception("Failed to compile BPF module %s" % src_file) Exception: Failed to compile BPF module — Reply to this email directly or view it on GitHub https://github.com/iovisor/bcc/issues/270.

[brendangregg]

Thanks, I see what you're doing, but it doesn't like it:

        // check this is TCP
        u8 protocol = 0;
        // workaround for reading the sk_protocol bitfield:
        bpf_probe_read(&protocol, 1, (void *)((long)&newsk->sk_wmem_queued) - 3);
        if (newsk->sk_protocol == IPPROTO_TCP)
                return 0;

# ./tcpaccept 
bpf: Permission denied
0: (79) r6 = *(u64 *)(r1 +80)
1: (85) call 14
2: (15) if r6 == 0x0 goto pc+78
 R0=inv R6=inv R10=fp
3: (b7) r7 = 0
4: (73) *(u8 *)(r10 -1) = r7
5: (bf) r3 = r6
6: (07) r3 += 329
7: (bf) r1 = r10
8: (07) r1 += -1
9: (b7) r2 = 1
10: (85) call 4
11: (61) r1 = *(u32 *)(r6 +328)
R6 invalid mem access 'inv'

[drzaeus77]

Don't you want to:

if (protocol == IPPROTO_TCP)

instead of newsk->

[brendangregg]

Ah, your'e right, sorry. And I'd flipped the operator while testing. Now works.

[matthew-olson-intel]

If I'm not mistaken, this is no longer an issue for the CO-RE tools, right?

https://nakryiko.com/posts/bpf-core-reference-guide/#reading-bitfields-and-integers-of-varying-sizes

I also see BPF_CORE_READ_BITFIELD_PROBED being used in the CO-RE versions of bindsnoop and tcppktlat.

Is there some change on the Python, non-CO-RE side that I'm not aware of?

[brendangregg]

Thanks @matthew-olson-intel , I think the fix here is to add a something to the bcc refererce guide to say "bitfields are tricky but can be done, look over here at CO-RE and BTF", so that anyone searching on "bitfield" finds it. Once that's there we can close this ticket.