aya-rs / aya

Aya is an eBPF library for the Rust programming language, built with a focus on developer experience and operability.
https://aya-rs.dev/book/
Apache License 2.0
3.2k stars 286 forks source link

long tc program names are not truncated, causing a netlink error #610

Open Sherlock-Holo opened 1 year ago

Sherlock-Holo commented 1 year ago

when this error message reported, what is the real reason? I can't attach this https://github.com/Sherlock-Holo/mahiro/commit/fe85104164ade7fd7fec944cfb2eae2f7efbb73e#diff-b47e6e512f3e362859296f1fca2c484e0699ca24db91fcf4813a04af127701a4R20 bpf program, however, I can attach https://github.com/Sherlock-Holo/mahiro/commit/fe85104164ade7fd7fec944cfb2eae2f7efbb73e#diff-b47e6e512f3e362859296f1fca2c484e0699ca24db91fcf4813a04af127701a4R16

Sherlock-Holo commented 1 year ago

oh... I change #[classifier(name = "dnat_ingress_with_redirect_route")] to #[classifier(name = "dnat_ingress_with_redirect")] and it works that makes me more confused...

dave-tucker commented 1 year ago

I would guess the name is too long. Would be interesting where the ENOSPC comes from though. Either running with strace or Aya's debug logging enabled would be helpful.

Sherlock-Holo commented 1 year ago

it seems the error reported by this https://github.com/aya-rs/aya/blob/58f1ecbf0089194d729327692adca6391fc24932/aya/src/sys/netlink.rs#L491

dave-tucker commented 1 year ago

Ah yeah here's the issue. Name should probably be truncated here: https://github.com/aya-rs/aya/blob/58f1ecbf0089194d729327692adca6391fc24932/aya/src/sys/netlink.rs#L134

pooladkhay commented 1 year ago

According to the kernel code, the name can be up to 256 bytes long: https://github.com/torvalds/linux/blob/master/net/sched/cls_bpf.c#L28

#define CLS_BPF_NAME_LEN    256

And realised you have set the total length of attributes to 64 bytes: https://github.com/aya-rs/aya/blob/main/aya/src/sys/netlink.rs#L254

#[repr(C)]
struct TcRequest {
    header: nlmsghdr,
    tc_info: tcmsg,
    attrs: [u8; 64],
}

I increased the length of the attributes and it works fine until the name reaches 256 bytes limit enforced by kernel and this error appears:

Error: netlink error while attaching ebpf program to tc

Caused by:
    Invalid argument (os error 22)

So we can conclude that the limit is enforced by Aya not the kernel. Are there any particular reasons behind choosing 64 as the length of the attributes ?

tamird commented 1 year ago

@pooladkhay could you send a patch that increases this limit with a test? I'd be happy to guide you through writing the test.

pooladkhay commented 1 year ago

@pooladkhay could you send a patch that increases this limit with a test? I'd be happy to guide you through writing the test.

@tamird Yeah I'd love to do that, For the actual size, from what I saw it always requires 33 bytes for values other than name (289 bytes in total) but it just sounds like a magic number. I'll try to figure out the reason behind that 33 and if it can ever increase and will send a patch.

In the meantime I'd really appreciate it if you tell me more about the test.

tamird commented 1 year ago

Ah, sorry I missed your reply. The tests are in the integration-test directory. I think the smoke test is closest to what you're looking for, but have a look around. You can run these tests locally using cargo xtask integration-test -- <my-test-name>.

tamird commented 1 year ago

Actually, this may be the test you're looking for: https://github.com/aya-rs/aya/blob/445cb8b46318a13a94e10e11000232d4bd5b23af/test/integration-test/src/tests/load.rs#L17

pooladkhay commented 1 year ago

@tamird No worries, Thank you, I'll update the comment and add the test tomorrow.