Kmesh eBPF log user space dumping

[nlgwcy]

What would you like to be added: Kmesh logs governance process information via BPF_LOG. Logs are expected to be dumped to the user space:

1. Log information should be dumped to user space via ringbuf.
2. In user space, the log information should be obtained through epoll and formatted for printing.

reference: https://github.com/anakryiko/bpf-ringbuf-examples/blob/main/src/ringbuf-output.bpf.c

Why is this needed: The current API has the following issues:

1. Logs are recorded in memory through trace_pipe and can be viewed using bpftool prog tracelog, limited space.
2. Format parameters such as "%pI4h" are supported latter than v5.13 kernel version, not yet supported in 5.10 (such as openEuler 22.03 SP3),
3. Only three parameters are supported.

[bfforever]

/assign

[hzxuzhonghu]

Logs are recorded in memory through trace_pipe and can be viewed using bpftool prog tracelog, limited space.

Not an expert on this, does it occupy kernel memory?

[bfforever]

Maybe we need to firstly define some simple fixed format，because ebpf prog seems not support var args. For example:

• no args BPF_LOG(ERR, SOCKOPS, "set sockops cb failed!\n")
• 1 int args or 2 int args BPF_LOG(ERR, KMESH, "set bypass value failed!, err is %d\n", err) and BPF_LOG(DEBUG, KMESH, "bpf find frontend addr=[%pI4h:%u]\n", &ip, bpf_ntohs(ctx->user_port));, or more int args.
• Also can add a flag to represent which kind of log is occured. Just as this implement https://github.com/cilium/cilium/blob/d18d41e6995679624980671c46b25e5a80d40136/bpf/lib/dbg.h#L8

I am not sure whether the format should mix with string args %s, the situation may become complex. Dose anything need to supplement?

The struct possible like this:

struct log_event {
    __u8 type;
    __u8 level;
    char fmt[MAX_MSG_LEN];
    __u32 arg1;
    __u32 arg2;
    __u32 arg3;
};

[nlgwcy]

Logs are recorded in memory through trace_pipe and can be viewed using bpftool prog tracelog, limited space.

Not an expert on this, does it occupy kernel memory?

This is a debugfs file system, and it will be cleared after the system reboot.

[nlgwcy]

Maybe we need to firstly define some simple fixed format，because ebpf prog seems not support var args. For example:

• no args BPF_LOG(ERR, SOCKOPS, "set sockops cb failed!\n")
• 1 int args or 2 int args BPF_LOG(ERR, KMESH, "set bypass value failed!, err is %d\n", err) and BPF_LOG(DEBUG, KMESH, "bpf find frontend addr=[%pI4h:%u]\n", &ip, bpf_ntohs(ctx->user_port));, or more int args.
• Also can add a flag to represent which kind of log is occured. Just as this implement https://github.com/cilium/cilium/blob/d18d41e6995679624980671c46b25e5a80d40136/bpf/lib/dbg.h#L8

I am not sure whether the format should mix with string args %s, the situation may become complex. Dose anything need to supplement?

The struct possible like this:

struct log_event {
    __u8 type;
    __u8 level;
    char fmt[MAX_MSG_LEN];
    __u32 arg1;
    __u32 arg2;
    __u32 arg3;
};

As describe, the situation may become complex because of the formatting parameters. IMO, we can use BPF_SNPRINTF to store format string, and then record into ringbuf map for user-space dumping. But I didn't test the feasibility, for details about BPF_SNPRINTF usage example, see: https://github.com/torvalds/linux/commit/c2e39c6bdc7eb48459ec1d34d4f27eb82299f4b7

[bfforever]

Resolve 5.10 problem.

1. Define a flag to indicate whether vargs include ipaddr info.
2. By kernel version, determine using BPF_LOG_K or BPF_LOG_U, these 2 marco be included in BPF_LOG.