namhyung / uftrace

Function graph tracer for C/C++/Rust/Python
https://uftrace.github.io/slide/
GNU General Public License v2.0
3.05k stars 474 forks source link

investigate shmem buffer allocation failure during kernel tracing #1624

Open honggyukim opened 1 year ago

honggyukim commented 1 year ago

I have seen many lost events during kernel tracing when the workload is heavy. I saw that the lost comes from the shmem buffer allocation failure at https://github.com/namhyung/uftrace/blob/v0.13.1/libmcount/record.c#L107-L116.

This makes the record make as LOST at https://github.com/namhyung/uftrace/blob/v0.13.1/libmcount/record.c#L165-L178.

We should investigate why shmem buffer allocation fails here.

honggyukim commented 1 year ago

This problem might be related to rlimit as @namhyung mentioned.

honggyukim commented 1 year ago

We can try to see if --num-thread or --rt-prio makes difference.