iovisor / bcc

BCC - Tools for BPF-based Linux IO analysis, networking, monitoring, and more
Apache License 2.0
20.57k stars 3.88k forks source link

biotop crashes on kernel 5.8.0-rc5: cannot attach kprobe, probe entry may not exist #3023

Open cmurf opened 4 years ago

cmurf commented 4 years ago
# /usr/share/bcc/tools/biotop
cannot attach kprobe, probe entry may not exist
Traceback (most recent call last):
  File "/usr/share/bcc/tools/biotop", line 181, in <module>
    b.attach_kprobe(event="blk_account_io_completion",
  File "/usr/lib/python3.8/site-packages/bcc/__init__.py", line 676, in attach_kprobe
    raise Exception("Failed to attach BPF program %s to kprobe %s" %
Exception: Failed to attach BPF program b'trace_req_completion' to kprobe b'blk_account_io_completion'

kernel-5.8.0-0.rc5.1.fc33.x86_64+debug clang-10.0.0-2.fc32.x86_64 bcc-tools-0.15.0-2.fc32.x86_64

Works OK with kernel-5.7.6-200.fc32.x86_64

yonghong-song commented 4 years ago

The following commit fixed the issue: https://github.com/iovisor/bcc/commit/95c9229ea9f029a1b9e8dcbe86fc67f037c0dfa2 Could you try latest trunk bcc?

cmurf commented 4 years ago

I'll ask the Fedora maintainer if it's possible to apply that commit.

Is this the same problem or should I file a separate issue?

# /usr/share/bcc/tools/xfsslower 5
cannot attach kprobe, probe entry may not exist
Traceback (most recent call last):
  File "/usr/share/bcc/tools/xfsslower", line 267, in <module>
    b.attach_kprobe(event="xfs_file_read_iter", fn_name="trace_rw_entry")
  File "/usr/lib/python3.9/site-packages/bcc/__init__.py", line 676, in attach_kprobe
    raise Exception("Failed to attach BPF program %s to kprobe %s" %
Exception: Failed to attach BPF program b'trace_rw_entry' to kprobe b'xfs_file_read_iter'
yonghong-song commented 4 years ago

Exception: Failed to attach BPF program b'trace_rw_entry' to kprobe b'xfs_file_read_iter'

This is a different issue. What is your kernel version? Could you check whether xfs_file_read_iter symbol exists in your /proc/kallsyms file?

cmurf commented 4 years ago

5.8.0-0.rc7.1.fc33.x86_64 and 5.8.0-0.rc7.1.fc33.x86_64+debug

$ cat /proc/kallsyms | grep xfs_file_read_iter
$ cat /proc/kallsyms | grep xfs
0000000000000000 t copy_to_xfs_dqblk
0000000000000000 B selinuxfs_mount
yonghong-song commented 4 years ago

This means the symbol xfs_file_read_iter is not available in the system for xfsslower to attach. Could you check your system config? Maybe you did not have xfs enabled (CONFIG_XFS_FS)?

cmurf commented 4 years ago

It's user error. The xfs module hasn't yet loaded. Simply doing modprobe xfs and now it works OK. Sorry for the noise.

kunzeng-ch commented 2 years ago

It's user error. The xfs module hasn't yet loaded. Simply doing modprobe xfs and now it works OK. Sorry for the noise.

Hello! I had the same problem as you! What do you mean by the above description. how to solve this problem?