Open changbindu opened 5 years ago
Could you add
#define asm_volatile_goto(x...)
to the beginning of the bpf program to see whether it helps or not?
In the past, the fix likes below (in arch/x86/include/asm/cpufeature.h)
#if defined(__clang__) && !defined(CC_HAVE_ASM_GOTO)
/*
* Workaround for the sake of BPF compilation which utilizes kernel
* headers, but clang does not support ASM GOTO and fails the build.
*/
#ifndef __BPF_TRACING__
#warning "Compiler lacks ASM_GOTO support. Add -D __BPF_TRACING__ to your compiler arguments"
#endif
#define static_cpu_has(bit) boot_cpu_has(bit)
#else
/*
* Static testing of CPU features. Used the same as boot_cpu_has().
* These will statically patch the target code for additional
* performance.
*/
static __always_inline __pure bool _static_cpu_has(u16 bit)
{
asm_volatile_goto("1: jmp 6f\n"
.....
}
#endif
Find this https://reviews.llvm.org/D53765 in llvm phabricator. Looks like llvm asm-goto support still not there. Considering asm_goto is now required to compile x86, some implementation may start to use it without alternative implementation. Fixing the kernel may be viable in the long run. I will see whether we can fix on bcc side (with proper kernel help if needed).
The following bcc change can fix the problem:
[yhs@localhost tools]$ git diff
diff --git a/src/cc/export/helpers.h b/src/cc/export/helpers.h
index 84b7fe6d..ad87ccca 100755
--- a/src/cc/export/helpers.h
+++ b/src/cc/export/helpers.h
@@ -17,6 +17,11 @@ R"********(
#ifndef __BPF_HELPERS_H
#define __BPF_HELPERS_H
+#ifdef asm_volatile_goto
+#undef asm_volatile_goto
+#define asm_volatile_goto(x...)
+#endif
+
#include <uapi/linux/bpf.h>
#include <uapi/linux/if_packet.h>
#include <linux/version.h>
[yhs@localhost tools]$
I will do a little more analysis to make sure I cover all cases before proposing a formal patch.
yonghong-song's patch fixed the problem. Could you consider upstreaming it? thanks.
Sure. Will upstream this or a similar version in the next couple of days.
Just a thought... wouldn't it be safer to "disable" the kernel config options that require asm goto?
Something like this:
diff --git a/src/cc/export/helpers.h b/src/cc/export/helpers.h
index 84b7fe6d92b0..118a3827bf52 100755
--- a/src/cc/export/helpers.h
+++ b/src/cc/export/helpers.h
@@ -17,6 +17,9 @@ R"********(
#ifndef __BPF_HELPERS_H
#define __BPF_HELPERS_H
+#undef CONFIG_CC_HAS_ASM_GOTO
+#undef CONFIG_JUMP_LABEL
+
#include <uapi/linux/bpf.h>
#include <uapi/linux/if_packet.h>
#include <linux/version.h>
Thanks!
The above change does work for opensnoop.py
. What I am thinking is to find a generic solution. For example, if user tries to access linux/access.h
diff --git a/tools/opensnoop.py b/tools/opensnoop.py
index 3d6cc154..85a4f560 100755
--- a/tools/opensnoop.py
+++ b/tools/opensnoop.py
@@ -82,6 +82,7 @@ bpf_text = """
#include <uapi/linux/ptrace.h>
#include <uapi/linux/limits.h>
#include <linux/sched.h>
+#include <linux/uaccess.h>
struct val_t {
u64 id;
The above approach won't work.
In file included from /virtual/main.c:5:
In file included from include/linux/uaccess.h:11:
In file included from ./arch/x86/include/asm/uaccess.h:692:
./arch/x86/include/asm/uaccess_64.h:138:3: error: 'asm goto' constructs are not supported yet
__put_user_asm(*(u16 *)src, (u16 __user *)dst,
^
./arch/x86/include/asm/uaccess.h:477:11: note: expanded from macro '__put_user_asm'
retval = __put_user_failed(x, addr, itype, rtype, ltype, errret); \
^
./arch/x86/include/asm/uaccess.h:472:3: note: expanded from macro '__put_user_failed'
__put_user_goto(x,addr,itype,rtype,ltype,__puflab); \
^
./arch/x86/include/asm/uaccess.h:463:2: note: expanded from macro '__put_user_goto'
asm_volatile_goto("\n" \
^
include/linux/compiler_types.h:188:37: note: expanded from macro 'asm_volatile_goto'
#define asm_volatile_goto(x...) asm goto(x)
^
Do you know whether there is a good way to disable asm_volatile_goto
for linux/uaccess.h
?
Yeah, unfortunately there's not an easy way to disable the usage of asm_volatile_goto()
in uaccess.h
. I think the only way is to patch the kernel, I'll investigate a little bit more and see what I can do.
But, in general, I'd prefer to fail with an explicit error, rather than silently suppress some asm_volatile_goto()
statements, that may produce unexpected behavior. Do you agree?
From bcc perspective, suppressing asm_volatile_goto
is okay since bcc programs should not use any code involving asm_volatile_goto
. All arch-specific asm's in the header file will be eventually thrown way.
In general, undefining config options is not a good idea either. Some config options have impact on kernel data structure size/layout. Undefining them could make bcc's view structure different from kernel.
We can trigger an explicit error even with suppressing asm_volatile_goto
. We just defined it to some asm code which bpf does not support. This will flag out if user happens to use it. I will craft a patch based on this.
From bcc perspective, suppressing
asm_volatile_goto
is okay since bcc programs should not use any code involvingasm_volatile_goto
. All arch-specific asm's in the header file will be eventually thrown way.
True.
In general, undefining config options is not a good idea either. Some config options have impact on kernel data structure size/layout. Undefining them could make bcc's view structure different from kernel.
Also true.
We can trigger an explicit error even with suppressing
asm_volatile_goto
. We just defined it to some asm code which bpf does not support. This will flag out if user happens to use it. I will craft a patch based on this.
Honestly I think this is the best solution (workaround) so far, at least it's a safe approach and we can report an explicit error, instead of giving, potentially, a different "view" of the kernel because a particular statement isn't supported by the compiler.
Thanks for merging the patch!
I have same issue. I'm using CentOS 7.6, bcc-tools-0.6.1-2.el7.x86_64 and kernel 5.0.13 that I compiled by myself. I applied @yonghong-song solution. Erros lessen but it didn't work. Any suggestion?
/usr/share/bcc/tools/opensnoop -T In file included from :3: In file included from /virtual/include/bcc/helpers.h:23: In file included from /lib/modules/5.0.13-1.lbr.x86_64/build/include/linux/log2.h:16: In file included from /lib/modules/5.0.13-1.lbr.x86_64/build/include/linux/bitops.h:19: /lib/modules/5.0.13-1.lbr.x86_64/build/arch/x86/include/asm/bitops.h:209:9: error: 'asm goto' constructs are not supported yet return GEN_BINARY_RMWcc(LOCK_PREFIX _ASM_SIZE(bts), *addr, c, "Ir", nr); ^ /lib/modules/5.0.13-1.lbr.x86_64/build/arch/x86/include/asm/rmwcc.h:60:32: note: expanded from macro 'GEN_BINARY_RMWcc'
^ /lib/modules/5.0.13-1.lbr.x86_64/build/arch/x86/include/asm/rmwcc.h:10:28: note: expanded from macro 'RMWcc_CONCAT'
^ /lib/modules/5.0.13-1.lbr.x86_64/build/arch/x86/include/asm/rmwcc.h:9:30: note: expanded from macro '__RMWcc_CONCAT'
^ note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all) /lib/modules/5.0.13-1.lbr.x86_64/build/arch/x86/include/asm/rmwcc.h:54:2: note: expanded from macro 'GEN_BINARY_RMWcc_6' GEN_RMWcc(op " %[val], " arg0, var, cc, ^ /lib/modules/5.0.13-1.lbr.x86_64/build/arch/x86/include/asm/rmwcc.h:21:2: note: expanded from macro 'GEN_RMWcc' asm_volatile_goto (fullop "; j" #cc " %l[cc_label]" ^ /lib/modules/5.0.13-1.lbr.x86_64/build/include/linux/compiler_types.h:188:37: note: expanded from macro 'asm_volatile_goto'
^ In file included from :3: In file included from /virtual/include/bcc/helpers.h:23: In file included from /lib/modules/5.0.13-1.lbr.x86_64/build/include/linux/log2.h:16: In file included from /lib/modules/5.0.13-1.lbr.x86_64/build/include/linux/bitops.h:19: /lib/modules/5.0.13-1.lbr.x86_64/build/arch/x86/include/asm/bitops.h:255:9: error: 'asm goto' constructs are not supported yet return GEN_BINARY_RMWcc(LOCK_PREFIX _ASM_SIZE(btr), *addr, c, "Ir", nr); ^ /lib/modules/5.0.13-1.lbr.x86_64/build/arch/x86/include/asm/rmwcc.h:60:32: note: expanded from macro 'GEN_BINARY_RMWcc'
^ /lib/modules/5.0.13-1.lbr.x86_64/build/arch/x86/include/asm/rmwcc.h:10:28: note: expanded from macro 'RMWcc_CONCAT'
^ /lib/modules/5.0.13-1.lbr.x86_64/build/arch/x86/include/asm/rmwcc.h:9:30: note: expanded from macro '__RMWcc_CONCAT'
^ note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all) /lib/modules/5.0.13-1.lbr.x86_64/build/arch/x86/include/asm/rmwcc.h:54:2: note: expanded from macro 'GEN_BINARY_RMWcc_6' GEN_RMWcc(op " %[val], " arg0, var, cc, ^ /lib/modules/5.0.13-1.lbr.x86_64/build/arch/x86/include/asm/rmwcc.h:21:2: note: expanded from macro 'GEN_RMWcc' asm_volatile_goto (fullop "; j" #cc " %l[cc_label]" ^ /lib/modules/5.0.13-1.lbr.x86_64/build/include/linux/compiler_types.h:188:37: note: expanded from macro 'asm_volatile_goto'
^ In file included from :3: In file included from /virtual/include/bcc/helpers.h:23: In file included from /lib/modules/5.0.13-1.lbr.x86_64/build/include/linux/log2.h:16: In file included from /lib/modules/5.0.13-1.lbr.x86_64/build/include/linux/bitops.h:19: /lib/modules/5.0.13-1.lbr.x86_64/build/arch/x86/include/asm/bitops.h:308:9: error: 'asm goto' constructs are not supported yet return GEN_BINARY_RMWcc(LOCK_PREFIX _ASM_SIZE(btc), *addr, c, "Ir", nr); ^ /lib/modules/5.0.13-1.lbr.x86_64/build/arch/x86/include/asm/rmwcc.h:60:32: note: expanded from macro 'GEN_BINARY_RMWcc'
^ /lib/modules/5.0.13-1.lbr.x86_64/build/arch/x86/include/asm/rmwcc.h:10:28: note: expanded from macro 'RMWcc_CONCAT'
^ /lib/modules/5.0.13-1.lbr.x86_64/build/arch/x86/include/asm/rmwcc.h:9:30: note: expanded from macro '__RMWcc_CONCAT'
^ note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all) /lib/modules/5.0.13-1.lbr.x86_64/build/arch/x86/include/asm/rmwcc.h:54:2: note: expanded from macro 'GEN_BINARY_RMWcc_6' GEN_RMWcc(op " %[val], " arg0, var, cc, ^ /lib/modules/5.0.13-1.lbr.x86_64/build/arch/x86/include/asm/rmwcc.h:21:2: note: expanded from macro 'GEN_RMWcc' asm_volatile_goto (fullop "; j" #cc " %l[cc_label]" ^ /lib/modules/5.0.13-1.lbr.x86_64/build/include/linux/compiler_types.h:188:37: note: expanded from macro 'asm_volatile_goto'
^ 3 errors generated. Traceback (most recent call last): File "/usr/share/bcc/tools/opensnoop", line 141, in b = BPF(text=bpf_text) File "/usr/lib/python2.7/site-packages/bcc/init.py", line 318, in init raise Exception("Failed to compile BPF text") Exception: Failed to compile BPF text
Could you try this one? https://github.com/iovisor/bcc/commit/a74c0429396f3180ba5b20f4a4eefc233b681cb4
I am using 5.0.0-1018-azure, and trying to use opensnoop-bpfcc which I installed as instructed by the readme, I get the error:
error: 'asm goto' constructs are not supported yet
Do I need to use the upstream package or should this be present already?
@ibotheperfect could you show me your source change with https://github.com/iovisor/bcc/commit/a74c0429396f3180ba5b20f4a4eefc233b681cb4 and the error message? I would like to see why it does not work?
@gowenpotato this probably needs change like https://github.com/iovisor/bcc/commit/a74c0429396f3180ba5b20f4a4eefc233b681cb4.
That is, the helpers.h
should have the following code (in trunk already):
#ifdef asm_volatile_goto
#undef asm_volatile_goto
#endif
#define asm_volatile_goto(x...) asm volatile("invalid use of asm_volatile_goto")
I had the same issue on CentOS 7.6 with 5.3 kernel from elrepo. Then I built LLVM 9, built bcc from git and things work.
installation steps mentioned here helped me http://www.brendangregg.com/blog/2016-06-14/ubuntu-xenial-bcc-bpf.html
# echo "deb [trusted=yes] https://repo.iovisor.org/apt/xenial xenial-nightly main" | sudo tee /etc/apt/sources.list.d/iovisor.list
deb [trusted=yes] https://repo.iovisor.org/apt/xenial xenial-nightly main
# sudo apt-get update
# sudo apt-get install bcc-tools
Thanks @palashkulsh, that helped!
For anyone who's looking for a quick fix that works not only for xenial but for other Ubuntu versions, the universal snippet is:
echo "deb [trusted=yes] https://repo.iovisor.org/apt/$(lsb_release -c -s) $(lsb_release -c -s)-nightly main" | sudo tee /etc/apt/sources.list.d/iovisor.list
sudo apt-get update && sudo apt-get install -y bcc-tools
Note that for some reason tools don't appear in the PATH, you'll have to find them in /usr/share/bcc/tools
.
Thanks @kirs , it works for me on ubuntu 18.04
Thanks @palashkulsh, that helped!
For anyone who's looking for a quick fix that works not only for xenial but for other Ubuntu versions, the universal snippet is:
echo "deb [trusted=yes] https://repo.iovisor.org/apt/$(lsb_release -c -s) $(lsb_release -c -s)-nightly main" | sudo tee /etc/apt/sources.list.d/iovisor.list sudo apt-get update && sudo apt-get install -y bcc-tools
Note that for some reason tools don't appear in the PATH, you'll have to find them in
/usr/share/bcc/tools
.
The workaround listed above - installing from the iovisor repo, does not work for me.
root@ip-10-0-2-253:~# /usr/share/bcc/tools/cachestat
In file included from /virtual/main.c:2:
In file included from include/uapi/linux/ptrace.h:143:
In file included from ./arch/x86/include/asm/ptrace.h:5:
./arch/x86/include/asm/segment.h:266:2: error: expected '(' after 'asm'
alternative_io ("lsl %[seg],%[p]",
^
./arch/x86/include/asm/alternative.h:240:2: note: expanded from macro 'alternative_io'
asm_inline volatile (ALTERNATIVE(oldinstr, newinstr, feature) \
^
include/linux/compiler_types.h:214:24: note: expanded from macro 'asm_inline'
#define asm_inline asm __inline
^
1 error generated.
Traceback (most recent call last):
File "/usr/share/bcc/tools/cachestat", line 96, in <module>
b = BPF(text=bpf_text)
File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 343, in __init__
raise Exception("Failed to compile BPF module %s" % (src_file or "<text>"))
Exception: Failed to compile BPF module <text>
Relevant system info:
root@ip-10-0-2-253:~# uname -a && lsb_release -a
Linux ip-10-0-2-253 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04.6 LTS
Release: 18.04
Codename: bionic
Either disable 'asm goto' in kernel or add support for llvm?
$ sudo argdist -C 't:irq:irq_handler_entry():int:args->irq'