Failed to compile BPF module /usr/bin/ebpf-http-statistics.c

[botzill]

Such error while installing plugins on a 3 node k8s cluster:

chdir(/lib/modules/4.19.0-0.bpo.2-amd64/build): No such file or directory
Traceback (most recent call last):
  File "/usr/bin/http-statistics.py", line 294, in <module>
    kernel_inspector = KernelInspector()
  File "/usr/bin/http-statistics.py", line 57, in __init__
    self.bpf = bcc.BPF(EBPF_PROGRAM)
  File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 284, in __init__
    raise Exception("Failed to compile BPF module %s" % src_file)
Exception: Failed to compile BPF module /usr/bin/ebpf-http-statistics.c

[anakaiti]

Having the same issue in a plain docker coreos deployment. Kernel 4.19.23-coreos-r1 on image 2f782b82816b

[dguyonvarch]

Idem with the docker image

docker run --rm -it \                    
          --privileged --net=host --pid=host \
          -v /lib/modules:/lib/modules \
          -v /usr/src:/usr/src \
          -v /sys/kernel/debug/:/sys/kernel/debug/ \
          -v /var/run/scope/plugins:/var/run/scope/plugins \
          --name weaveworksplugins-scope-http-statistics weaveworksplugins/scope-http-statistics
In file included from /virtual/main.c:2:
In file included from /lib/modules/4.15.0-48-generic/build/include/net/sock.h:64:
/lib/modules/4.15.0-48-generic/build/include/linux/filter.h:463:11: error: use of undeclared identifier 'BPF_TAG_SIZE'
        u8                      tag[BPF_TAG_SIZE];
                                    ^
1 error generated.
Traceback (most recent call last):
  File "/usr/bin/http-statistics.py", line 294, in <module>
    kernel_inspector = KernelInspector()
  File "/usr/bin/http-statistics.py", line 57, in __init__
    self.bpf = bcc.BPF(EBPF_PROGRAM)
  File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 284, in __init__
    raise Exception("Failed to compile BPF module %s" % src_file)
Exception: Failed to compile BPF module /usr/bin/ebpf-http-statistics.c

@dholbach : Does this plugin still work ? :wink:

[TomaszKlosinski]

Same story here.

[AntonioDiSalvo]

/lib/modules/4.19.0-0.bpo.2-amd64/build is missing for @botzill , should check if kernel version is the same across the cluster and if you have appropriate module directory. Not in scope of scope-http-plugin (sorry couldn't resist) @dguyonvarch here https://github.com/iovisor/bcc/issues/1407 a similar issue is solved by updating bcc tools to the latest version. Hope it helps investigate & close the issue. @TomaszKlosinski same as in missing BPF_TAG_SIZE declaration ?

[TomaszKlosinski]

@AntonioDiSalvo , no, I meant, same as @botzill (both on k8s and using docker on my laptop):

$ docker run --rm -it --privileged --net=host --pid=host \
          -v /lib/modules:/lib/modules \
          -v /usr/src:/usr/src \
          -v /sys/kernel/debug/:/sys/kernel/debug/ \
          -v /var/run/scope/plugins:/var/run/scope/plugins \
          --name weaveworksplugins-scope-http-statistics weaveworksplugins/scope-http-statistics

chdir(/lib/modules/4.19.121-linuxkit/build): No such file or directory
Traceback (most recent call last):
  File "/usr/bin/http-statistics.py", line 294, in <module>
    kernel_inspector = KernelInspector()
  File "/usr/bin/http-statistics.py", line 57, in __init__
    self.bpf = bcc.BPF(EBPF_PROGRAM)
  File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 284, in __init__
    raise Exception("Failed to compile BPF module %s" % src_file)
Exception: Failed to compile BPF module /usr/bin/ebpf-http-statistics.c

[AntonioDiSalvo]

@AntonioDiSalvo , no, I meant, same as @botzill (both on k8s and using docker on my laptop):

$ docker run --rm -it --privileged --net=host --pid=host \
          -v /lib/modules:/lib/modules \
          -v /usr/src:/usr/src \
          -v /sys/kernel/debug/:/sys/kernel/debug/ \
          -v /var/run/scope/plugins:/var/run/scope/plugins \
          --name weaveworksplugins-scope-http-statistics weaveworksplugins/scope-http-statistics

chdir(/lib/modules/4.19.121-linuxkit/build): No such file or directory
Traceback (most recent call last):
  File "/usr/bin/http-statistics.py", line 294, in <module>
    kernel_inspector = KernelInspector()
  File "/usr/bin/http-statistics.py", line 57, in __init__
    self.bpf = bcc.BPF(EBPF_PROGRAM)
  File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 284, in __init__
    raise Exception("Failed to compile BPF module %s" % src_file)
Exception: Failed to compile BPF module /usr/bin/ebpf-http-statistics.c

Is https://github.com/iovisor/bcc/issues/1407#issuecomment-338704338 of any help ? Also very important : is linux 4.19.121 your mandatory target ? In some recent trials I run there is a problem with a now-blacklisted kprobe which would prevent the execution. The culprit seems to be copy_from_iter as mentioned here https://github.com/iovisor/bcc/issues/2509#issuecomment-530191199 for this issue https://github.com/iovisor/bcc/issues/2509 . Please think about implementing the "solution" as it is a dangerous one (https://cateee.net/lkddb/web-lkddb/KPROBE_EVENTS_ON_NOTRACE.html).