Log flooded with: ERR [eBPF events #1] error: unexpected EOF

[Selora]

Describe the bug

opensnitch.log gets flooded with the following message:

[2024-02-29 02:24:55] ERR [eBPF events #1] error: unexpected EOF

Include the following information:

• OpenSnitch version: 1.6.2
• OS: ArchLinux
• Window Manager: Doesn't matter
• Kernel version: Linux 6.7.6-arch1-1 #1 SMP PREEMPT_DYNAMIC Fri, 23 Feb 2024 16:31:48 +0000 x86_64 GNU/Linux

To Reproduce Not sure what I can do. It spams several of these events. In a day, before the log files are rotated, they can easily grow to several dozen MB. Latest is 45MB.

[2024-02-18 04:05:11]  ERR  [eBPF events #1] error: unexpected EOF
[2024-02-18 04:05:11]  ERR  [eBPF events #1] error: unexpected EOF
[2024-02-18 04:05:11]  ERR  [eBPF events #1] error: unexpected EOF
[2024-02-18 04:05:11]  ERR  [eBPF events #1] error: unexpected EOF
[2024-02-18 04:05:11]  ERR  [eBPF events #1] error: unexpected EOF
[2024-02-18 04:05:11]  ERR  [eBPF events #1] error: unexpected EOF
[2024-02-18 04:05:11]  ERR  [eBPF events #2] error: unexpected EOF
[2024-02-18 04:05:11]  ERR  [eBPF events #0] error: unexpected EOF
[2024-02-18 04:05:11]  ERR  [eBPF events #3] error: unexpected EOF
[2024-02-18 04:05:11]  ERR  [eBPF events #1] error: unexpected EOF
[2024-02-18 04:05:20]  ERR  [eBPF events #2] error: unexpected EOF
[2024-02-18 04:05:20]  ERR  [eBPF events #0] error: unexpected EOF

These are the last events before my logs were flooded:

[2024-02-14 14:43:30]  IMP  Got signal: terminated
[2024-02-14 14:43:30]  ERR  Connection to the UI service lost.
[2024-02-14 14:43:36]  WAR  queue (2569042938) stuck, closing by timeout
[2024-02-14 14:43:36]  WAR  Queue.destroy() idx=2569042938, nfq_close() not closed: -1
[2024-02-14 14:44:21]  IMP  Start writing logs to /var/log/opensnitchd.log
[2024-02-14 14:44:21]  ERR  [eBPF events #0] error: unexpected EOF

Additional context

Everything seems to be working. If I switch the interception backend to audit or process, then this goes away. However, with the eBPF backend, interception still works. I would like to provide more info but I have very little understanding of how eBPF programs works.

[gustavo-iniguez-goya]

Hi @Selora ,

This could be a problem with mixed versions of the ebpf modules, i.e., v1.6.5 modules loaded by daemon v1.6.2. There has been a recent change on v1.6.5 that could cause these errors if they're loaded by the daemon v1.6.2 Verify that you don't have for example the AUR

Could you post the output of : objdump -h /usr/lib/opensnitchd/ebpf/opensnitch-procs.o ?

if in the output appears: sys_exit_execve/execveat, then the modules are for v1.6.5 version, no v1.6.2.

Also check that you don't have other modules under /etc/opensnitchd/ or /usr/local/lib/opensnitchd/ebpf/

[Selora]

Ah-ah! You got it!! Haven't solved it yet, but here's what I got setup:

pacman -Qs opensnitch
local/opensnitch 1.6.4-1
    A GNU/Linux application firewall
local/opensnitch-ebpf-module 1.6.5-1
    eBPF process monitor module for opensnitch

It looks like the arch package in the extra repository haven't received its version bump yet, but the AUR ebpf module is tracking releases properly.

I'll update when I figure out how I want to deal with version mismatch, but everything coincides: the time of the 1.6.5 release, version mismatch, etc.

Thanks a lot for the help, and thanks for maintaining this wonderful piece of software!

[Marcool04]

Good catch!

I was having the exact same issue as @Selora. I tried building the opensnitch PKGBUILD after a simple version-bump but that isn't working (linker error in the build stage... beyond me). Another solution is to rebuild opensnitch-ebpf-module from the 1.6.4 tag of the AUR package. That does work:

$ git clone https://aur.archlinux.org/opensnitch-ebpf-module.git
[snip]
$ cd opensnitch-ebpf-module
$ git checkout HEAD~1
$ extra-x86_64-build
[snip]
$ sudo pacman -U opensnitch-ebpf-module-1.6.4-1-x86_64.pkg.tar.zst
$ sudo systemctl restart opensnitchd

[nnsee]

Hello, maintainer of the ebpf module AUR package here. Bumping the ebpf module without checking whether the main opensnitch package has been updated was my bad, and won't happen in the future. For now, I can't exactly downgrade the package so we'll just have to wait until the main package is updated. Those who have already upgraded should downgrade like the above comment mentions. Sorry about that!

[gustavo-iniguez-goya]

Hey @nnsee , thank you for maintaining that package!!

I'll add a check to disable this module if there're more than n errors not to flood the logs, and display a visual warning to let the users know that the module is incompatible.

[Marcool04]

A big thank you @gustavo-iniguez-goya for such a constructive and helpful response! I feel like a number of devs would have said "somebody else is messing up" and done nothing here. It's great that you're taking this as an opportunity to make things better on your end. Way to go! 👏

[atomGit]

tried to downgrade but extra-x86_64-build is not available - is that supposed to be in the opensnitch-ebpf-module repo, or is that part of some other package?

also thanks to @nnsee for chiming in here and handling the problem for the future

[lainedfles]

@atomGit extra-x86_64-build is a clean-build utlility provided by the devtools package. 👍🏻

[nnsee]

I've just pushed 1.6.5-2 of the ebpf module on the AUR, which is really 1.6.4 in disguise, and should fix most issues until 1.6.5 is actually released and the ebpf module package updated. Most people having issues right now should upgrade the opensnitch-ebpf-module package and it should in theory be fine for the time being.

[atomGit]

perfect - thanks!

[molitona]

@nnsee @gustavo-iniguez-goya

i get this always when changing monitor method to ebpf on arch, why ? i get proc as default one always!

here is daemon+ui versions

i have opensnitch-ebpf-module 1.6.5-3 latest one on aur opensnitch-git (1.6.0rc5.r110.01069d3-1)

[Marcool04]

@molitona : you have different versions of opensnitch-ebpf-module and opensnitch. Your opensnitch is 1.6.0 and the module is 1.6.5. These have to be the same otherwise you'll get errors, as indicated clearly in the first response to this bug report : https://github.com/evilsocket/opensnitch/issues/1099#issuecomment-1984354713

Also, this comment is pinned on the opensnitch-ebpf-module AUR page:

This is the latest RELEASE version of opensnitch's eBPF module. It is meant to be used with the regular opensnitch package, not the -git version in the AUR. If you're using the -git version of opensnitch, you're looking for this version of the eBPF module package instead.

I intend to keep this up to date with the OpenSnitch releases (as soon as the main package updates).

[molitona]

Good catch!

I was having the exact same issue as @Selora. I tried building the opensnitch PKGBUILD after a simple version-bump but that isn't working (linker error in the build stage... beyond me). Another solution is to rebuild opensnitch-ebpf-module from the 1.6.4 tag of the AUR package. That does work:

$ git clone https://aur.archlinux.org/opensnitch-ebpf-module.git
[snip]
$ cd opensnitch-ebpf-module
$ git checkout HEAD~1
$ extra-x86_64-build
[snip]
$ sudo pacman -U opensnitch-ebpf-module-1.6.4-1-x86_64.pkg.tar.zst
$ sudo systemctl restart opensnitchd

cloning doesn't work. downloaded snapshot don't contain .git

@Marcool04

[Marcool04]

@molitona this isn't really the right place to discuss this. The issue is closed because it was not an issue with opensnitch at all. It had to do with a premature version bump by the AUR package maintainer for opensitch-epbf-module.

That whole situation is resolved now, so none of the above is relevant to your own problems. As I pointed out you had a mismatch between versions of opensnitch and the epbf-module.

About this:

cloning doesn't work. downloaded snapshot don't contain .git

I have no idea what you mean.

[molitona]

I've just pushed 1.6.5-2 of the ebpf module on the AUR, which is really 1.6.4 in disguise, and should fix most issues until 1.6.5 is actually released and the ebpf module package updated. Most people having issues right now should upgrade the opensnitch-ebpf-module package and it should in theory be fine for the time being.

with latest opensitch on official repo + ur latest one on aur of ebpf i still get

/usr/lib/python3.12/site-packages/opensnitch/ui_pb2_grpc.py:21: RuntimeWarning: The grpc package installed is at version 1.62.1, but the generated code in ui_pb2_grpc.py depends on grpcio>=1.64.0. Please upgrade your grpc module to grpcio>=1.64.0 or downgrade your generated code using grpcio-tools<=1.62.1. This warning will become an error in 1.65.0, scheduled for release on June 25, 2024.
  warnings.warn(
     ~ OpenSnitch GUI - 1.6.5.1 ~
    protobuf: 4.25.3 - grpc: 1.62.1
-------------------------------------------------- 

gRPC Max Message Length: None
                  Bytes: 4194304
is new file, or IN MEMORY, setting initial schema version
setting schema version to: 3
setting schema version to: 3
Setting journal_mode:  OFF
Setting DB memory optimizations
schema version: 3
db schema is up to date
Loading translations: /usr/lib/python3.12/site-packages/opensnitch/utils/../i18n locale: en_US
exception loading ipasn db: No module named 'pyasn'
Install python3-pyasn to display IP's network name.
'_Server' object has no attribute 'add_registered_method_handlers'

[molitona]

@molitona this isn't really the right place to discuss this. The issue is closed because it was not an issue with opensnitch at all. It had to do with a premature version bump by the AUR package maintainer for opensitch-epbf-module.

That whole situation is resolved now, so none of the above is relevant to your own problems. As I pointed out you had a mismatch between versions of opensnitch and the epbf-module.

About this:

cloning doesn't work. downloaded snapshot don't contain .git

I have no idea what you mean.

type that cloning cmd in terminal and see

i have latest opensnitch and the epbf-module now and got

/usr/lib/python3.12/site-packages/opensnitch/ui_pb2_grpc.py:21: RuntimeWarning: The grpc package installed is at version 1.62.1, but the generated code in ui_pb2_grpc.py depends on grpcio>=1.64.0. Please upgrade your grpc module to grpcio>=1.64.0 or downgrade your generated code using grpcio-tools<=1.62.1. This warning will become an error in 1.65.0, scheduled for release on June 25, 2024.
  warnings.warn(
     ~ OpenSnitch GUI - 1.6.5.1 ~
    protobuf: 4.25.3 - grpc: 1.62.1
-------------------------------------------------- 

gRPC Max Message Length: None
                  Bytes: 4194304
is new file, or IN MEMORY, setting initial schema version
setting schema version to: 3
setting schema version to: 3
Setting journal_mode:  OFF
Setting DB memory optimizations
schema version: 3
db schema is up to date
Loading translations: /usr/lib/python3.12/site-packages/opensnitch/utils/../i18n locale: en_US
exception loading ipasn db: No module named 'pyasn'
Install python3-pyasn to display IP's network name.
'_Server' object has no attribute 'add_registered_method_handlers'

cloning command don't clone anything test it

[Marcool04]

@molitona this isn't really the right place to discuss this. The issue is closed because it was not an issue with opensnitch at all. It had to do with a premature version bump by the AUR package maintainer for opensitch-epbf-module. That whole situation is resolved now, so none of the above is relevant to your own problems. As I pointed out you had a mismatch between versions of opensnitch and the epbf-module. About this:

cloning doesn't work. downloaded snapshot don't contain .git

I have no idea what you mean.

type that cloning cmd in terminal and see

I copied that from my own terminal months ago... And it works just fine now too.

To reiterate though: this is not the place to debug your issues installing these tools. Please try IRC: ircs://irc.libera.chat/archlinux or the arch forums: https://bbs.archlinux.org/

[molitona]

@molitona this isn't really the right place to discuss this. The issue is closed because it was not an issue with opensnitch at all. It had to do with a premature version bump by the AUR package maintainer for opensitch-epbf-module. That whole situation is resolved now, so none of the above is relevant to your own problems. As I pointed out you had a mismatch between versions of opensnitch and the epbf-module. About this:

cloning doesn't work. downloaded snapshot don't contain .git

I have no idea what you mean.

type that cloning cmd in terminal and see

I copied that from my own terminal months ago... And it works just fine now too.

To reiterate though: this is not the place to debug your issues installing these tools. Please try IRC: ircs://irc.libera.chat/archlinux or the arch forums: https://bbs.archlinux.org/

for some reason i dunno cloning is stuck for me. i solved it by cloning in other machine. could u tell me what extra-x86_64-build do ?

[molitona]

got this why @Marcool04

sorry i cannot use forum now and never used irc

╰─ extra-x86_64-build                                                                                                               ─╯
==> ERROR: '/var/lib/archbuild/extra-x86_64/root' does not appear to be an Arch chroot.
==> ERROR: Aborting...