Closed kortschak closed 1 month ago
thank you @kortschak for the detailed information!
I just installed GUI v1.4.1 deb package on Ubuntu 18.04, then download opensnitch1.4.0 branch, and copy all *.py files to /usr/lib/python3/dist-packages/opensnitch/. It works fine. In my case I installed grpcio manually: pip3 install grpcio==1.16.1
Can you post your grpcio and protobuf versions? On the other hand, does v1.4.1 version works fine ?
grpcio==1.37.0 protobuf==3.15.8
Installing the python3-opensnitch-ui_1.4.1-1_all.PIP.deb package and bypassing the locally built ui still results in a segfault (so no running the local ui at all). Same debug output.
$ GRPC_VERBOSITY=debug /usr/bin/opensnitch-ui
Loading translations: /usr/local/lib/python3.6/dist-packages/opensnitch/i18n locale: en_AU
D1204 07:00:21.972334097 18846 ev_posix.cc:173] Using polling engine: epollex
D1204 07:00:21.972459367 18846 lb_policy_registry.cc:42] registering LB policy factory for "grpclb"
D1204 07:00:21.972465697 18846 lb_policy_registry.cc:42] registering LB policy factory for "priority_experimental"
D1204 07:00:21.972487587 18846 lb_policy_registry.cc:42] registering LB policy factory for "weighted_target_experimental"
D1204 07:00:21.972492378 18846 lb_policy_registry.cc:42] registering LB policy factory for "pick_first"
D1204 07:00:21.972496641 18846 lb_policy_registry.cc:42] registering LB policy factory for "round_robin"
D1204 07:00:21.972501281 18846 dns_resolver_ares.cc:491] Using ares dns resolver
D1204 07:00:21.972525858 18846 certificate_provider_registry.cc:33] registering certificate provider factory for "file_watcher"
D1204 07:00:21.972530502 18846 lb_policy_registry.cc:42] registering LB policy factory for "cds_experimental"
D1204 07:00:21.972537722 18846 lb_policy_registry.cc:42] registering LB policy factory for "xds_cluster_impl_experimental"
D1204 07:00:21.972556179 18846 lb_policy_registry.cc:42] registering LB policy factory for "xds_cluster_resolver_experimental"
D1204 07:00:21.972560415 18846 lb_policy_registry.cc:42] registering LB policy factory for "xds_cluster_manager_experimental"
D1204 07:00:21.973838696 18846 tcp_server_utils_posix_common.cc:163] Node does not support SO_ZEROCOPY, continuing.
D1204 07:00:22.257388580 18860 tcp_posix.cc:1831] cannot set inq fd=41 errno=95
D1204 07:00:22.257539869 18860 tcp_posix.cc:1831] cannot set inq fd=44 errno=95
zsh: segmentation fault (core dumped) GRPC_VERBOSITY=debug /usr/bin/opensnitch-ui
sorry @kortschak , I've been unable to compile that grpcio version on my ubuntu 18, and the grpcio debug logs doesn't seem to provide useful information :(
I installed grpcio 1.37.0 on kubuntu 20, but it worked fine.
As it seems to work fine for me with grpcio==1.16.1, could you uninstall your grpcio version and install 1.16? pip3 install grpcio==1.16.0
Be sure to delete every grpcio that you might have: pip3 uninstall grpcio
(as user and root) . Sometimes there're conflicts among versions installed.
Thanks. I'll give that a go. Python is such a wonderful language.
That still segfaults — though earlier. This seems pretty brittle. It also troubles me that pip is called by root.
$ GRPC_VERBOSITY=debug opensnitch-ui
Loading translations: /usr/local/lib/python3.6/dist-packages/opensnitch/i18n locale: en_AU
I1205 06:39:43.088557043 19473 ev_epoll1_linux.cc:114] grpc epoll fd: 36
D1205 06:39:43.088598117 19473 ev_posix.cc:140] Using polling engine: epoll1
D1205 06:39:43.088624646 19473 dns_resolver.cc:331] Using native dns resolver
zsh: segmentation fault (core dumped) GRPC_VERBOSITY=debug opensnitch-ui
sorry @kortschak , I have no idea why it's crashing.
Last days I've been analyzing a GUI segfault, and as far as I can tell, this usually occurs when accesing the DB (well, QsqlDatabase object really) from different threads, specially when saving events to disk. But since your segfault occurs always, I think that it might be thing something different.
I'll add logging options to the GUI, because in issues like this one it'd come in handy.
Thanks for looking in to it.
I have an additional observation, though I don't know how useful it will be; if opensnitchd
is not running, then the GUI does not segfault, starting the daemon however will cause the GUI to segfault.
Thank you @kortschak , that's useful. It could be related to database updates when a new node connects.
In order to dump a stacktrace when the segfault occurs, you can enable faulthandler in service.py. I've attached the file with the changes (2 lines). service.py.txt
Backup /usr/lib/python3/dist-packages/opensnitch/service.py
and replace it with the one attached, then restart the GUI.
Here is the result of reproducing segfaults with the faulthandler enabled. It does seem to confirm your suggestion that it is due to database interaction, the fault is at a QSqlQuery
call.
First case, starting opensnitch-ui with the daemon running.
$ opensnitch-ui
Loading translations: /usr/local/lib/python3.6/dist-packages/opensnitch/i18n locale: en_AU
Fatal Python error: Segmentation fault
Thread 0x00007fc82f7fe700 (most recent call first):
File "/usr/local/lib/python3.6/dist-packages/opensnitch/service.py", line 499 in Ping
File "/home/user/.local/lib/python3.6/site-packages/grpc/_server.py", line 385 in _call_behavior
File "/home/user/.local/lib/python3.6/site-packages/grpc/_server.py", line 487 in _unary_response_in_pool
File "/usr/lib/python3.6/concurrent/futures/thread.py", line 62 in run
File "/usr/lib/python3.6/concurrent/futures/thread.py", line 67 in _worker
File "/usr/lib/python3.6/threading.py", line 864 in run
File "/usr/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/usr/lib/python3.6/threading.py", line 884 in _bootstrap
Current thread 0x00007fc82ffff700 (most recent call first):
File "/usr/local/lib/python3.6/dist-packages/opensnitch/database.py", line 250 in update
File "/usr/local/lib/python3.6/dist-packages/opensnitch/nodes.py", line 264 in update
File "/usr/local/lib/python3.6/dist-packages/opensnitch/nodes.py", line 46 in add
File "/usr/local/lib/python3.6/dist-packages/opensnitch/service.py", line 547 in Subscribe
File "/home/user/.local/lib/python3.6/site-packages/grpc/_server.py", line 385 in _call_behavior
File "/home/user/.local/lib/python3.6/site-packages/grpc/_server.py", line 482 in _unary_response_in_pool
File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56 in run
File "/usr/lib/python3.6/concurrent/futures/thread.py", line 69 in _worker
File "/usr/lib/python3.6/threading.py", line 864 in run
File "/usr/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/usr/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007fc8453e3700 (most recent call first):
File "/home/user/.local/lib/python3.6/site-packages/grpc/_server.py", line 704 in _serve
File "/usr/lib/python3.6/threading.py", line 864 in run
File "/usr/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/usr/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007fc847e1c700 (most recent call first):
File "/home/user/.local/lib/python3.6/site-packages/pyinotify.py", line 1194 in check_events
File "/home/user/.local/lib/python3.6/site-packages/pyinotify.py", line 1381 in loop
File "/usr/local/lib/python3.6/dist-packages/opensnitch/desktop_parser.py", line 172 in run
File "/usr/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/usr/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007fc84870b700 (most recent call first):
File "/home/user/.local/lib/python3.6/site-packages/pyinotify.py", line 1194 in check_events
File "/home/user/.local/lib/python3.6/site-packages/pyinotify.py", line 1381 in loop
File "/usr/local/lib/python3.6/dist-packages/opensnitch/desktop_parser.py", line 172 in run
File "/usr/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/usr/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007fc8870c8740 (most recent call first):
File "/usr/local/lib/python3.6/dist-packages/opensnitch/database.py", line 202 in _insert
File "/usr/local/lib/python3.6/dist-packages/opensnitch/database.py", line 244 in insert
File "/usr/local/lib/python3.6/dist-packages/opensnitch/service.py", line 377 in _populate_stats
File "/usr/local/lib/python3.6/dist-packages/opensnitch/service.py", line 210 in _on_update_stats
File "/usr/local/bin/opensnitch-ui", line 89 in <module>
sh: segmentation fault (core dumped) opensnitch-ui
Second case, running opensnitch-ui, then start daemon in another terminal
$ opensnitch-ui
Loading translations: /usr/local/lib/python3.6/dist-packages/opensnitch/i18n locale: en_AU
Fatal Python error: Segmentation fault
Thread 0x00007f636a7fc700 (most recent call first):
File "/usr/lib/python3.6/threading.py", line 295 in wait
File "/usr/lib/python3.6/queue.py", line 164 in get
File "/usr/lib/python3.6/concurrent/futures/thread.py", line 67 in _worker
File "/usr/lib/python3.6/threading.py", line 864 in run
File "/usr/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/usr/lib/python3.6/threading.py", line 884 in _bootstrap
Current thread 0x00007f636affd700 (most recent call first):
File "/usr/local/lib/python3.6/dist-packages/opensnitch/database.py", line 250 in update
File "/usr/local/lib/python3.6/dist-packages/opensnitch/nodes.py", line 264 in update
File "/usr/local/lib/python3.6/dist-packages/opensnitch/nodes.py", line 46 in add
File "/usr/local/lib/python3.6/dist-packages/opensnitch/service.py", line 547 in Subscribe
File "/home/user/.local/lib/python3.6/site-packages/grpc/_server.py", line 385 in _call_behavior
File "/home/user/.local/lib/python3.6/site-packages/grpc/_server.py", line 482 in _unary_response_in_pool
File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56 in run
File "/usr/lib/python3.6/concurrent/futures/thread.py", line 69 in _worker
File "/usr/lib/python3.6/threading.py", line 864 in run
File "/usr/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/usr/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f636bfff700 (most recent call first):
File "/home/user/.local/lib/python3.6/site-packages/grpc/_server.py", line 704 in _serve
File "/usr/lib/python3.6/threading.py", line 864 in run
File "/usr/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/usr/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f637aeea700 (most recent call first):
File "/home/user/.local/lib/python3.6/site-packages/pyinotify.py", line 1194 in check_events
File "/home/user/.local/lib/python3.6/site-packages/pyinotify.py", line 1381 in loop
File "/usr/local/lib/python3.6/dist-packages/opensnitch/desktop_parser.py", line 172 in run
File "/usr/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/usr/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f637b7d9700 (most recent call first):
File "/home/user/.local/lib/python3.6/site-packages/pyinotify.py", line 1194 in check_events
File "/home/user/.local/lib/python3.6/site-packages/pyinotify.py", line 1381 in loop
File "/usr/local/lib/python3.6/dist-packages/opensnitch/desktop_parser.py", line 172 in run
File "/usr/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/usr/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f63ba196740 (most recent call first):
File "/usr/local/bin/opensnitch-ui", line 89 in <module>
sh: segmentation fault (core dumped) opensnitch-ui
Thank you @kortschak ! it seems the database updates are what is causing the segfaults.
Latest sources contain several fixes that (should) solve these crashes. Could you try them out? I can create a deb package with latest changes. Otherwise, just for testing purposes (daemon 1.4.x should work fine with gui 1.5.x, but no guarantees):
$ git clone https://github.com/evilsocket/opensnitch
$ cd opensnitch/ui
$ sudo cp -r opensnitch/* /usr/lib/python3/dist-packages/opensnitch/
If it doesn't crash (there's still one change to be commited), I can merge these changes into the 1.4.0 branch.
Still seg-faults at the same place when building from ac3965e3c00f617edc64b00d4619051f8075b23a.
$ opensnitch-ui
Loading translations: /usr/local/lib/python3.6/dist-packages/opensnitch/i18n locale: en_AU
exception loading ipasn db: No module named 'pyasn'
Fatal Python error: Segmentation fault
Thread 0x00007ff1edffb700 (most recent call first):
File "/usr/local/lib/python3.6/dist-packages/opensnitch/database.py", line 261 in _insert
File "/usr/local/lib/python3.6/dist-packages/opensnitch/database.py", line 303 in insert
File "/usr/local/lib/python3.6/dist-packages/opensnitch/nodes.py", line 69 in add_rule
File "/usr/local/lib/python3.6/dist-packages/opensnitch/nodes.py", line 80 in add_rules
File "/usr/local/lib/python3.6/dist-packages/opensnitch/nodes.py", line 58 in add_data
File "/usr/local/lib/python3.6/dist-packages/opensnitch/nodes.py", line 45 in add
File "/usr/local/lib/python3.6/dist-packages/opensnitch/service.py", line 602 in Subscribe
File "/home/user/.local/lib/python3.6/site-packages/grpc/_server.py", line 385 in _call_behavior
File "/home/user/.local/lib/python3.6/site-packages/grpc/_server.py", line 482 in _unary_response_in_pool
File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56 in run
File "/usr/lib/python3.6/concurrent/futures/thread.py", line 69 in _worker
File "/usr/lib/python3.6/threading.py", line 864 in run
File "/usr/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/usr/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007ff1ee7fc700 (most recent call first):
File "/usr/lib/python3.6/threading.py", line 295 in wait
File "/usr/lib/python3.6/queue.py", line 164 in get
File "/usr/lib/python3.6/concurrent/futures/thread.py", line 67 in _worker
File "/usr/lib/python3.6/threading.py", line 864 in run
File "/usr/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/usr/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007ff1ef7fe700 (most recent call first):
File "/home/user/.local/lib/python3.6/site-packages/grpc/_server.py", line 704 in _serve
File "/usr/lib/python3.6/threading.py", line 864 in run
File "/usr/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/usr/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007ff1fe92a700 (most recent call first):
File "/home/user/.local/lib/python3.6/site-packages/pyinotify.py", line 1194 in check_events
File "/home/user/.local/lib/python3.6/site-packages/pyinotify.py", line 1381 in loop
File "/usr/local/lib/python3.6/dist-packages/opensnitch/desktop_parser.py", line 172 in run
File "/usr/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/usr/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007ff1ff1d9700 (most recent call first):
File "/home/user/.local/lib/python3.6/site-packages/pyinotify.py", line 1194 in check_events
File "/home/user/.local/lib/python3.6/site-packages/pyinotify.py", line 1381 in loop
File "/usr/local/lib/python3.6/dist-packages/opensnitch/desktop_parser.py", line 172 in run
File "/usr/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/usr/lib/python3.6/threading.py", line 884 in _bootstrap
Current thread 0x00007ff23dbd6740 (most recent call first):
File "/usr/local/lib/python3.6/dist-packages/opensnitch/database.py", line 309 in update
File "/usr/local/lib/python3.6/dist-packages/opensnitch/nodes.py", line 89 in update_rule_time
File "/usr/local/lib/python3.6/dist-packages/opensnitch/service.py", line 441 in _populate_stats
File "/usr/local/lib/python3.6/dist-packages/opensnitch/service.py", line 228 in _on_update_stats
File "/usr/local/bin/opensnitch-ui", line 89 in <module>
sh: segmentation fault (core dumped) opensnitch-ui
Is it possible that there is some fs state that is causing this?
BTW are the plans to not use pip as root in the makefile during install?
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Is it possible that there is some fs state that is causing this?
I have no idea really. As far as I can tell, performing DB operations on the thread that opened the connection solves the crashes.
BTW are the plans to not use pip as root in the makefile during install?
Not for now.
If it doesn't crash (there's still one change to be commited)
This is the change that it's not yet commited to the master branch: service.py.txt Could you test it with latest sources?
I have no idea really. As far as I can tell, performing DB operations on the thread that opened the connection solves the crashes.
I'm wondering if previous version install could be causing the issue.
Not for now.
That's unfortunate.
This is the change that it's not yet commited to the master branch: service.py.txt Could you test it with latest sources?
Which branch is that on, it's easier for me to just check it out.
Still segfaults, same place.
<snip>
Current thread 0x00007f70d9156740 (most recent call first):
File "/usr/local/lib/python3.6/dist-packages/opensnitch/database.py", line 309 in update
File "/usr/local/lib/python3.6/dist-packages/opensnitch/nodes.py", line 264 in update
File "/usr/local/lib/python3.6/dist-packages/opensnitch/nodes.py", line 46 in add
File "/usr/local/lib/python3.6/dist-packages/opensnitch/service.py", line 519 in _on_node_actions
File "/usr/local/bin/opensnitch-ui", line 89 in <module>
sh: segmentation fault (core dumped) opensnitch-ui
ok, the only thing I can suggest is to launch it from a virtualenv, isolated from the rest of the system:
$ sudo apt install virtualenv
$ git clone https://github.com/evilsocket/opensnitch
$ cd opensnitch/
opensnitch/ $ make protocol
opensnitch/ $ cd ui/
opensnitch/ui/ $ virtualenv -p python3 .
opensnitch/ui/ $ source bin/activate
(ui) opensnitch/ui/ $ make
(ui) opensnitch/ui/ $ make install ( ->> or pip3 install . )
(ui) opensnitch/ui/ $ ./bin/opensnitch-ui
^C
(ui) opensnitch/ui/ $ deactivate
It should install all the needed packages under opensnitch/ui/lib/*
I haven't asked you this, but I assume that you haven't changed Database configuration nor daemon configuration, have you? I'm unable to reproduce these crashes, no matter what I try.
After some time away from this, I've just completely hosed all files associated with the previous opensnitch installation and reinstalled from the debs v1.5.1 (with the python-notify2 deb used for that dep). I see the same behaviour.
exception loading ipasn db: No module named 'pyasn'
Install python3-pyasn to display IP's network name.
Fatal Python error: Segmentation fault
Thread 0x00007f2718a5b700 (most recent call first):
File "/usr/lib/python3.6/threading.py", line 295 in wait
File "/usr/lib/python3.6/queue.py", line 164 in get
File "/usr/lib/python3.6/concurrent/futures/thread.py", line 69 in _worker
File "/usr/lib/python3.6/threading.py", line 864 in run
File "/usr/lib/python3.6/threading.py", line 916 in new node connected, listening for client responses... /local
_bootstrap_inner
File "/usr/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f271925c700 (most recent call first):
File "/usr/lib/python3.6/threading.py", line 295 in wait
File "/usr/lib/python3.6/queue.py", line 164 in get
File "/usr/lib/python3.6/concurrent/futures/thread.py", line 67 in _worker
File "/usr/lib/python3.6/threading.py", line 864 in run
File "/usr/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/usr/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f271a25e700 (most recent call first):
File "/home/daniel/.local/lib/python3.6/site-packages/grpc/_server.py", line 704 in _serve
File "/usr/lib/python3.6/threading.py", line 864 in run
File "/usr/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/usr/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f271bab5700 (most recent call first):
File "/home/daniel/.local/lib/python3.6/site-packages/pyinotify.py", line 1194 in check_events
File "/home/daniel/.local/lib/python3.6/site-packages/pyinotify.py", line 1381 in loop
File "/usr/lib/python3/dist-packages/opensnitch/desktop_parser.py", line 178 in run
File "/usr/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/usr/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f271c324700 (most recent call first):
File "/home/daniel/.local/lib/python3.6/site-packages/pyinotify.py", line 1194 in check_events
File "/home/daniel/.local/lib/python3.6/site-packages/pyinotify.py", line 1381 in loop
File "/usr/lib/python3/dist-packages/opensnitch/desktop_parser.py", line 178 in run
File "/usr/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/usr/lib/python3.6/threading.py", line 884 in _bootstrap
Current thread 0x00007f275b9a7740 (most recent call first):
File "/usr/lib/python3/dist-packages/opensnitch/database.py", line 322 in update
File "/usr/lib/python3/dist-packages/opensnitch/nodes.py", line 264 in update
File "/usr/lib/python3/dist-packages/opensnitch/nodes.py", line 46 in add
File "/usr/lib/python3/dist-packages/opensnitch/service.py", line 535 in _on_node_actions
File "/usr/bin/opensnitch-ui", line 103 in <module>
closing, as I've never managed to reproduce this issue, and at least for now no one else has reported similar issues. If you happen to try the latest version and it keeps crashing I can try to reproduce it again.
Please, check the FAQ and Known Problems pages before creating the bug report: https://github.com/evilsocket/opensnitch/wiki/FAQs https://github.com/evilsocket/opensnitch/wiki/Known-problems
Describe the bug A clear and concise description of what the bug is.
Running
opensnitch-ui
at v1.4.2 results in a segfaultInclude the following information:
To Reproduce Describe in detail as much as you can what happened.
Steps to reproduce the behavior:
opensnitch-ui
Post error logs: If it's a crash of the GUI:
Relevant lines of daemon log:
If the daemon doesn't start:
/var/log/opensnitchd.log
# /usr/bin/opensnitchd -rules-path /etc/opensnitchd/rules
) and post the errors logged to the terminal.If the deb or rpm packages fail to install:
$ sudo dpkg -i opensnitch*
/$ sudo yum install opensnitch*
), and post the errors logged to stdout.Expected behavior (optional) A clear and concise description of what you expected to happen.
Screenshots If applicable, add screenshots to help explain your problem. It may help to understand the issue much better.
Additional context Add any other context about the problem here.