search

evilsocket / opensnitch

OpenSnitch is a GNU/Linux interactive application firewall inspired by Little Snitch.
GNU General Public License v3.0
10.92k stars 510 forks source link

Slow performance loading some web pages #481

Closed Techtonictools closed 2 years ago

Techtonictools commented 3 years ago

I've been using OpenSnitch starting on a laptop with infrequent use and now I'm using it on my main PC and laptop on a daily basis. It is such a great tool. I've never used or seen anything like it (I don't run mac now) and I'm using it to block Microsoft from trying to update my VM through their back doors and hacks they added so that they can force-feed updates and control my PC. It is also useful to block data mining of my activity on the web and data mining on the PC itself. OpenSnitch additionally enables me to reduce being manipulated through ads on the web and it brings awareness about what apps want to phone home, giving me control of deciding who can phone home and who cannot.

Describe the bug I have OpenSnitch configured so that everything is blocked except for domains in an exception list. Then I maintain that list of approved sites. The issue I've been hitting is it takes a long time to load some of the heavy websites. Sites that seem to have lots of libraries, requests to complete and code to load like weather underground (WU) does, those sites reproduce a performance issue. Some sites load quickly enough and there isn't any perf issue, like when using old.reddit.com, posting, searching, etc.

I had previously used a tool called Fiddler on Windows to block all http requests that respected the system proxy, except for websites in a list, and I'd surf the web using it for protection. When I was using that setup and loaded WU, it did not have a slow down. I know that Fiddler is not a full software proxy like OpenSnitch, but I think that when using Fiddler in that way to load WU, it demonstrated that failing requests doesn't make the page take a long time to load.

The wait time I'm experiencing to load WU with OpenSnitch is about 68 seconds before seeing any rendering at all and before load completes. With OpenSnitch off and clearing the browser cache, the page loaded in 4 seconds. Tor (unrestricted in OpenSnitch) loads the page in 7 seconds. Tor-like speed would be great. Could there be performance improvements made for the web surfing scenario in OpenSnitch?

To Reproduce

  1. Modify the attached rules to include a correct directory path to store the exception list in
  2. Create the exception list from using the domains listed below and stick it in that path in step 1
  3. Launch Firefox and clear the cache if the WU site has been visited before
  4. Enter WU in the address bar, invoke it and begin timing https://www.wunderground.com/forecast/us/wa/seattle
  5. End timing when you see the page beginning to render

Expected behavior (optional)

  1. Some fonts won't load, there will be spinners that never die and the page may look incomplete.
  2. The page will have what looks like an HTML canvas drawing of the 10 day forecast that is completely legible and usable with a mouse-over.
  3. Less than 15 second load time. I don't know what is reasonable since I don't know what work is being done for filtering the requests. 15 seconds is just an arbitrary number.

OS (please complete the following information): The perf issue reproduces on my Macbook Pro 17", but it also reproduces on a custom PC that is much faster. Both have an unused wired NIC and WiFi. Both use the same internet connection/gateway. The config:

OpenSnitch 1.4.0rc4
Debian GNU/Linux 11 (bullseye)
Gnome 3.38.5
Firefox 78.13.0esr 
Tor 10.5.5
USB NIC: Bus 003 Device 003: ID 148f:761a Ralink Technology, Corp. MT7610U ("Archer T2U" 2.4G+5G WLAN Adapter
00:19.0 Ethernet controller: Intel Corporation 82579V Gigabit Network Connection (rev 05)
Intel® Core™ i7-3930K CPU @ 3.20GHz × 12 
24 GB RAM

Additional context Traceroute

anon@dev:~$ traceroute wunderground.com
traceroute to wunderground.com (23.212.36.241), 30 hops max, 60 byte packets
 1  _gateway (192.168.43.1)  2.769 ms  3.063 ms  4.153 ms
 2  202.sub-66-174-26.myvzw.com (66.174.26.202)  74.140 ms  67.462 ms  74.276 ms
 3  * * *
 4  50.sub-69-83-152.myvzw.com (69.83.152.50)  67.260 ms  73.449 ms  73.770 ms
 5  * * *
 6  76.sub-69-83-128.myvzw.com (69.83.128.76)  73.207 ms  76.739 ms  60.019 ms
 7  135.sub-69-83-129.myvzw.com (69.83.129.135)  71.213 ms  71.544 ms  71.528 ms
 8  0.csi1.WJRDUT30-MSE01-BB-SU1.ALTER.NET (140.222.7.46)  76.963 ms  77.048 ms  76.751 ms
 9  * * *
10  * * *
11  0.ae2.GW8.LAX15.ALTER.NET (140.222.3.107)  96.008 ms  90.581 ms 0.ae1.GW8.LAX15.ALTER.NET (140.222.3.105)  90.940 ms
12  152.179.21.130 (152.179.21.130)  91.456 ms  87.283 ms  109.814 ms
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  * * *
26  * * *
27  * * *
28  * * *
29  * * *
30  * * *
anon@dev:~$

Exception list for WU

127.0.0.1 api3.weather.com
127.0.0.1 api2.weather.com
127.0.0.1 api0.weather.com
127.0.0.1 api1.weather.com
127.0.0.1 api.weather.com
127.0.0.1 api.mapbox.com
127.0.0.1 dsx.weather.com 
127.0.0.1 www.wunderground.com
127.0.0.1 wunderground.com

rules.zip

gustavo-iniguez-goya commented 3 years ago

The wait time I'm experiencing to load WU with OpenSnitch is about 68 seconds before seeing any rendering at all and before load completes. With OpenSnitch off and clearing the browser cache, the page loaded in 4 seconds. Tor (unrestricted in OpenSnitch) loads the page in 7 seconds. Tor-like speed would be great. Could there be performance improvements made for the web surfing scenario in OpenSnitch?

Sure. I think in this case the problem will be the blocklists, I've experienced similar issues on some websites, did you try it out without the blocklists rules?

I'll take a look at it.

Techtonictools commented 3 years ago

The wait time I'm experiencing to load WU with OpenSnitch is about 68 seconds before seeing any rendering at all and before load completes. With OpenSnitch off and clearing the browser cache, the page loaded in 4 seconds. Tor (unrestricted in OpenSnitch) loads the page in 7 seconds. Tor-like speed would be great. Could there be performance improvements made for the web surfing scenario in OpenSnitch?

Sure. I think in this case the problem will be the blocklists, I've experienced similar issues on some websites, did you try it out without the blocklists rules?

I'll take a look at it.

Awesome. With OpenSnitch running and unrestricted firefox rules (as was Tor in the 7 sec load result), it took 4 seconds to load WU.

gustavo-iniguez-goya commented 3 years ago

Yes, I've reproduced it, and it's the same problem I already saw. I think the problem is that we're just dropping the requests instead of rejecting them , so in some cases the apps wait until the DNS resolution times out. I didn't find a way of rejecting them with libnetfilter-queue. One way of fixing this issue could be replying a DNS answer poiting to localhost.

Techtonictools commented 3 years ago

That makes sense, I did see a host request name bubble in firefox sit there for a long time. I wouldn't know what to suggest other than close the socket, but that may not be supported for all protocols. I've used ss -K dst xxx to kill requests, but it is limited.

gustavo-iniguez-goya commented 3 years ago

well, mmmh, now that you mention killing requests, I guess that we could kill connections via netlink.conntrack .. interesting. So conman.Parse() could return an action along with the properties of a connection and kill it on demand. I'll write a PoC some day in the near future :+1:

edit: killing connections doesn't make any difference.

gustavo-iniguez-goya commented 3 years ago

hey @Techtonictools , could you test it with this daemon? opensnitchd-reject-#481.gz

Stop the running service, rename the daemon to opensnitchd if you wish, and launch it from a terminal: ./opensnitchd-reject -rules-path /etc/opensnitchd/rules/ Edit the .json that blocks everything with a text editor (not the GUI) and change the Action to "reject".

I think it's working as expected, killing the outgoing requests. WU loads much faster.

Techtonictools commented 3 years ago

Absolutely. On my laptop I edited the default-config.json and changed the DefaultAction field to reject, stopped the service and launched the private binary. The program spewed an error and didn't start. 

{"Server": {"Address": "unix:///tmp/osui.sock", "LogFile": "/var/log/opensnitchd.log"}, "DefaultAction": "reject", "DefaultDuration": "once", "InterceptUnknown": false, "ProcMonitorMethod": "proc", "LogLevel": 0}

anon@debian:~/Downloads$ ./opensnitchd-reject-.481 -rules-path /etc/opensnitchd/rules/
[2021-08-26 16:50:15]  IMP  Starting opensnitch-daemon v1.4.0rc4
[2021-08-26 16:50:15]  INF  Loading rules from /etc/opensnitchd/rules ...
[2021-08-26 16:50:15]  INF  loading domains lists: lists, lists.domains, /home/anon/opensnitch
[2021-08-26 16:50:15]  INF  loading domains lists: lists, lists.domains, /home/anon/opensnitch/
[2021-08-26 16:50:15]  INF  monitor lists started: /home/anon/opensnitch
[2021-08-26 16:50:15]  INF  monitor lists started: /home/anon/opensnitch/
[2021-08-26 16:50:15]  INF  loading domains lists: lists, lists.domains, /home/anon/opensnitch
[2021-08-26 16:50:15]  INF  clearing domains lists: 0 - /home/anon/opensnitch/
[2021-08-26 16:50:15]  INF  clearing domains lists: 0 - /home/anon/opensnitch
[2021-08-26 16:50:15]  INF  monitor lists started: /home/anon/opensnitch
[2021-08-26 16:50:15]  INF  clearing domains lists: 0 - /home/anon/opensnitch
[2021-08-26 16:50:15]  INF  loading domains lists: lists, lists.domains, /home/anon/opensnitch
[2021-08-26 16:50:15]  WAR  Is opensnitchd already running?
[2021-08-26 16:50:15]  !!!  Error while creating queue #0: Error unbinding existing q handler from AF_INET protocol family: operation not permitted
anon@debian:~/Downloads$ sudo systemctl status opensnitch
● opensnitch.service - OpenSnitch is a GNU/Linux application firewall.
   Loaded: loaded (/lib/systemd/system/opensnitch.service; enabled; vendor preset: enabled)
   Active: inactive (dead) since Thu 2021-08-26 09:42:06 MST; 8min ago
     Docs: https://github.com/gustavo-iniguez-goya/opensnitch/wiki
  Process: 3332 ExecStartPre=/bin/mkdir -p /etc/opensnitchd/rules (code=exited, status=0/SUCCESS)
  Process: 3333 ExecStart=/usr/bin/opensnitchd -rules-path /etc/opensnitchd/rules (code=exited, status=0/SUCCESS)
 Main PID: 3333 (code=exited, status=0/SUCCESS)

Aug 26 09:39:22 debian opensnitchd[3333]: [2021-08-26 16:39:22]  INF  loading domains lists: lists, lists.domains, /home/anon/opensnitch
Aug 26 09:39:22 debian opensnitchd[3333]: [2021-08-26 16:39:22]  INF  clearing domains lists: 0 - /home/anon/opensnitch/
Aug 26 09:39:22 debian opensnitchd[3333]: [2021-08-26 16:39:22]  INF  loading domains lists: lists, lists.domains, /home/anon/opensnitch
Aug 26 09:39:22 debian opensnitchd[3333]: [2021-08-26 16:39:22]  INF  monitor lists started: /home/anon/opensnitch
Aug 26 09:39:22 debian opensnitchd[3333]: [2021-08-26 16:39:22]  INF  clearing domains lists: 0 - /home/anon/opensnitch
Aug 26 09:39:22 debian opensnitchd[3333]: [2021-08-26 16:39:22]  INF  monitor lists started: /home/anon/opensnitch
Aug 26 09:39:22 debian opensnitchd[3333]: [2021-08-26 16:39:22]  INF  clearing domains lists: 0 - /home/anon/opensnitch
Aug 26 09:42:06 debian systemd[1]: Stopping OpenSnitch is a GNU/Linux application firewall....
Aug 26 09:42:06 debian systemd[1]: opensnitch.service: Succeeded.
Aug 26 09:42:06 debian systemd[1]: Stopped OpenSnitch is a GNU/Linux application firewall..
anon@debian:~/Downloads$

debian version:

Linux version 4.19.0-17-amd64 (debian-kernel@lists.debian.org) (gcc version 8.3.0 (Debian 8.3.0-6)) #1 SMP Debian 4.19.194-1 (2021-06-10)
gustavo-iniguez-goya commented 3 years ago

oops, I see, you need to launch it as root:

anon@debian:~/Downloads$ sudo su

# service opensnitch stop
# ./opensnitchd-reject-.481 -rules-path /etc/opensnitchd/rules/
Techtonictools commented 3 years ago

When running the private as root, I didn't notice a perf change. It took 38 seconds to load WU on the laptop with the private bin and on my main pc on the same gateway with the release version it took 37 seconds. How long does it take for it to load on your host?

gustavo-iniguez-goya commented 3 years ago

Without killing connections (Action: deny) it takes 1' to just display some text of the page and 1'40" for the loading progress bar to stop (I'm on a really slow connection, 50-500KB/s).

Killing connections (Action: reject) the loading progress bar stops in about 10-22s, almost at the same time that any text is displayed. Although it seems to load faster, I can see that it keeps loading resources in background, and it takes 40s in total +-.

I think I'll push these changes to a new branch in order to save it, and maybe add the "reject" Action for the v1.5.0.

In any case, we could also reply to DNS queries pointing blocked domains to localhost. If someone wants to work on this and make a PoC it'd be more than welcome.

gustavo-iniguez-goya commented 3 years ago

for anyone interested and future reference: https://github.com/evilsocket/opensnitch/commit/1b8ec1644149191c0b561ca79d0898c01003ec70

feedback welcome by the way.

Techtonictools commented 3 years ago

Without killing connections (Action: deny) it takes 1' to just display some text of the page and 1'40" for the loading progress bar to stop (I'm on a really slow connection, 50-500KB/s).

Killing connections (Action: reject) the loading progress bar stops in about 10-22s, almost at the same time that any text is displayed. Although it seems to load faster, I can see that it keeps loading resources in background, and it takes 40s in total +-.

I think I'll push these changes to a new branch in order to save it, and maybe add the "reject" Action for the v1.5.0.

In any case, we could also reply to DNS queries pointing blocked domains to localhost. If someone wants to work on this and make a PoC it'd be more than welcome.

I'd like to look at the code, but I'm not sure I can fix it and for sure can't do it right away. Where in the source is the logic that determines if a request will be allowed or denied?

gustavo-iniguez-goya commented 3 years ago

There're 2 points in the code that you can look at:

  1. outbound connections:

If the domain is in a block/allowlist the response could be crafted here.

  1. DNS responses:

    Intercepted by a rule in the INPUT chain of ip/nftables:

    target     prot opt source               destination         
NFQUEUE    udp  --  anywhere             anywhere             udp spt:domain NFQUEUE num 0 bypass

Modifying the packet here wouldn't have much sense I guess, since we would have to wait for the response.

SetVerdict*() funcs are defined in netfilter/packet.go and they send the verdict through a channel (netfilter/queue.go), optionally

Techtonictools commented 3 years ago

Thanks for laying that out, gustavo. I started to read the code path going backwards from onPacket for this issue and also for issue #482 . All these types, libraries and golang are new to me. I want to read up on all the libraries and learn how this program works so I can understand it.

gustavo-iniguez-goya commented 3 years ago

I'm starting to feel confident with the Action: reject, it seems to work fine (if it improves web pages loading times is another story). Maybe I'll add it for this release:

deny:

$ time curl duckduckgo.com
curl: (6) Could not resolve host: duckduckgo.com

real    0m20,049s
user    0m0,007s
sys     0m0,012s

reject:

$ time curl duckduckgo.com
curl: (6) Could not resolve host: duckduckgo.com

real    0m0,043s
user    0m0,012s
sys     0m0,006s

I don't know if you mentioned it already, but could you reproduce this problem using "proc" monitor method? I want to discard that it's not due to the ebpf module.

Techtonictools commented 3 years ago

That looks like a substantial improvement. I set the Process monitor method in the UI to Proc just now and re-ran the scenario. I didn't see a performance change. WU took longer than a minute time to begin to load.

On Mon, 2021-09-06 at 06:22 -0700, Gustavo Iñiguez Goia wrote:

I'm starting to feel confident with the Action: reject, it seems to work fine (if it improves web pages loading times is another story). Maybe I'll add it for this release: deny: $ time curl duckduckgo.com curl: (6) Could not resolve host: duckduckgo.com

real 0m20,049s user 0m0,007s sys 0m0,012s reject: $ time curl duckduckgo.com curl: (6) Could not resolve host: duckduckgo.com

real 0m0,043s user 0m0,012s sys 0m0,006s I don't know if you mentioned it already, but could you reproduce this problem using "proc" monitor method? I want to discard that it's not due to the ebpf module. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

gustavo-iniguez-goya commented 2 years ago

hey! I think this issue is resolved in latest RC2: https://github.com/evilsocket/opensnitch/releases/tag/v1.5.0-rc.2

So I'm closing this now. If the issue remains, please reopen it and we'll take a look.