Closed Techtonictools closed 3 years ago
uh, interesting @Techtonictools . I've reproduced this problem, but curiously it has worked for the first requests to startpage.com but after a while the second rule (001-vm-deny-all) has started taking precedence...
Is this a supported scenario with VM software (vmware or virtualbox)?
Yes, it's just another process opening connections.
I'll analyze it.
uh, interesting @Techtonictools . I've reproduced this problem, but curiously it has worked for the first requests to startpage.com but after a while the second rule (001-vm-deny-all) has started taking precedence...
ok, definetely there's a problem. I've reapplied the rule (by editing it and clicking on Apply) and it has started working as expected. But after that, I've deleted the allowing rule (000-allow..), recreate it from scratch and it's not allowing the domain.
uh, interesting @Techtonictools . I've reproduced this problem, but curiously it has worked for the first requests to startpage.com but after a while the second rule (001-vm-deny-all) has started taking precedence...
ok, definetely there's problem. I've reapplied the rule (by editing it and clicking on Apply) and it has started working as expected. But after that, I've deleted the allowing rule (000-allow..), recreate it from scratch and it's not allowing the domain.
I was seeing some weirdness too where a rule would get deleted. I added vmwares rule and a firefox rule was deleted and a new one created automatically. It took a big of managing to get all 6 rules for firefox, pandora and vmware.
Ok, here's one bug (note to myself):
I've got a fix for this bug. We weren't resetting correctly rule data when opening the rules editor dialog, so if you had edited a rule before, when adding a new one it was detecting it as a rule change, rather than adding a new rule, thus causing to delete the "old rule".
If you want to test it, these 2 lines fixes the issue:
(add them just after def _reset_state(self):
, in /usr/lib/python3/dist-packages/opensnitch/dialogs/ruleseditor.py
, line 170)
self._old_rule_name = None
self.rule = None
I added that change and re-launched the UI. Then deleted the vmware rules and re-created them, but the rule priority issue remains and the request to startpage.com gets denied by rule 001-vmware deny all.
:-1: for me, I'll try to reproduce it again
one more change @Techtonictools ?
Change this line from "" to "list": https://github.com/evilsocket/opensnitch/blob/95a30c4959963377c9e3324878f89a37753060e1/ui/opensnitch/dialogs/ruleseditor.py#L556
(Yours will be empty). And after adding it, just open each vmware related rule and click on apply, that would be enough to reapply the rules, and see if it works. It's working for me now.
one more change @Techtonictools ?
Change this line from "" to "list":
(Yours will be empty). And after adding it, just open each vmware related rule and click on apply, that would be enough to reapply the rules, and see if it works. It's working for me now.
The operator_operand field in the rules view is now 'list' for rule 000-vmware allow after making that change. I re-tried the repro scenario and found some interesting results. The VM can load sites now like www.weather.gov.
The other interesting result is something I haven't mentioned before and noticed it now and that is the host/debian machine can't get startpage.com loaded because it gets blocked. So I looked for startpage.com in the exception list and it is there. I don't recall that problem earlier but I've noticed some sites in the exception list get blocked.
Using Pithos, there are a bunch of server names to add to the exception list and after adding them all, some sites get denied. It was t1-1.p-cdn.us
that was in the exception list but denied. Pithos will use another server so it isn't easily apparent unless I"m looking in the event view tab.
I initially thought that having a comment at the end of the line in the exception file was causing it 127.0.0.1 t1-1.p-cdn.us \#pandora
and I removed the comment but that had no affect. However, I have several comments throughout the file so I'm not sure if there is a parsing error. Is putting the comment at the end of the line like that OK? If it is, I need to inspect the file more closely to look for errors.
the host/debian machine can't get startpage.com loaded because it gets blocked. So I looked for startpage.com in the exception list and it is there. I don't recall that problem earlier but I've noticed some sites in the exception list get blocked.
Let's see if we can find out what's going on here! You can change the log level to DEBUG (Preferences->Nodes) and monitor the log file:
# tail -f /var/log/opensnitchd.log | grep -A 1 "list match"
[2021-06-02 11:14:18] DBG domain list match: mozilla.cloudflare-dns.com, /etc/opensnitchd/allowlists/allowlist.txt
[2021-06-02 11:14:18] DBG ✔ /usr/lib/firefox-esr/firefox-esr -> mozilla.cloudflare-dns.com:443 (000-allow-domains)
[2021-06-02 11:01:45] DBG domain list match: www.gravatar.com, /etc/opensnitchd/blocklists/domains/tracking-aggressive-extended.txt
[2021-06-02 11:01:45] DBG ✘ /usr/lib/chromium/chromium -> www.gravatar.com:53 (000-block-domains)
Be sure that the exception rule has the checkbox [x] Priority rule
marked, etc.
Is putting the comment at the end of the line like that OK?
no, that's not supported, the format should be: 127.0.0.1 yyy.domain.com
You can write comments above or below the lines, but not after them. Basically, lines that not start with 127.0.0.1
or 0.0.0.0
are ignored. Also the domains "local", "localhost", "localhost.localdomain" and "broadcasthost" are ignored.
I spotted (and fixed) a bug yesterday on how temporary rules are managed. In order to avoid weird behaviours, don't change the Duration of a rule if it's temporary. i.e.: don't change from 5m
to until restart
/always
. In these cases the change doesn't invalidate the old timer, so the rule gets deleted even if the Duration changed.
For me, the denied requests problem where the request is set to be allowed but is denied repros easily by using the attached rules. After exporting and cleaning the rules on my laptop I realized that I had probably mozilla and the vm set to allow so I could use the machine so the rules need to be edited as well as changing the exception list directory.
As you know, it is a lot of work to create an exception list so below are pandoras sites. t1-1.p-cdn.us
was getting denied and it is in the list. Other sites in the exception list such as startpage.com were being allowed in the host one day while not being allowed when from the vm routing binary. Then the next day startpage.com would get denied in the host and startpage.com was still in there unchanged.
127.0.0.1 www.pandora.com
127.0.0.1 tuner.pandora.com
127.0.0.1 t1-1.p-cdn.us
127.0.0.1 t1-2.p-cdn.us
127.0.0.1 t1-3.p-cdn.us
127.0.0.1 t1-4.p-cdn.us
127.0.0.1 t1-5.p-cdn.us
127.0.0.1 mediaserver-cont-dc6-2-v4v6.pandora.com
127.0.0.1 mediaserver-cont-sv5-1-v4v6.pandora.com
127.0.0.1 mediaserver-cont-sv5-2-v4v6.pandora.com
127.0.0.1 mediaserver-cont-dc6-1-v4v6.pandora.com
127.0.0.1 mediaserver-cont-usc-mp1-1-v4v6.pandora.com
127.0.0.1 mediaserver-cont-usc-mp1-2-v4v6.pandora.com
127.0.0.1 cont.p-cdn.us
127.0.0.1 cont-1.p-cdn.us
127.0.0.1 cont-1.p-cdn.us
127.0.0.1 cont-2.p-cdn.us
127.0.0.1 cont-4.p-cdn.us
127.0.0.1 cont-5.p-cdn.us
127.0.0.1 audio-usc-mp1-t1-2-v4v6.pandora.com
127.0.0.1 audio-sv5-t1-2-v4v6.pandora.com
127.0.0.1 audio-dc6-t1-1-v4v6.pandora.com
127.0.0.1 audio-dc6-t1-2-v4v6.pandora.com
127.0.0.1 audio-sv5-t1-1-v4v6.pandora.com
127.0.0.1 audio-sv5-t1-2-v4v6.pandora.com
127.0.0.1 audio-usc-mp1-t1-1-v4v6.pandora.com
The HOSTS formatted exception list in the post above, that is for the Pithos app on the apt store in debian. The repro with that may take a while streaming. If you're not into pandora then just surfing, adding sites and then using them would be the alternative repro scenario.
The repro steps with Pithos is just to run it and when you see a song in the UI queue skips, that is when to look at the opensnitch event viewer for a request denial. If the server is in the exception list already, that is the repro. But it seems to happen with all apps I tried, VMWare, Pithos and Firefox.
To clarify what I meant by a song skip, that is when a song that is shown to be played next in the UI doesn't play. Instead, it is dropped and another will load after that and start playing because a request to the server was denied. An important note is that behavior also occurs when the server isn't in the exception list yet.
Thank you very much for the lists @Techtonictools !
I've realized that you're using the same directory of blocking lists in several rules, to apply it to different apps. It makes a lot of sense, but I didn't even think in that use case... I'll try to reproduce again with your rules.
Edit: ~but I didn't even think in that use case..~
sorry, lists work per-rule. A rule can have multiple lists (distributed in several files in the same directory), and only non-repeated domains will be loaded for that rule. So the same dir of lists should work just fine with several rules. The only downside is that the lists will be loaded in memory several times.
sorry @Techtonictools , I've tried to reproduce it with your rules and your list of domains, but it works fine for me using firefox.
I'd have liked to test it with Pithos, but Pandora is not available in Europe. Anyways, I think I can mimik the use case with spotify or rhythmbox
sorry @Techtonictools , I've tried to reproduce it with your rules and your list of domains, but it works fine for me using firefox.
I'd have liked to test it with Pithos, but Pandora is not available in Europe. Anyways, I think I can mimik the use case with spotify or rhythmbox
Where in the source code is the logic for determining if a request is going to be granted or denied?
First of all, rules are checked in alphabetical order.
There're 2 main points in he code to decide if a rule matches or not:
It receives a connection, and iterates over all the rules to see if any matches.
If there's a match, and the action of the rule is Deny or is a priority rule, it stop iterating and returns the rule that matched, applying the action of that rule.
I could compile a daemon with more log traces to debug all the rules operations (listing order, what rule is checked and what no, if the domain is found in the list of domains or not, etc). I can upload it here.
Looking at the repro behavior now the 001-firefox deny all
rule is picked first every time. The 000-firefox allow
rule isn't being used and it is enabled. I tested the theory of that priority rule not being used by flipping the favored rule to allow (001-firefox deny all
was set to Allow
) and the priority rule 000-firefox allow
was then set to Deny
. I loaded some web pages and the Event log does not show any entries for 000-firefox allow
in either scenario (block or allow) and in the theory test all pages loaded even pages from domains not listed in the HOSTS formatted file (because the priority rule isn't picked up?).
For the rule matching code logic I don't know go
but it looks like there is a list of rules iterated over to pick a rule with the same operator_operand and if that check passes and if the given rule has a Deny
action it is a match. And if it isn't set to Deny
and it is a Precedence
rule it is also a match. I didn't see any other checks besides the action, operator_operand and priority.
I'm curious how the request (a binary from firefox wanting to reach out on the network) matches the rule when there aren't checks to find 000-firefox allow
. Would there be additional logic to pick the firefox rules like by binary path or any other properties?
I think the logging of the rule list like you suggest would be helpful (printing the sorted rules before iterating for a match) as well as the other logging suggestions.
I'm curious how the request (a binary from firefox wanting to reach out on the network) matches the rule when there aren't checks to find 000-firefox allow. Would there be additional logic to pick the firefox rules like by binary path or any other properties?
nope, if a connection doesn't match any rules it should ask to allow or deny it.
Here you have, the daemon compiled from latest sources with all the fixes. The extra logs are in DEBUG level, so set LogLevel to 0 or configure it from the GUI. opensnitchd-debug-429.gz
It'll print all the rules that are evaluated + the operands tested:
[2021-06-08 23:49:09] DBG new connection udp => 31167:172.17.0.3 -> 111.111.111.111:443 uid: 1000
(...)
[2021-06-08 23:47:34] DBG 8 - 001-deny-spotify
[2021-06-08 23:47:34] DBG >> op, operand: process.path, data: /usr/share/spotify/spotify
[2021-06-08 23:47:34] DBG ¡¡ MATCH !!
I performed the repro with the instrumented binary and attached the log from during the repro. In the repro spew, Lithos/Pandora requested tuner.pandora.com
and that host is in the exception list pointed by the 000-python37 allow
rule and I think the request should have been granted.
opensnitchdlog429.log
Thank you!
The domain is checked against the list of domains under /home/theUser/opensnitch
, which is something good, at least the rule is being evaluated, and the order they're being checked looks correct.
[2021-06-09 00:40:04] DBG 2 - 000-python37 allow
[2021-06-09 00:40:04] DBG >> op, operand: list, data: [{"type": "simple", "operand": "process.path", "data": "/usr/bin/python3.7", "sensitive": false
}, {"type": "lists", "operand": "lists.domains", "data": "/home/theUser/opensnitch", "sensitive": false}]
[2021-06-09 00:40:04] DBG >> op, operand: process.path, data: /usr/bin/python3.7
[2021-06-09 00:40:04] DBG >> op, operand: lists.domains, data: /home/theUser/opensnitch
[2021-06-09 00:40:04] DBG domainsListCmp() checking host:%!(EXTRA string=tuner.pandora.com)
But it doesn't seem to find it in the list of that directory. If it'd have been matched, there should have appeared a log like this one:
[2021-06-09 10:08:08] DBG domain list match: adeventtracker.spotify.com, /etc/opensnitchd/blocklists/domains/xxx/adaway-hosts.txt
Edit and save the file with the domains and see if the list is reloaded after save the file, you should see logs similar to these ones:
[2021-06-09 09:27:36] DBG list changed: 2021-06-09 01:24:36.614288505 +0200 CEST, 2021-06-09 11:27:32.886002357 +0200 CEST, /etc/opensnitchd/allowlists/spotify/spotify.txt
[2021-06-09 09:27:36] INF clearing domains lists: 7 - /etc/opensnitchd/allowlists/spotify
[2021-06-09 09:27:36] DBG Loading domains list: /etc/opensnitchd/allowlists/spotify/spotify.txt
[2021-06-09 09:27:36] DBG domains list size: 245
[2021-06-09 09:27:36] INF 7 domains loaded, /etc/opensnitchd/allowlists/spotify/spotify.txt
[2021-06-09 09:27:36] INF 1 lists loaded, 7 domains, 1 duplicated
Verify that the file is parsed and loaded and that the number of domains is not 0. You can also add a file with just 1 domain (tuner.pandora.com), verify that the domains loaded is 1, and see if it matches.
How do I verify the file is parsed and loaded and how do I verify that the domains loaded is 1? The file was re-saved and there was no change. There are 28 lines in the exception file and there is nothing in the log or settings that I found that shows the exception file is loaded.
When the daemon starts, there should be logs like these ones:
[2021-06-09 23:32:01] IMP Starting opensnitch-daemon v1.4.0rc2
[2021-06-09 23:32:01] INF Loading rules from /etc/opensnitchd/rules ...
[2021-06-09 23:32:01] INF loading domains lists: lists, lists.domains, /etc/opensnitchd/allowlists/personal
[2021-06-09 23:32:01] INF monitor lists started: /etc/opensnitchd/allowlists/personal, 1
If you don't see "monitor lists started..." then something is not working as it should.
In order to verify that the lists of domains are being loaded and reloaded, try the following:
create the directory: $ mkdir /tmp/x
write a file with a domain inside that directory: $ echo "127.0.0.1 tuner.pandora.com" > /tmp/x/test.txt
create a simple rule, [x] Enable
, Action: allow
, Duration: until reboot
, [x] to this list of domains: /tmp/x/
just after creating the rule, there should appear in the log something similar to this:
[2021-06-09 09:27:36] INF 1 domains loaded, /tmp/x/test.txt
[2021-06-09 09:27:36] INF 1 lists loaded, 1 domains, 0 duplicated
try adding a new domain: $ echo "127.0.0.1 tuner2.pandora.com" >> /tmp/x/test.txt
again, there should appear new logs:
[2021-06-09 09:27:36] INF 1 domains loaded, /tmp/x/test.txt
[2021-06-09 09:27:36] INF 1 lists loaded, 2 domains, 0 duplicated
If this works, edit your rule 000-python37 allow
, click on Save and see if the reloading traces appear in the log.
I don't have that loading rules nor loading domains spew. I did a systemctl stop opensnitch
, then cleared the log and then a start and this is the log:
[2m[2021-06-10 00:08:42][0m [97m[104m IMP [0m Start writing logs to /var/log/opensnitchd.log
[2m[2021-06-10 00:08:42][0m [2m[30m[100m DBG [0m UI service poller started for socket /tmp/osui.sock
[2m[2021-06-10 00:08:42][0m [97m[42m INF [0m Process monitor method /proc
[2m[2021-06-10 00:08:42][0m [97m[42m INF [0m Running on netfilter queue #0 ...
[2m[2021-06-10 00:08:43][0m [97m[42m INF [0m Connected to the UI service on /tmp/osui.sock
[2m[2021-06-10 00:08:43][0m [97m[42m INF [0m Start receiving notifications
That's really strange. You are using daemon v1.4.0rc2, aren't you? Try creating the dummy rule with the directory and 1 list + 1 domain, and set the log level to DEBUG (although the logs should appear with INFO)
I'm using the binary you attached to this log the other day and DEBUG is on (the second line in the log reflects DBG). I'm not sure what you mean with the dummy rule. I don't understand the directions. What directory and what is 1 list + 1 domain?
Sorry, I mean, in order to verify that the lists are working, create a simple rule with these options:
name: test
[x] Enable
Action: Allow
Duration: Always
[x] To this list of domains: /tmp/x/
create the directory /tmp/x/
write a file (test.txt) to that directory with this content: 127.0.0.1 tuner.pandora.com
After saving the rule, you should see in the logs these messages:
[2021-06-09 23:32:01] INF loading domains lists: lists, lists.domains, /tmp/x
[2021-06-09 23:32:01] INF monitor lists started: /tmp/x, 1
[2021-06-09 09:27:36] INF 1 domains loaded, /tmp/x/test.txt
[2021-06-09 09:27:36] INF 1 lists loaded, 1 domains, 0 duplicated
(Setting the log level to INFO will ease to see only those messages).
If you don't see those messages, then for some reason the lists of domains are not working.
If you see those messages, open the file /tmp/x/test.txt
and add another domain, and verify that the messages appear again, indicating that the rule has been loaded and that the loaded domains are 2.
I revisited this and put my old exception list file back in the list directory ( that was previously not working), rebooted for the service to start correctly. Now it is fine now and pandora is working, including tuner.pandora.com which is in both exception list files.
thank you very much for the feedback @Techtonictools !! let me know if you find anything else.
I'll publish a new release soon with all the fixes, knowing now that it's working as expected.
The problem was reproducing again. I tried to attach gdb to opensnitch and set a breakpoint on domainsListCmp so that I can see why the domain in the list isn't found, but the function wasn't found. Since golang doesn't seem to be fully supported in gdb I'm going to give delve another try after I upgrade the go version in debian so I can build it.
I wanted to step through the daemon during a repro and see why domainsListCmp doesn't find the domain I want to allow, but doing so is not straightforward and requires a bigger time investment than I can spend. It isn't as simple as attaching a debugger and walking through it for me. For anyone that has this scenario working (block all requests except for domains in a list), what distro are you using?
In my case is Debian Sid, and I've also tested it on Kubuntu 20.04 and Debian Buster (livecd iso).
I'd try to reproduce it with just 2 rules.
hey @Techtonictools , there're new packages with fixes in the lists of domains, could you install them and see if these problems are solved?
I gave the new release a try. The upgrade process was seamless. The unexpected block problem is still happening though. In one case, api0.weather.com was blocked when requested by firefox on the host.
The other fail case was in a VM (virtual machine manager running Windows), wunderground.com was blocked. That too is in the exception list and it had just loaded on the host in firefox.
The repro scenario is to load that wunderground.com page, enter a location, click on the WUNDERMAP tab and enable radar. Doing that will invoke the api*.weather.com requests.
Attached is the log file, the rules and pictures of the specific exception and the correlating denial. Let me know if you need more information.
127.0.0.1 mapbox.com
127.0.0.1 api3.weather.com
127.0.0.1 api1.weather.com
127.0.0.1 api2.weather.com
127.0.0.1 api0.weather.com
127.0.0.1 api.weather.com
127.0.0.1 www.wunderground.com
127.0.0.1 wunderground.com
127.0.0.1 api.mapbox.com
127.0.0.1 events.mapbox.com
As an update, I reproduced this problem where domains are blocked that are in the exception list on a new machine. I put debian on my main PC, replacing windows, and imported the opensnitch rules after installing 1.4.0.rc4-1. api[x].weather.com is getting blocked following the most recent repro scenario listed above with weather underground.
I'm not sure if the following scenario would impact setting up the repro after having a different configuration for anyone else, but one thing noteworthy doing the repro on a new machine was that I initially forgot to edit the file path of the exception list in the .json rule files in /etc/opensnitchd/rules after copying them from my laptop. Once I found that issue, I did a string replace to have the proper path in the json rule and the UI was still reflecting the old path. I stopped and restarted, closed and reopened the UI and still the old path was there in the rules in the UI but the proper path was in the json files. After a few more stop/start and close/open the correct path was present in the UI.
As an aside, previously (months ago?) Pandora (pithos) was getting some hosts blocked. But for the past 3 weeks I've been streaming pithos all day and it hasn't blocked any sites in the exception list for that app. However, other sites in the list do get blocked when coming from other apps. Let me know if there is anything I can do.
I root caused one issue that is causing the unexpected blocks and found one whitespace char was at the end of the string and before the newline with some of the domains. That makes sense because when looking at the opensnitch logs and the code, the domain being passed to domainsListCmp()
wasn't found in the exception list because the requested domain did not have a whitespace char after it. The other reason it makes sense is because when I discovered the whitespace before the newline and removed that char and retried, the domain would no longer get blocked.
I'm not sure if that was the cause of all the blocks yet, but for sure that was the cause for some of them. I'm going to install virt-manager and get VM's making requests in parallel with other apps that are also making requests out to the internet to see if the unexpected block problem reproduces in another way.
thank you @Techtonictools ! that's very interesting, I'll try to reproduce it ASAP.
ok, confirmed, if the entries of a blocklist have spaces (or other control characters) the rule doesn't block/allow connections. Here you have an opensnitchd that strips out "\n\r\t " from the hosts when loading lists of domains: opensnitchd-trim-lists.zip
That resolved it, thanks.
I'm looking to use opensnitch to block all requests from vmware and only allow requests in an exception list.
After creating one rule to block the vmware nat program and then another rule to hook the exception list for the vmware nat binary, I'm finding all requests are getting denied. I'm hoping to get requests to domains in the exception list to succeed and all others die. Is this a supported scenario with VM software (vmware or virtualbox)?
Post error logs: The opensnitch events tab shows an entry of the request to the host I'm trying to access (startpage.com) from the VM and the request is blocked because of rule 001-vmware deny all
Versions :
Additional context The same exception list is used in opensnitch rules for multiple binaries, firefox and pandora (Pithos/python3.7), to accomplish the same task of only allowing hosts I want and blocking all others. Those two scenarios are working but I'm not able to get vmware nat stuff to work using the same approach.