search

jpgpi250 / piholemanual

files referred to in my pihole installation manual
114 stars 14 forks source link

cdnjs.cloudflare.com breaks when using DOH ipv4 list #3

Closed DrGonzoNL closed 4 years ago

DrGonzoNL commented 4 years ago

I use your DoH IP list in my OPNSense firewall, it will block 104.16.132.229 on port 443. This breaks cdnjs.cloudflare.com, result is that websites using this will load very very slow. An example is www.rtvnoord.nl An NSLookup of cdnjs.cloudflare.com gives: Non-authoritative answer: Name: cdnjs.cloudflare.com Addresses: 2606:4700::6810:85e5 2606:4700::6810:84e5 104.16.132.229 104.16.133.229

Is it possible to remove these IP's? If not, is there documentation or some proof that these IP's provide DoH? If yes, then I have to consider if whitelisting is an option.

jpgpi250 commented 4 years ago

The following entries have been removed from the lists 104.16.132.229 104.16.133.229 2606:4700::6810:84e5 2606:4700::6810:85e5

The entry dns.cloudflare.com (resolves to the above addresses) can be found in the following lists: https://raw.githubusercontent.com/Sekhan/TheGreatWall/master/TheGreatWall.txt https://raw.githubusercontent.com/oneoffdallas/dohservers/master/list.txt https://raw.githubusercontent.com/vysecurity/DoH-Servers/master/README.md https://download.dnscrypt.info/dnscrypt-resolvers/json/public-resolvers.json

The domain dns.cloudflare.com has been added to the whitelist entries, the above IP's will never appear in the lists again.

The repository has been updated.

If these IP's are the IP's of DOH servers, the removal will break the protection!

Thanks for reporting this.

prunes-git commented 4 years ago

I also noticed this, I tried querying the addresses using the typical cloud flare url format with no response.

You may be able to intergrade a tool like this in your scripts to test and verify the addresses https://github.com/picatz/doh However the URLs may differ as quad9 uses port 5053 doesn't comply to the standards format of "https://dnsserver.example.net/dns-query{?dns}"

jpgpi250 commented 4 years ago

Your original reply included the question: Is there a way to add ports to the block list? as I noticed quad9 also use 5053?

A long time ago, when I first started using pfsense, I followed the recommendations of an article on the netgate forum that explained, you should configure the firewall to block everything (all ports), than add a firewall rule to allow specific ports only. When using this strategy, there is no need to block additional ports, as, for example port 5053, would never be allowed in the first place.

image

The list of allowed ports I use is very small: image

If I than need to allow a port, not in the list, I create a rule that targets this port combined with a specific target IP image Over the years, I have been using pfsense, I hardly ever needed to create such rules, the current count is less than 10

jpgpi250 commented 4 years ago

I don't like removing entries from the IP lists, using a whitelist. Every DOH provider, that hosts both a DOH server and content (or a service), such as for example dns.cloudflare.com and cdnjs.cloudflare.com (same IP) would end up on that whitelist, thus DOH NOT blocked.

I'm currently considering the following:

DrGonzoNL commented 4 years ago

Thanks for your quick actions and replies!

I think your method of the two seperate lists would be a good workaround. For me it will be about 80% percent of my devices that browse. So it is still an opening, but at least it is only very specific. I use it in my home network which isn't very big. I am just very privacy and security minded.

For me the ultimate solution would be if this would find it's way in the upstream of Suricata.

jpgpi250 commented 4 years ago

so I assume OPNsense also has a package suricata, I was wondering if it did. Another + in the migrate from pfsense to OPNsense (I want IPv6 NAT, pfsense can't do that)

regarding your ultimate solution, asked and answered here.

I've started the necessary coding for implementing the above described method (two separate lists). The github repository now has two extra files, DOH443ipv4.txt and DOH443ipv6.txt. These files currently contain the addresses for dns.cloudflare.com (= also cdnjs.cloudflare.com)

Could you please implement the allow rules (port 443 - specific devices) and let me know when this is done, I will than update the scripts to include the IP addresses again into the original lists (undo the exclusion).

DrGonzoNL commented 4 years ago

OPNSense does indeed have a Suricata package. For me it is my first time setting up a firewall so I don't have experience with PFSense. The frequent updates and reported stability made me choose for OPNSense. So far I really like it and has been solid as a rock.

For me everything is in place, so you can add them back to the original list.

jpgpi250 commented 4 years ago

done. The GitHub repository has been updated. The DOHipvX.txt files now contain the entries you mentioned originally (cdnjs.cloudflare.com and dns.cloudflare.com blocked again). The DOH443ipvX.txt files contain the entries you need to allow for specific clients.

I think this approach is the best possible solution, to avoid unwanted blocking (browser), but still prevent IOT and other devices from using DOH.

I will be writing a document to explain the required setup in detail, this may take a while.

This issue will be closed in 21 days.

DrGonzoNL commented 4 years ago

Thanks for all your effort. Keep up the good work! Your manual really helped me setting up my Pihole in a secure way

prunes-git commented 4 years ago

I edited the ports part out of my post because I realised it wouldn't really be a workable solution, thanks for replying to it anyway, as far as only allowing specific ports, I have to cater for the fact others are on the network and if usability suffers too much then they will just physically bypass the firewall.

I like your solution, I will implement it on my firewall today.

Currently I'm catching all 53 and 853 traffic from my internal networks and redirecting them to my firewalls resolver, blocking DoH should mean that a vast majority of devices will be forced through my filtered DNS without having to change any settings on them.

jpgpi250 commented 4 years ago

I've changed the filenames of the exceptions files, because they didn't make any sense in the documentation.

DOH443ipv4.txt -> DOHexceptionsIPv4.txt DOH443ipv6.txt -> DOHexceptionsIPv6.txt

Sorry for the inconviniance...

jpgpi250 commented 4 years ago

Would you please review and comment on the document, I have created

Thank you for your time and effort

jpgpi250 commented 4 years ago

closed after 21 days