search

pi-hole / pi-hole

A black hole for Internet advertisements
https://pi-hole.net
Other
48.57k stars 2.68k forks source link

Black list options #672

Closed emmtte closed 8 years ago

emmtte commented 8 years ago

Hello,

I need to remove every domains like exemple :

sort.test.com
sort1.test.com
sort2.test.com
...

Is that possible to use the syntax

*.test.com

I try but not working

brantje commented 8 years ago

You can use sed to find and replace the matching domains. Untested: sed "s/*.test.com/" "" To use sed you have to login to the pi-hole via ssh.

stonedbovines commented 8 years ago

sed -i '/\.test\.com/d' blacklist.list

each '.' needs to be escaped, or it literally stands to mean any single character

PromoFaux commented 8 years ago

@ManuCart To clarify what you're asking..

Do you wish to be able to blacklist *.test.com without having to type all the possible permutations to do so?

If so, currently wildcard white/blacklisting isn't officially supported at the moment.

HOWEVER! There is possible a solution:

Create the file /etc/dnsmasq.d/02-wildcards.conf, and add in your *.test.com etc there in this format: address=/.test.com/192.168.0.1 (where 192.168.0.1) is your Pi-hole's IP.

emmtte commented 8 years ago

@PromoFaux Thanks for your answer. This what I need. I'll try it next time

TylonHH commented 8 years ago

@manucart APPROVED That's awesome. Now I don't have to enter 1000(!) subdomains. But I've questions: How do I add an other main domain .test.com and .blub.com? Do I add the same line with the other domain?

Will this blocked subdomains shown in the queries log on the web interface? On the main page I can see the subdomains in the table stats but not in the query log. Bug?

PromoFaux commented 8 years ago

Sorry @TylonHH, I am not sure why I didn't think of this earlier when I saw you asking about it...

If you want to add another, just add a second (or third, or fourth) line to that file with the domains you wish to blacklist, that should do the trick!

currently wildcard white/blacklisting isn't officially supported at the moment.

The gist of this is basically the web stats will probably still show the queries as not Pi-holed. as the PHP does not take any additional files into consideration. In order to determine whether or not it is blocked, we look for lines that contain /etc/pihole/gravity.list.

TylonHH commented 8 years ago

Oh damn, I guess https://github.com/pi-hole/pi-hole/issues/700 it's not relevant because it's not official...

emmtte commented 8 years ago

Thanks it's works. But now I need to insert some in white list

ex :: blacklist *.test.com execpt white list blub.test.com

How to do this?

TylonHH commented 8 years ago

Will those blacklisted domains ever appear in stats as blocked?

emmtte commented 8 years ago

Yes they are in stats it's work thanks. But I need to add one not blocked like blub.test.com How to do this because with the stat tools I have an error

TylonHH commented 8 years ago

I ment @PromoFaux But does your blocked domains appear in the query log as pi-holed?

emmtte commented 8 years ago

Before reboot pi yes but after not

stonedbovines commented 8 years ago

@ManuCart in /etc/dnsmasq.d/02-wildcards.conf

address=/.test.com/192.168.0.1 address=/blub.test.com/99.99.99.99

with 99.99.99.99 being the real IP of blub.test.com

PromoFaux commented 8 years ago

Will those blacklisted domains ever appear in stats as blocked? Normal blacklist ones, yes..

The Wildcard ones... never say never! We'll just need to add some additional parsing logic to the web page.

@ManuCart Just played with this and it is indeed possible.. For example, let's say i've blacklisted .xyz with the following in /etc/dnsmasq.d/02-wildcards.conf:

address=/.xyz/192.168.1.253

That works, and I can no longer connect to anything on the xyz TLD. However, I know that I want to be able to access Alphabet's site (https://abc.xyz). In order to do this, I can either create another file (to keep things seperate (e.g /etc/dnsmasq.d/03-exceptions.conf) or add my exceptions to 02-wildcards.conf like so:

server=/abc.xyz/8.8.8.8 server=/abc.xyz/8.8.4.4 (where 8.8.8.8 and 8.8.4.4 are your chosen upstream DNS IPs)

So, to tie it all together, the entire contents of my additional config file (the name really doesn't matter, but they are read into dnsmasq in alphanumerical order):

pi@raspberrypi:~ $ cat /etc/dnsmasq.d/02-cust.conf
#Block Entire XYZ TLD
address=/.xyz/192.168.1.253

#Exceptions below:
#Alphabet:
server=/abc.xyz/8.8.8.8
server=/abc.xyz/8.8.4.4

remember to reload and restart dnsmasq sudo service dnsmasq reload sudo service dnsmasq restart

PromoFaux commented 8 years ago

@stonedbovines That also works, but if the real IP changes, you'll have to update the file! By using server= it allows dnsmasq to do an upstream lookup for the real IP. :)

PromoFaux commented 8 years ago

@TylonHH screenshot in answer to your question:

image

emmtte commented 8 years ago

@PromoFaux Thanks very mutch for your long answer and comprehensive. I need to try this. I will tell you if it's working

stonedbovines commented 8 years ago

@PromoFaux The edit isn't (shouldn't be?) required as any dots before the name get ignored, no? No, it doesn't hurt having the dot there, but I don't believe it is necessary as that initial dot doesn't force it to only look at subdomains. I'm not currently at a system to test that though.

PromoFaux commented 8 years ago

@stonedbovines, Sorry, edited it as I thought you'd accidentally omitted the ..

According to this forum post the . is necessary.

That said... I've just tried it on my machine and it seems happy without the . Strange!

stonedbovines commented 8 years ago

@PromoFaux No worries. I have seen posts making both claims. The last one was on SO and hadn't been vociferously shouted down, so I took that as, at worst, probably not completely wrong.

PromoFaux commented 8 years ago

hadn't been vociferously shouted down

I like that, I think from this point on I'm going to use that as a measure of whether or not a SO answer is any good!

nomoreads commented 7 years ago

@PromoFaux

Seems i don't get this working correctly. Even while i added "address=/.xyz/192.168.1.253" (my Pi IP) to 02-wildcards.conf i can still visit https://blog.heckel.xyz/ while it should block ANY domain with .XYZ

Yes i saved the file, ran sudo service dnsmasq reload and sudo service dnsmasq restart.

PromoFaux commented 7 years ago

Is your Pi's IP 190.168.1.253? That was the one I used in my example......

nomoreads commented 7 years ago

@PromoFaux I have a different IP but i have (of course) set the IP in that line to the IP address of my own Pi.

nomoreads commented 7 years ago

@PromoFaux Well i now see that the whole file doesn't work at all on my end!

For instance i also added address=/qualaroo.com/192.168.1.200 (notice that 200 is my Pi ip) to the 02-wildcards.conf file but it still goes to that website (even while i've add that domain a few days ago to the list).

Soooo.... it seems that wildcards blocking isn't working at all, at least not on my end.

TylonHH commented 7 years ago

Does your PiHole work if you put this domain to the normal blacklist?

nomoreads commented 7 years ago

@TylonHH Yes that works but is not what i want.

Wildcard support is a true musthave to stop pesky websites with tons of subdomains.

address=/.xyz/YOU_PI_IP_HERE should ban literally all .xyz domains (since this domain extension is massivley used by hackers and spammers, same as .TK domains i ban them once and for all).

But doesn't work (currently....)

PromoFaux commented 7 years ago

Hmmm, give me a sec, I'm trying something else RN, but the example I gave above should be working fine.

nomoreads commented 7 years ago

UPDATE: Seems to work now!

Weird guys.... I know it takes a while before Pihole works after adding a domain to blocklist (cache time?) but i was just surfing and stumbled upon a .xyz domain trough Google and it slipped trough, sooo... i've seen a thread here recently that once in a while blocked domains still slip trough, i have to dig into that.

Oh well... clicked a few domains trough Google and found out that www.generalassembly.xyz redirects to www.generalassembly.ly So wildcards does not seem to work properly.... The browser visits www.generalassembly.xyz and actually should immediately be stopped (because it's a .xyz domain) but instead it redirects to www.generalassembly.ly so it seems that when a website owner has set a redirect to another TLD Pihole just ignores the ban and pass trough. Should not happen if you ask me...

nomoreads commented 7 years ago

Another issue seems that it isn't possible to block an IP address). There is an openx script hosted on 213.206.98.70 which is directly been embedded on a site but address=/213.206.98.70/192.168.1.200 isn't working and can't ban the IP trough Pihole Admin.

Script is called on a site with:

<\script type="text/javascript" src="http://213.206.98.70/etc/etc/etc"><\/script>

nomoreads commented 7 years ago

UPDATE: Notice that it seems not work if a domain has HTTPS in it.... There are already a few threads here that reports issues with HTTPS and Pihole issues.

Still able to visit https://blog.heckel.xyz/ while .xyz is blocked.

dschaper commented 7 years ago

Thanks for the reports, but adding multiple unrelated issues on a closed thread probably isn't going to get you the kind of resolution you are hoping for. They need to be issues that we can track and not multiple issues in a single thread. Thanks.

nomoreads commented 7 years ago

@dschaper Oops sorry you're right. I'll create new issues. Thanks.