ragibkl / adblock-dns-server

Adblock DNS Server powered by Bancuh DNS and dnsdist-acme
https://bancuh.com/
MIT License
65 stars 14 forks source link

Google ads #116

Closed trimechee closed 2 years ago

trimechee commented 2 years ago

Hello, ouahh Adblock Dns BANCUH is so amazing awesome fantastic exceptional ! thank you very much ! In a French forum of Free FAI internet provider, a user tested several DNS and he says that Bancuh is the best and the powerful to block ads in the smart TV after setting the router to use Bancuh !!

https://forum.universfreebox.com/viewtopic.php?p=732342#732342

I used the DNS of our beloved dns Bancuh and then I used this ad block test detecor website and Banuh gets a score of 98% ad blocked !! this is better thant a lot of other famous DNS ad blocker! It's breathtaking! Just I notice that Bancuh does not block Google's ads according to the results of this test site! I specify that I did the test with after I disabled my ad blocker:

https://d3ward.github.io/toolz/adblock.html

It will be great to be able to block Google ads via Banuch and thank you for integrating a porn filter because porn is disgusting and dinging and destroying society and I hope that you will keep this essentiel beneficial filter by default :) thanks again :) therefore we will not have a need for ad blocker extension and app in sha Allah.

It will be even more awesome to integrate a default blocking of all the frundulous crypto currency mining sites, I do not know if Bancuh already blocks the sites of coiner mining ....

Perhaps these filters can help, I found them in the page of website of this DNS reserved for Malaysia and Singapore called DNSKU:

"What ad blockers do we use?

Our blockclist currently blocks over 1.3 million ad domains and counting. We use a single Curated List Below for Ads and Malware on Top of Staying Behind CloudFlare Own DNS Protection.

https://oisd.nl/

In Addition To The Above, For The Adult Content, We Use Both CloudFlare's Adult Content Blocking DNS Coupled With The Below Curated List:

https://raw.githubusercontent.com/stevenblack/hosts/master/alternates/porn/hosts

WE ALSO USE A WHITELIST TO ENSURE THAT LEGIT SITES ARE PROTED FROM ANY FALSE POSITIVE, WE USE THE FOLLOWING WHITELIST

https://github.com/anudeepnd/whitelist "
ragibkl commented 2 years ago

Hi @trimechee , thanks for reaching out.

I haven't updated the blocklist and whitelist for a while. Another user from France has spent a lot of time testing different blocklist, so let's ask him for opinion.

@Tomatoide , do you remember if you tried any of the above lists? What do you think about this one?

Tomatoide commented 2 years ago

@ragibkl I think all these lists are already included

Tomatoide commented 2 years ago
https://ads.google.com
https://googleadservices.com
https://freshmarketer.com/
https://stats.wp.com
https://data.mistat.intl.xiaomi.com

these were the hosts not blocked in that test, maybe we could add them to blacklist, also check maybe they are in any of the old whitelist files

trimechee commented 2 years ago

Fantastic ! Awesome ! Yes I think that will be a great idea to add these addresses to blacklist to make Bancuh DNS even more powerful !

I tested several dns adbLocker and I was surprised that almost all these famous or unknown DNS do not block Google ads or Amazon ads ! it's strange, maybe google and amazon added new addresses or there are has a mistake in the lists on which these dns are based .... anyway thank you very much for all your efforts Mr Ragibklde and Mr Tomatoide to test and to offer us this amazing DNS that avoid us to use a block :)

Without transition, I found this software that allows to circumvent censorship, is it possible to implement it at Bancuh DNS to circumvent the government censorship? I do not have many computer knowledge, it's just a question or suggestion :)

https://github.com/ValdikSS/GoodbyeDPI

trimechee commented 2 years ago

Hello, I don't know if that's the case, we hope that our beloved Bancuh dns adds these filters because crypto mining currency are dangerous for the cpu of our laptops and consumes a lot of energy, and it's great to add these filters of Malicious URL blocklist , a lot of these filters are integrated in ublock origin and that's the reason why I use ublock origin that's why it will be great if dns integrates them natively so we no longer need an extension in sha Allah :

uBlock filters – Resource abuse ! Description: To foil sites potentially abusing CPU/bandwidth resources without informed ! consent. Any such resource-abuse scripts MUST be opt-in, with complete ! informed consent from the visitor.

(https://raw.githubusercontent.com/uBlockOrigin/uAssets/master/filters/resource-abuse.txt)

https://github.com/uBlockOrigin/uAssets

Online Malicious URL Blocklist

https://curben.gitlab.io/malware-filter/urlhaus-filter-online.txt

https://gitlab.com/curben/urlhaus-filter

NoCoin adblock list

https://raw.githubusercontent.com/hoshsadiq/adblock-nocoin-list/master/nocoin.txt

https://github.com/hoshsadiq/adblock-nocoin-list

We can check if these filters are activated by the DNS by going to this site which tests adblock detection Subscriptions and we hope to see the lists of ublock Online Malicious URL Blocklist and uBlock filters – Resource abuse and NoCoin adblock list that I proposed to appear! Thank you very much :)

https://browserleaks.com/proxy

Tomatoide commented 2 years ago

as for cryptominers they are already blocked in aggregate lists eg OISD

Tomatoide commented 2 years ago

after #118 these still pass

https://ads.google.com
https://googleadservices.com
https://data.mistat.intl.xiaomi.com

they pass in the logs as non blocked, you can access them as urls (no connection error), when tried with adaway enabled with lists that are already in bancuh they get blocked no problem, don't know where the issue is in bancuh @ragibkl

ragibkl commented 2 years ago

after #118 these still pass

https://ads.google.com
https://googleadservices.com
https://data.mistat.intl.xiaomi.com

they pass in the logs as non blocked, you can access them as urls (no connection error), when tried with adaway enabled with lists that are already in bancuh they get blocked no problem, don't know where the issue is in bancuh @ragibkl

Hi @Tomatoide , @trimechee ,

It seems that they are included in this whitelist. It seems that there are good reasons to whitelist them. I found the following in the whitelist and the reasoning:

# Not used to serve ads from (some people require it for their jobs as webdeveloper)
 ads.google.com
 adwords.google.com

# Google shopping
 googleadservices.com
 www.googleadservices.com

# https://github.com/notracking/hosts-blocklists/issues/589
 data.mistat.intl.xiaomi.com

I don't know, maybe it's fine to whitelist them for now, as not much impact on user experience?

Tomatoide commented 2 years ago

ok that makes sense, but if it's up to me I'd probably still block those two google ads domains, but it's your call at the end

ragibkl commented 2 years ago

Fantastic ! Awesome ! Yes I think that will be a great idea to add these addresses to blacklist to make Bancuh DNS even more powerful !

I tested several dns adbLocker and I was surprised that almost all these famous or unknown DNS do not block Google ads or Amazon ads ! it's strange, maybe google and amazon added new addresses or there are has a mistake in the lists on which these dns are based .... anyway thank you very much for all your efforts Mr Ragibklde and Mr Tomatoide to test and to offer us this amazing DNS that avoid us to use a block :)

Without transition, I found this software that allows to circumvent censorship, is it possible to implement it at Bancuh DNS to circumvent the government censorship? I do not have many computer knowledge, it's just a question or suggestion :)

https://github.com/ValdikSS/GoodbyeDPI

Hi @trimechee ,

I read through the project page. I don't think we can include that project into Bancuh DNS. The two projects are doing completely different things. Technically impossible to combine into a DNS server.

If you need the GoodbyeDPI project, I suggest you use that separately.

Tomatoide commented 2 years ago

so i checked that whitelist again and while it seems like a nice list, I don't like that ''Google shopping'' section:

# Google shopping
 googleadservices.com
 www.googleadservices.com
 doubleclick.net
 pagead.l.doubleclick.net

and it kinda explains why i was seeing some google ads before I added an additional local hosts blocker to my devices, can we add these to the misc blacklist but give it a higher priority or enforce it over the whitelist?

ragibkl commented 2 years ago

Hello, I don't know if that's the case, we hope that our beloved Bancuh dns adds these filters because crypto mining currency are dangerous for the cpu of our laptops and consumes a lot of energy, and it's great to add these filters of Malicious URL blocklist , a lot of these filters are integrated in ublock origin and that's the reason why I use ublock origin that's why it will be great if dns integrates them natively so we no longer need an extension in sha Allah :

uBlock filters – Resource abuse ! Description: To foil sites potentially abusing CPU/bandwidth resources without informed ! consent. Any such resource-abuse scripts MUST be opt-in, with complete ! informed consent from the visitor.

(https://raw.githubusercontent.com/uBlockOrigin/uAssets/master/filters/resource-abuse.txt)

https://github.com/uBlockOrigin/uAssets

Online Malicious URL Blocklist

https://curben.gitlab.io/malware-filter/urlhaus-filter-online.txt

https://gitlab.com/curben/urlhaus-filter

NoCoin adblock list

https://raw.githubusercontent.com/hoshsadiq/adblock-nocoin-list/master/nocoin.txt

https://github.com/hoshsadiq/adblock-nocoin-list

We can check if these filters are activated by the DNS by going to this site which tests adblock detection Subscriptions and we hope to see the lists of ublock Online Malicious URL Blocklist and uBlock filters – Resource abuse and NoCoin adblock list that I proposed to appear! Thank you very much :)

https://browserleaks.com/proxy

@trimechee , if you want to suggest more sources for the blocklist or whitelist, let me share a little about our configuration.

Our main config file is here. Feel free to look at it to see what is currently included.

We can only accept blocklist in the format of hosts file like this or domains file like this.

We cannot use this one, because the format is not compatible. I think this can only use with ublock browser plugin. uBlock filters: https://raw.githubusercontent.com/uBlockOrigin/uAssets/master/filters/resource-abuse.txt https://github.com/uBlockOrigin/uAssets

The following that you shared are not compatible format, but the authors also provide the same lists in hosts format: Online Malicious URL Blocklist: https://gitlab.com/curben/urlhaus-filter https://curben.gitlab.io/malware-filter/urlhaus-filter-online.txt hosts file: https://curben.gitlab.io/malware-filter/urlhaus-filter-hosts.txt

NoCoin adblock list https://github.com/hoshsadiq/adblock-nocoin-list https://raw.githubusercontent.com/hoshsadiq/adblock-nocoin-list/master/nocoin.txt https://raw.githubusercontent.com/hoshsadiq/adblock-nocoin-list/master/hosts.txt

@Tomatoide , what do you think about the urlhaus filter and the no coin above? should we include them?

ragibkl commented 2 years ago

so i checked that whitelist again and while it seems like a nice list, I don't like that ''Google shopping'' section:

# Google shopping
 googleadservices.com
 www.googleadservices.com
 doubleclick.net
 pagead.l.doubleclick.net

and it kinda explains why i was seeing some google ads before I added an additional local hosts blocker to my devices, can we add these to the misc blacklist but give it a higher priority or enforce it over the whitelist?

@Tomatoide , we currently don't have a feature for prioritized lists. We can probably use the existing overrides list. I've pushed a commit to try this.

Let me know if things are working or breaking!

Tomatoide commented 2 years ago

It works! thank you. regarding the urlhaus and nocoin lists, you can add them, although I think they (or similar lists) are already included in some shape or form

trimechee commented 2 years ago

Magnificent ! thank you very much for your efforts to make Bancuh even more efficient!

Yes of course in sha Allah, if I find other blocklists I will share them here :)

I use this web browser: ungoogled chromium, windows version made by Marmaduke:

https://chromium.woolyss.com/

and ungoogled chromium has disabled google safe browsing API that blocks malware websites, so it is necessary to have a dns that blocks phishing malware and viruses at the source, that's why I proposed these updated lists.....

I understand that GoodbyeDPI cannot be technically implemented, it was just a suggestion, there is no worries :) I don't have any censorship issues at the moment and I know some free vpn that can circumvent censorship like psiphon and riseupvpn and proton vpn.....

Thanks again for this fantastic dns Mr @ragibkl!

ragibkl commented 2 years ago

I've added the following hosts to our config. You can see this commit: https://github.com/ragibkl/adblock-dns-server/commit/177d28512b0a73fb8724dddf26f0febaf77061b1

I did some rough tests, and seems like no issues so far. I believe we've answered majority of the request from this issue.

I think we can close this issue now. As always, if you encountered any issues, do report back here, maybe just open another issue ticket.

Thanks @trimechee for reporting the issue! Thanks @Tomatoide for helping debugging! Now and in the past you have been very helpful to the project!

trimechee commented 2 years ago

Yes of course ! If I meet problems or I have suggestions, I'll inform you in Sha Allah!

Thank you very much for your help and to listen to users users and for your great generosity and kindness :)