StevenBlack / hosts

πŸ”’ Consolidating and extending hosts files from several well-curated sources. Optionally pick extensions for porn, social media, and other categories.
MIT License
26.17k stars 2.18k forks source link

Adding to the list with new items #1655

Closed ghost closed 3 years ago

ghost commented 3 years ago

Hello, Can you please add all the lists from NextDNS

@StevenBlack Can you please add one more thing, lets say the domain has been expired for more than 90 days or something like that, and it won't come on the list, I went over the list and found so many domains that have not even been registered

welcome[bot] commented 3 years ago

Hello! Thank you for opening your first issue in this repo. It’s people like you who make these host files better!

StevenBlack commented 3 years ago

@Prajwal-Koirala I'm gonna need a link to the NextDNS list you're alluding, and a case elucidating why we should carry it.

Here's my view on expired and non-resolving domains, and subdomains:

Imagine, for a moment, that you're in the malware business. List smart things that you could do.

Issuing queries to non-resolving domains and subdomains is a very smart tactic.

Because these can be activated, whenever you're ready.

There are actually bots that clean domains and subdomains from blocklists that are known request targets in the wild, but flushed simply because DNS doesn't presently resolve after a minimal, cursory check.

I'm very happy if the list contains expired or presently not-resolving domains and subdomains. Sleeping malware is the worst.

ghost commented 3 years ago

https://nextdns.io/ They are the best DNS level ad blockers; I don't want to use them because they aren't open source, but when I tested them they blocked 99 percent of all ads in the United States. (If you add their lists, it will catch everything.)

Also I am not talking about non-resolving domains, I am talking about expired domains, Lets say a domain registration has been expired, than that domain should be removed from the list. If a domain is not-resolving removed A, AAA record than it should stay on the list

Issuing queries to non-resolving domains and subdomains is a very smart tactic.
Because these can be activated, whenever you're ready.

I understand you, but I disagree with you, cause there is no solution to this, "I'm very happy if the list contains expired or presently not-resolving domains and subdomains. Sleeping malware is the worst." unless you block all new registered domains for 30 - 90 days.

https://lookup.icann.org/

StevenBlack commented 3 years ago

@Prajwal-Koirala I need a specific link to the exact list you think we need to carry. Link me to the license terms too, if those aren't evident.

Thanks.

ghost commented 3 years ago

Hey,

Repo: https://github.com/AdguardTeam/AdguardSDNSFilter
License: https://github.com/AdguardTeam/AdGuardSDNSFilter/blob/master/LICENSE
List: https://adguardteam.github.io/AdGuardSDNSFilter/Filters/filter.txt
---
Repo: https://github.com/hoshsadiq/adblock-nocoin-list/
License: https://github.com/hoshsadiq/adblock-nocoin-list/blob/master/LICENSE
List: https://raw.githubusercontent.com/hoshsadiq/adblock-nocoin-list/master/hosts.txt
---
Website: https://oisd.nl/
List: https://abp.oisd.nl/
---
Repo: https://github.com/notracking/hosts-blocklists
License: https://creativecommons.org/licenses/by-nc-sa/4.0/
List: https://raw.githubusercontent.com/notracking/hosts-blocklists/master/adblock/adblock.txt
---
Repo: https://github.com/jerryn70/GoodbyeAds
License: https://github.com/jerryn70/GoodbyeAds/blob/master/LICENSE
List: https://raw.githubusercontent.com/jerryn70/GoodbyeAds/master/Hosts/GoodbyeAds.txt
---
Website: https://easylist.to/
Note: please add all of them.
---
Repo: https://github.com/AdAway/adaway.github.io
License: https://github.com/AdAway/adaway.github.io/blob/master/LICENSE.md
List: https://raw.githubusercontent.com/AdAway/adaway.github.io/master/hosts.txt
---
Repo: https://github.com/anudeepND/blacklist
License: https://github.com/anudeepND/blacklist/blob/master/LICENSE
List: https://raw.githubusercontent.com/anudeepND/blacklist/master/adservers.txt
---
Repo: https://github.com/crazy-max/WindowsSpyBlocker
License: https://github.com/crazy-max/WindowsSpyBlocker/blob/master/LICENSE
Note: There are alot of lists here.
---
List: https://someonewhocares.org/hosts/
---
Repo: https://gitlab.com/quidsup/notrack-blocklists
License: https://gitlab.com/quidsup/notrack-blocklists/-/blob/master/LICENSE
List: https://gitlab.com/quidsup/notrack-blocklists/raw/master/notrack-blocklist.txt
---
Repo: https://github.com/lightswitch05/hosts
License: https://github.com/lightswitch05/hosts/blob/master/LICENSE
List: https://www.github.developerdan.com/hosts/lists/ads-and-tracking-extended.txt
---
Repo: https://github.com/Yhonay/antipopads
License: https://github.com/Yhonay/antipopads/blob/master/LICENSE
List: https://github.com/Yhonay/antipopads/blob/master/popads.txt
---
Repo: https://github.com/EnergizedProtection/block
License: https://github.com/EnergizedProtection/block/blob/master/LICENSE
Multiple Lists: https://github.com/EnergizedProtection/block
---
Repo: https://github.com/badmojr/1Hosts/
Multiple Lists: https://github.com/badmojr/1Hosts/
---
Repo; https://github.com/Perflyst/PiHoleBlocklist
License: https://github.com/Perflyst/PiHoleBlocklist/blob/master/LICENSE

There are so many more lists, please check it here. https://my.nextdns.io/178678/privacy

dnmTX commented 3 years ago

! Title: oisd full
! Syntax: abp
! Entries: 428986 <--- COOL πŸ˜„

ghost commented 3 years ago

! Title: oisd full ! Syntax: abp ! Entries: 428986 <--- COOL πŸ˜„

I am new to the project, can u please the explain.

dnmTX commented 3 years ago

Hmm sure. Allow the explain: It's the word COOL(google it) and smile emoji(google that also) How is that?

lightswitch05 commented 3 years ago

Looks like my list is referenced here. I've really enjoyed not being a popular 'default' list. Please ignore the request to add my list πŸ‘

ghost commented 3 years ago

Hey @lightswitch05 Curious why not?

StevenBlack commented 3 years ago

Thank you @Prajwal-Koirala but I'm going to decline.

There are many lists out there listing 10^6++ domains, but this isn't one of them.

We are an amalgamation of actively curated lists that serve a wide variety of purposes including effective coverage on modest devices like mobile phones and consumer-grade userland computing platforms such as Windows. We've been doing this a long time.

Thank you for your input.

Closing.

lightswitch05 commented 3 years ago

@Prajwal-Koirala I'm pretty opinionated about what should or shouldn't be on my list. I don't know anything about NextDNS - but I would not be surprised if they apply whitelists to my list. I prefer people to add my list as extra rather then being included in a default list. That way they can be less surprised if something goes wrong. Also people who go out of their way to add a list is more likely to be able to handle whitelisting something on their own if it breaks instead of demanding I remove it from the list. I enjoy a friendly discussion in tickets about what should or shouldn't be blocked - but at the end of the day its my decision, and that doesn't always line up with what people consider a 'default' list to do.

ghost commented 3 years ago

@lightswitch05

I agree with you, and want to use all of your lists in my personal projects, but can u make the lists very separated, for example you have a list called facebook and it blocks all the social services related to Facebook and not only Facebook, that's the only thing I would change, so for facebook list it would only block facebook and not WhatsApp too.

Note: you do not have to make the change if you don't want to, it would just be easier for other people to use your list.

Also question why in the world are you blocking AMP? I really don't see anything wrong with it.

One last thing, i love this list https://www.github.developerdan.com/hosts/lists/hate-and-junk-extended.txt

https://github.com/complexorganizations/unbound-manager

lightswitch05 commented 3 years ago

I don't believe that is possible to do with a hosts formatted block list - at least not effectively. Facebook reuses some servers/domains between their various services, and I'm not interested in navigating which ones are required between the various severs to not break anything. Beyond that, I don't have Facebook or WhatsApp - so I could not test reports of broken functionality and would be completely dependent user reports to determine required domains - which I'm also not interested in doing.

Example whatsapp subdomains that are using Facebook's root domain:

Anyways, we're off topic for this project and I'm sure there are plenty of people getting notifications who would rather not be disturbed. Please open a ticket in my project if you wish to further discuss my lists.