Combined list TLD's - Githubissues

devipasigner commented 1 year ago

Missing Abused TLDs from yokoffing nextdns

.agency .ci .fun .link .live .shop .win

Thank you, I have some other personal abused TLDs, I will send them for review once I am home

yokoffing commented 1 year ago

Odd that uBo and AGH have different rules, although I guess it makes sense for a DNS blocker

$denyallow in uBO requires being coupled with $domain, for some reason.

@gorhill

gorhill commented 1 year ago

The reason is that a long time ago there was a commit with denyallow mistakenly without a domain option and this broke the whole internet for uBO users.

hagezi commented 1 year ago

@yokoffing https://raw.githubusercontent.com/hagezi/dns-blocklists/main/adblock/spam-tlds-ublock.txt

hagezi commented 1 year ago

@gorhill thanks for the clarification.

yokoffing commented 1 year ago

https://raw.githubusercontent.com/hagezi/dns-blocklists/main/adblock/spam-tlds-ublock.txt

Works in uBO!

@hagezi Check your exclusions / exception lines. .cn .ru .in were reintroduced.

hagezi commented 1 year ago

@yokoffing only for uBlock version, not for DNS version.

yokoffing commented 1 year ago

only for uBlock version, not for DNS version.

Very nice! Everything looks great, then.

hagezi commented 1 year ago

Thanks for the great cooperation to all involved, that's how it has to be. I'll leave the issue open, there are still a few things to clarify and then you have a point of contact.

hagezi commented 1 year ago

The parsed data from the individual lists is in converted form in my Data Collection if anyone wants to track changes to the individual sources:

https://github.com/hagezi/dns-data-collection/commits/main/data/TLDs_spamhaus.txt https://github.com/hagezi/dns-data-collection/commits/main/data/TLDs_yokoffing.txt https://github.com/hagezi/dns-data-collection/commits/main/data/TLDs_dandelionsprout.txt

hagezi commented 1 year ago

@yokoffing Found an exception for CN: nvgs.nvidia.cn is required for the GEFORCE Experience login. This is called accounts.nvgs.nvidia.cn.

yokoffing commented 1 year ago

@hagezi https://github.com/yokoffing/filterlists/pull/42/commits/7c16d97f94ecbe887bb1b9186c68f2f0f07cecd7

devipasigner commented 1 year ago

@hagezi .site false positive, fitgirl-repacks.site

hagezi commented 1 year ago

Ping source: @yokoffing

yokoffing commented 1 year ago

So, some updates for my filterlist:

@bestplayerbot has added a ton of exceptions for .fun and .ru (thanks!). But I think it raises a larger concern of high false positives from blocking country-specific TLDs (which is why hagezi allowlists these for the DNS version).
I think many of these should be manually enabled by users who don't live in those countries.
- e.g., .in "[The filter list] blocks all banking websites in India and popular sites like amazon.in etc." https://github.com/yokoffing/filterlists/pull/59/files
Also, to speak bluntly, these are too much work to allowlist. It is an unreasonable burden for the maintainer to allowlist country-specific TLDs when the filterlist is used internationally.
- Therefore, I will be allowlisting .cn, .in, and .ru in the next major update.
- If you want to retain them, add the rules to your custom filters in uBO/AdGuard.
- If someone wants to create a "block country-specific TLDs" list and use my rules, they're welcome to do so.

devipasigner commented 1 year ago

So, some updates for my filterlist:

@bestplayerbot has added a ton of exceptions for .fun and .ru (thanks!). But I think it raises a larger concern of high false positives from blocking country-specific TLDs (which is why hagezi allowlists these for the DNS version).

I think many of these should be manually enabled by users who don't live in those countries.

e.g., .in "[The filter list] blocks all banking websites in India and popular sites like amazon.in etc." https://github.com/yokoffing/filterlists/pull/59/files

Also, to speak bluntly, these are too much work to allowlist.

Therefore, I will be allowlisting .cn, .in, and .ru in the next major update.

If you want to retain them as a filterlist, add them to your custom rules for uBO/AdGuard.

If someone wants to create a "block country-specific TLDs" list and use my rules, they're welcome to do so.

Thank you for coming to my TED Talk

Awesome! Thanks for the update as usual

hagezi commented 1 year ago

Thanks @yokoffing, I absolutely agree.

iam-py-test commented 1 year ago

If someone wants to create a "block country-specific TLDs" list and use my rules, they're welcome to do so.

I have copied a few of your entries, added some new domains to the allowlist, and uploaded them here: https://github.com/iam-py-test/my_filters_001/tree/main/region_blocklist So far I have in and ru, but I might add a few more.

ghost commented 1 year ago

.cn is still there

hagezi commented 1 year ago

.cn is still there

Thanks fixed. Came via spamHAUS.

RicoHeat commented 1 year ago

Just my 2 cents on the matter.. https://www.reddit.com/r/uBlockOrigin/comments/10s4n1a/how_to_combine_multiple_tlds_into_one/

https://hblock.molinero.dev/most_abused_suffixes.txt https://hblock.molinero.dev/most_abused_tlds.txt @hectorm

Excellent work everyone!!! 👍

hagezi commented 1 year ago

Thanks @RicoHeat

krystian3w commented 1 year ago

Why list have ||*.?

hagezi commented 1 year ago

Why list have ||*.?

https://github.com/hagezi/dns-blocklists/issues/143#issuecomment-1373833494

ghost commented 1 year ago

useful to find false positives

Allow

aclub.okinawa
activity.okinawa
aquadrop.okinawa
asatoyasuba.okinawa
beachboys.okinawa
churaumi.okinawa
dancestudioon.okinawa
diver.okinawa
ecco.okinawa
eight.okinawa
emoh.okinawa
fidelity.okinawa
growth.okinawa
gukuru.okinawa
holdbyhand.okinawa
jupiter.okinawa
kanoa.okinawa
kerouac.okinawa
kimishimagroup.okinawa
kokorononekko.okinawa
makeman.okinawa
mansaku.okinawa
minsyo.okinawa
morinaga.okinawa
naminouta.okinawa
nanaironote.okinawa
okmarine.okinawa
pacificview.okinawa
rentakun.okinawa
retime.okinawa
ryukyugobyou.okinawa
saya.okinawa
seek.okinawa
seitai.okinawa
selfstorage.okinawa
sgt.okinawa
shigaku.okinawa
simplelife.okinawa
sir.okinawa
sisei.okinawa
soilcure.okinawa
strawberry.okinawa
suba.okinawa
suncoast.okinawa
takakogyo.okinawa
thekucha.okinawa
tsukishiro.okinawa
yamabuki.okinawa
yorozu.okinawa
zwwa.okinawa

hagezi commented 1 year ago

@bestplayerbot Thanks, I have to adapt my script so that denyallow can be implemented for TLDs from the SpamHaus source. Have you checked the Okinawas to see what's behind them? They look wild ... :)

ghost commented 1 year ago

I visited all of them but I don't understand Japanese so I don't exactly know what they are.

yokoffing commented 1 year ago

Might need to use a browser with native translation support like Chrome or Edge in a private window to determine the content of the sites.

ghost commented 1 year ago

Visited all of them again on chrome growth.okinawa didn't load on chrome

hagezi commented 1 year ago

@bestplayerbot Thanks! Implemented exclusions for the SpamHAUS TLDs. The okinawa's are in now.

hagezi commented 1 year ago

@bestplayerbot If you want to add exclusions for the SpamHaus TLDs, just "pull request" this file. If new TLDs appear for which an exclusion is necessary, simply create a new line for the TLD. Only applies to TLDs that only come through SpamHaus.

RicoHeat commented 1 year ago

I visited all of them but I don't understand Japanese so I don't exactly know what they are.

@Yuki2718

Yuki2718 commented 1 year ago

Not scam, legit sites. .okinawa is a regional TLD same as .tokyo.

hagezi commented 1 year ago

Thanks @Yuki2718 ;)

ghost commented 1 year ago

Is *.okinawa going to get removed?

hagezi commented 1 year ago

@bestplayerbot No, why? https://github.com/hagezi/dns-blocklists/blob/a2c2bbeb58225b8c9fe2647c783276411b23bfcc/adblock/spam-tlds.txt#L31

ghost commented 1 year ago

I just wanted to know if it is getting removed.

Yuki2718 commented 1 year ago

why?

As it has nothing to do with scam/spam? There will be many more legitimate sites using .okinawa. e.g. tsurikatsu.okinawa, ocmla.okinawa

hagezi commented 1 year ago

@Yuki2718 https://www.spamhaus.org/statistics/tlds/ Rank 2 among the top spam TLDs ...

grafik

Yuki2718 commented 1 year ago

Do they provide actual example? Frankly, I haven't ever felt SH is very reliable.

ghost commented 1 year ago

False Positive: nic.*

nic.cf
nic.ga
nic.gq
nic.ml
nic.tk

doesn't load

ghost commented 1 year ago

nic.agency
nic.beauty
nic.bid
nic.cfd
nic.degree
nic.discount
nic.fit
nic.fun
nic.fyi
nic.gdn
nic.loan
nic.loans
nic.monster
nic.okinawa
nic.online
nic.ooo
nic.pw
nic.quest
nic.rest
nic.sbs
nic.site
nic.surf
nic.top
nic.uno
nic.zone

hagezi commented 1 year ago

So there would be exceptions for almost all spam TLDs. Does this really make sense for someone who uses a spam TLD list?

ghost commented 1 year ago

Every TLD should have a false positive.

ghost commented 1 year ago

nic.agency
nic.beauty
nic.bid
nic.cfd
nic.degree
nic.discount
nic.fit
nic.fun
nic.fyi
nic.gdn
nic.loan
nic.loans
nic.monster
nic.okinawa
nic.online
nic.ooo
nic.pw
nic.quest
nic.rest
nic.sbs
nic.site
nic.surf
nic.top
nic.uno
nic.zone

When will these get allowed?

hagezi commented 1 year ago

Does this really make sense?

bestplayerbotphone commented 1 year ago

It's not possible for a TLD to be 100% malicious

False positives for *.okinawa

enokinawa.okinawa
nic.okinawa

e.g. tsurikatsu.okinawa, ocmla.okinawa

*.okinawa should be removed because I didn't find a single malicious domain while searching site:okinawa in multiple search engines

hagezi commented 1 year ago

spamHAUS sees it differently, 69% bad domains, meanwhile on the index place 2:

https://www.spamhaus.org/statistics/tlds/

ghost commented 1 year ago

I think the TLDs added from spamhaus aren't getting removed when they are no longer on the top 10 list

hagezi commented 1 year ago

Yes, those that are on the top 10 list or have been on the top 10 list in the past will be added.

ghost commented 1 year ago

@hagezi the latest commit caused some problems

hagezi / dns-blocklists

Combined list TLD's #143