search

FiltersHeroes / KADhosts

Wersja hosts, PiHole, dnsmasq, domenowa (Forti Guard) filtrów KAD
https://kadantiscam.netlify.app/
Creative Commons Attribution Share Alike 4.0 International
55 stars 7 forks source link

Question about high amount of sleepers/parked domains #79

Closed ldlac closed 3 weeks ago

ldlac commented 3 weeks ago

Hi, first of all, thanks for your good work, you guys have a huge hosts list!

I was wondering why there is so many sleepers/parked domains in the list? I'm actively building an hosts list using multiple sources and I found out that you guys have around 31833 sleepers. Are you keeping them just in case they submerged? Is it safe to remove them?

hawkeye116477 commented 3 weeks ago

We're removing parked (based on parkers ns => https://github.com/FiltersHeroes/ScriptsPlayground/blob/master/domainParking.txt) and expired domains if our script will find them. How you found so many? Some domains can be offline, but not marked by whois as expired yet, so they're kept, cuz can always come back.

krystian3w commented 3 weeks ago

There may also be a problem that a given domain has a deliberately damaged homepage and only with unknown parameters or URI path it returns text worth blocking (Then for offline records it is safer to recognize deletion from WHOIS).

We certainly do not know such parameters and paths for CERT POLSKA (they may not necessarily be willing to publish the exact URLs from the reports to reduce block before his 180 days of quarantine).

ldlac commented 3 weeks ago

Thanks for your responses

I'm currently experimenting with https://pypi.org/project/dnspython/ simply trying to resolve the domain. So they simply do not have an A record which is why they are still on the list, my bad on this just figured that out. I recently discovered this tool too https://pypi.org/project/PyFunceble/ which could give me better results.

hawkeye116477 commented 3 weeks ago

We're also using dnspython for our script, but that's just one of the methods. You need more to make sure that domain won't come back. https://github.com/FiltersHeroes/ScriptsPlayground/blob/master/scripts/ECODFF.py.