mkb2091 / blockconvert

Malware, advert and tracking blacklist
MIT License
117 stars 14 forks source link
adblock adblock-list adblocker-lists ads blocklist blocklists dns filterlist hosts hosts-file hostsfile phishing-sites pihole-adblocker-list pihole-ads-list pihole-blocklists privacy-protection ublock-list ublock-origin

BlockConvert

Malware, advert and tracking blocklist which consolidates and improves upon many other blocklists.

What this blocks:

Advantages of using this list:

How to use:

Links

Adblock Plus format

Hosts file format

WARNING: Too large for Windows: https://github.com/mkb2091/blockconvert/issues/87

Domain list

Blocked IP address list

DNS Response Policy Zone(RPZ) format

As well as generating blocklists, this project also generates whitelists which are used in the process. If you maintain your own blocklist, you may find one of the following whitelists useful:

Whitelisted domains

Whitelisted ABP format

The Process

  1. Download all expired filterlists

  2. Combine and split all the filterlists based on their type. This splits the lines into seperate groups: Adblock rules, blocked domains, regexes of blocked domains, allowed domains, regex of allowed domains, ips which are blocked, ips which are allowed, subnets which are blocked, subnets which are allowed.

  3. Apply a regex to all the filterlists to extract domains and combine with other domains found via other means.

  4. For each of those domains, use DNS to check if the domain is still active. If the domain isn't in the allowed domains list, doesn't match any of the allowed regexes, isn't in allowed by an adblock exception rule and it is blocked, or one of its cnames/ips is blocked then add it to the output.

Sources: Sources