additional blocklists if you want more

[jawz101]

Basic tracking list by Disconnect‎ https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt

Disconnect Malvertising https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt

hosts-file.net phishing https://hosts-file.net/psh.txt

hosts-file.net misleading marketing https://hosts-file.net/mmt.txt

hosts-file.net hijack sites https://hosts-file.net/hjk.txt

hosts-file.net forumspam https://hosts-file.net/hfs.txt

hosts-file.net fraud sites https://hosts-file.net/fsa.txt

hosts-file.net exploit sites https://hosts-file.net/exp.txt

hosts-file.net malware sites https://hosts-file.net/emd.txt

securemecca http://securemecca.com/Downloads/hosts.txt

Cameleon hosts http://sysctl.org/cameleon/hosts

BBCan177 pfBlockerNG https://gist.githubusercontent.com/BBcan177/4a8bf37c131be4803cb2/raw

Eladkarako (huge) http://hosts.eladkarako.com/hosts.txt

StevenBlack https://github.com/StevenBlack/hosts/raw/master/hosts

YousList https://github.com/yous/YousList/raw/master/hosts.txt

[julian-klode]

We really want lists that are kept up-to-date and optimally have a reasonable procedure to add and remove entries. Preferably those lists should be accessible via HTTPS to prevent man-in-the-middle-attacks (as @dschuermann pointed out in #19 we should probably go https only) .

OK:

• Disconnect: Look good to me. Not sure if it's better than malwaredomainlist which is our current default (and yes, I noticed yesterday that that one is accessible via https too).

Neutral:

• securemecca/hostsfile.org: Not sure
• BBCan177: Looks OK, sort of
• YousList: Looks good to me too, but it's small. Not sure if it's not already contained in one of the other lists.

No go:

• hosts-file.net: As I said before, they are just too huge and uncontrolled. And adding all of the per-category entries would make things look really ugly. It also seems to be vastly focused around adding hosts, but there seems to be no way to remove hosts from the list (for example, some website that was cracked and served malware, but regained control in the meantime).
• sysctl.org: Not updated since February, so not really a good choice
• Eladkarako: Blocks access from non-Webbrowsers.
• StevenBlack: We already have the lists it is based on (that's the default), adding the merged list would be a bad user experience: Users would end up enabling the merged list and the basic lists and have longer startup times without any benefit to them. (If most users would know that and were clever enough, they would not do that, and cut their startup time in half, but unfortunately that's not the world we live in). It has one advantage: It's accessible via https - but I'm not sure if @StevenBlack verifies new updates from his http sources before merging them (might be a good way to get around the http-only components if he does).

[julian-klode]

One benefit of Disconnect vs MalwareDomainList is that it has a free software license attached to it. It's questionable if/where hosts files are copyrightable at all - most countries do not accept copyrights on databases IIRC, but having an explicit license is better than nothing.

[StevenBlack]

I always diff. One reason: on most days, yoyo.org changes just the date in their header :-)

Just kick me if you know this... https://github.com/StevenBlack/hosts/ comes in variants for porn, social media, gambling, and all combinations of those, in addition to the standard file.

Horses for courses.

[jawz101]

I wish there was a site like http://iplists.firehol.org/ but for DNS blocklists. It aggregates and analyzes all of the IP blocklists out there to see where there's overlap and how often they get updated. Kinda neat.

@StevenBlack one thing I noticed on your list is there aren't enough mobile ad networks & analytics domains on your list. I can rattle off several off the top of my head that I see when combing through Android apps as well as from NetGuard logs.

This is just the domains from the apps on my phone. The mobile app ecosystem has all sorts of domains that desktops simply don't have and none of the DNS blocklists are geared towards them. They're find for browsers and junk but basically you're phone is a sieve. I haven't found one blocklist, Android app, XPosed module which really addresses mobile junk. Maybe the closest is Adguard.

http://iamattila.com/wp-content/uploads/2013/09/Mobile-Advertising-Networks-1024x600.png

www.googleadservices.com www.googletagmanager.com secure.insightexpressai.com 169.54.33.154 9a.21.36a9.ip4.static.sl-reverse.com

... all of these IP's are for mixpanel analytics svcs through sl-reverse.com and mixpanel has something like a block of 128 IP's through sl-reverse.com with random names like the above 169.54.33.155 169.54.33.187 169.54.33.191 169.54.33.199 a.fiksu.com api.segment.io cdn.segment.com dc.services.visualstudio.com e.crashlytics.com mobile-service.segment.com settings.crashlytics.com analytics-prod-ingress.bricknet.com tags.otherlevels.com admin.brightcove.com app.adjust.com fonts.googleapis.com brahe.apptimize.com md-a-c.apptimize.com appload.ingest.crittercism.com s.amazon-adsystem.com device-metrics-us-2.amazon.com 14716.engine.mobileapptracking.com stats.appsflyer.com t.appsflyer.com t.singular.net control.kochava.com

[StevenBlack]

@jawz101 please open this as an issue here.

I only accept actively curated sources. Otherwise we end-up with host-file.net which has volume with minimal oversight, judging by how little it ever changes.

[jawz101]

Thanks

[pulento-1]

Hello. Does anyone know of the address to block "free" Amazon apps? Thanks

[guest4711]

To avoid long startup time and to optimize the tables I'd prefer splitting the actions:

1. Start service with old working copy (if any) of aggregrated filters.
2. Check for updates of the filterlists. Download updates to tempfile, replace on success. If all downloads are finished, create tables omitting duplicate entries.
3. Apply new tables by restarting service ...

This way it would be no bigg problem if users use redundant files. (AdAway aggregates the files into one hostfile omitting duplicates.)

[julian-klode]

Just to be clear: We do not have any tables, there is no database. The files are read when the service is (re)started or when a background update happens, and stored in one hashset in memory, no duplicates are ever stored.

There was an experimental database branch, but that was abandoned after I tweaked the file parser to have reasonable performance. The database branch was too fast to start, causing Android to mess up and break the VPN service on reconnects.

The service does not need to be restarted in either solution. Hostfile updates can happen while the service is running, they are immediately reflected in the service (but adding, removing, changing, or reordering host files are not).

Also, updates are completely uncoupled from the service. They have their own schedule managed by JobScheduler.