Closed dnmTX closed 5 years ago
The point of this is everyone here to review it and either agree or disagree before it gets removed.
Just for clarity, this is a list of domains that are currently in the whitelist, and this review is for removing them from the whitelist. Right?
Okay let's go, but first let me reexplain:
ALL .gov == .*\.gov$
that means that we match everything which ends with .gov
:wink: So for example cmicapui.ce.gov.br
is not excluded/whitelisted with ALL .gov
:smile_cat:
[x] Let's change ALL .digitaltrends.com
with REG ^(?!.*\.?(vstats)).*\.digitaltrends\.com$
which will match everything except the vstats
subdomain.
REG ^(?!.*\.?(vstats|subdomaintoexclude)).*\.digitaltrends\.com$
:+1: [x] The review of the ALL .gov
will be for me as it imply a more complicated rule :smile_cat:
[x] Let's change ALL .twimg.com
with REG ^(?!.*\.?(oem)).*\.twimg\.com$
which will match everything except the oem
subdomain.
[x] Let's change ALL .ubuntu.com
with REG ^(?!.*\.?(popcon)).*\.ubuntu\.com$
which will match everything except the popcon
subdomain.
[x] About allegro.pl
, amazon.co.uk
, amazon.de
and arstechnica.com
it does not match the list you constructed :+1:
@lightswitch05 Yes this is a review of the whitelist :+1:
If a domain is deleted, I think that it may be better to remove it from the core whitelist list and add it into a new (or an existent) category of https://github.com/Ultimate-Hosts-Blacklist/dev-center/tree/whitelisting/data
Keep up the good work :+1: Cheers, Nissar
OK,i did some adjustments to my first post. @funilrys
In the future we could then update that line with all subdomains which do not have to be whitelisted with something like REG ^(?!..?(vstats|subdomaintoexclude))..digitaltrends.com$ š
this should apply for ALL .twimg.com
and ALL .ubuntu.com
as well
P.S. Remember,once you do the changes to let me know so i can move to the next letter down.
@lightswitch05
Just for clarity, this is a list of domains that are currently in the whitelist, and this review is for removing them from the whitelist. Right?
Yes,the whitelist is way too agressive at the moment so i'm just making it easier for us to see which ones needs permanent removal.
@dnmTX All checked in the list (on my previous comment) let's move on :+1:
@lightswitch05 Actually the elements which were on the whitelist list were all completely bold lines as you can see in my previously linked commits.
B b.scorecardresearch.com
0.0.0.0 b.scorecardresearch.com
-lightswitch,anudeep,stevenblack (remove please)
banners.spiceworks.com0.0.0.0 banners.spiceworks.com
-stevenblack
bestbuy.com0.0.0.0 context.bestbuy.com
-anudeep0.0.0.0 smetrics.bestbuy.com
-anudeep,stevenblack0.0.0.0 metrics.bestbuy.com
-stevenblack
betanews.com0.0.0.0 ads.betanews.com
-stevenblack
bing.com0.0.0.0 bat.bing.com
-lightswitch,quidsup0.0.0.0 r.bat.bing.com
0.0.0.0 0.r.bat.bing.com
0.0.0.0 1040004.r.bat.bing.com
0.0.0.0 1181982.r.bat.bing.com
0.0.0.0 136002880.r.bat.bing.com
0.0.0.0 136004927.r.bat.bing.com
....many more0.0.0.0 778802.r.bat.bing.com
-anudeep0.0.0.0 bat.bing.com
0.0.0.0 bat.bing.com
-stevenblack0.0.0.0 c.bing.com
0.0.0.0 ads.bing.com
0.0.0.0 adserver.bing.com
bleacherreport.com0.0.0.0 analytics.bleacherreport.com
-stevenblack
blogger.com0.0.0.0 xyz.freeweblogger.com
-stevenblack0.0.0.0 stats.blogger.com
bloomberg.com0.0.0.0 ads.bloomberg.com
-stevenblack0.0.0.0 metrics.bloomberg.com
btstatic.com0.0.0.0 btstatic.com
-lightswitch,quidsup0.0.0.0 l.btstatic.com
0.0.0.0 s.btstatic.com
-lightsitch,stevenblack0.0.0.0 s-eu.btstatic.com
0.0.0.0 t.btstatic.com
0.0.0.0 www.btstatic.com
0.0.0.0 s.yjtag.btstatic.com
businessinsider.com0.0.0.0 oascentral.businessinsider.com
-stevenblack
buzzfeed.com0.0.0.0 pixiedust.buzzfeed.com
-stevenblack,quidsup0.0.0.0 buzzbox.buzzfeed.com
Note: I edited your comment so that I can have checkbox
@dnmTX All :heavy_check_mark:, let's move on :+1:
C c.msn.com
0.0.0.0 c.msn.com
-anudeep,stevenblack
cda.pl0.0.0.0 ebd.cda.pl
-stevenblack0.0.0.0 e-cda.pl
0.0.0.0 i-cda.pl
0.0.0.0 play-cda.pl
0.0.0.0 ebd.cda.pl
cdn.betrad.com0.0.0.0 cdn.betrad.com
-anudeep,stevenblack
cdn.mplxtms.com0.0.0.0 cdn.mplxtms.com
-lightswitch,stevenblack0.0.0.0 secure-cdn.mplxtms.com
cdn.optimizely.com0.0.0.0 cdn.optimizely.com
-lightswitch,stevenblack0.0.0.0 4044060348.cdn.optimizely.com
0.0.0.0 7852160167.cdn.optimizely.com
....many more
cloudfront.net0.0.0.0 d1spgg7psata7d.cloudfront.net
-lightswitch,anudeep,stevenblack,quidsup0.0.0.0 d21j0tl4id6lf2.cloudfront.net
0.0.0.0 d2leoy3k16s14d.cloudfront.net
...many many more
cnn.com0.0.0.0 mms.cnn.com
-lightswitch0.0.0.0 smetrics.cnn.com
0.0.0.0 www.smetrics.cnn.com
0.0.0.0 data.cnn.com
-stevenblack0.0.0.0 metrics.cnn.com
0.0.0.0 ads.cnn.com
collegehumor.com0.0.0.0 ads.collegehumor.com
-stevenblack
cookie.monster.com0.0.0.0 cookie.monster.com
-stevenblack
cracked.com0.0.0.0 oads.cracked.com
-stevenblack0.0.0.0 beacon.cracked.com
0.0.0.0 ads.cracked.com
0.0.0.0 track.cracked.com
crateandbarrel.com0.0.0.0 metrics.crateandbarrel.com
-lightswitch.....to be continue(tomorrow first thing)
@funilrys if whitelist entry is for example cnn.com
is that exact match or it will match also sub-domains,basically anything in the front like mms.cnn.com
? Just to narrow my search better and not waste time.
P.S. Please don't get to technical,basic explanation will suffice.Thanks.
@dnmTX exactmatch.com
and www.exactmatch.com
.
@dnmTX All :heavy_check_mark:, let's move on :+1:
D directvapplications.hb.omtrdc.net
0.0.0.0 directvapplications.hb.omtrdc.net
-anudeep
E
edge.quantserve.com0.0.0.0 edge.quantserve.com
-lightswitch,anudeep,stevenblack0.0.0.0 www.edge.quantserve.com
express.co.uk0.0.0.0 www.express.co.uk
-stevenblack(in "fake news",might have to vote on this one)Let's ask @xxcriticxx opinion on that last one express.co.uk
as it was his request mitchellkrogza/Ultimate.Hosts.Blacklist#424 :+1:
@dnmTX Let's move on, I'll check that later. I'm not against the "fake news" extension of Steve but it is really based on opinion sometimes ...
G g.msn.com
0.0.0.0 g.msn.com
-stevenblack
gekko.spiceworks.com0.0.0.0 gekko.spiceworks.com
-lightswitchgoogle-analytics.com
-REMOVE PLEASE
googlesyndication.com0.0.0.0 googlesyndication.com
-lightswitch,anudeep,stevenblack,quidsupH
hbogo.com
0.0.0.0 hbogo.com.112.207.net
-lightswitch(just in case,could be a match)
I
ipinfo.io
ipinfo.io #IPinfo
-quidsup
:heavy_check_mark: Let's move on :+1:
@funilrys what about gekko.spiceworks.com
?
I'm not removing ipinfo.io
. Let's move on to next letter :+1:
L l.betrad.com
0.0.0.0 l.betrad.com
-anudeep,stevenblack0.0.0.0 www.l.betrad.com
localhost
-why is this even here?Some hosts file have a line like
127.0.0.1 localhost
that line removes it as we generate our own "header" :smile_cat:
@dnmTX :heavy_check_mark:
doing it by hand,that's why it's slow but don't want to miss something.
M
metrics.plex.tv
metrics.plex.tv #PleX
-quidsup
:heavy_check_mark:
special request about openload.co
domain(s).If you can expend after the dot .
to cover more possibilities.
Anything with two letters to be considered.
O
oas.monster.com
0.0.0.0 oas.monster.com
-stevenblack
@dnmTX :heavy_check_mark:
P pixel.facebook.com
0.0.0.0 pixel.facebook.com
-stevenblack,quidsup
pixel.quantserve.com0.0.0.0 pixel.quantserve.com
-lightswitch,anudeep,stevenblack
px.spiceworks.com0.0.0.0 px.spiceworks.com
-lightswitch,anudeep@funilrys i'd need more info on REG ^ebay.(?:[a-z\.]+)$
to know what exactly is covered by it.
@dnmTX :heavy_check_mark:
@dnmTX ==> All ebay.xyz domains :)
R rad.msn.com
0.0.0.0 rad.msn.com
-anudeep,stevenblack
S
s-media-cache-ak0.pinimg.com0.0.0.0 s-media-cache-ak0.pinimg.com
-anudeep
s.shopify.com0.0.0.0 s.shopify.com
-stevenblack
s.youtube.com0.0.0.0 s.youtube.com # Blocking this will interfere with video watching history
-stevenblack
(it's still tracking/telemtry,has to go)
s.zkcdn.net0.0.0.0 s.zkcdn.net
-anudeep,stevenblack
secure-cdn.mplxtms.com0.0.0.0 secure-cdn.mplxtms.com
-lightswitch
secure-us.imrworldwide.com0.0.0.0 secure-us.imrworldwide.com
-stevenblack
settings-win.data.microsoft.com0.0.0.0 settings-win.data.microsoft.com
-stevenblack
ssl.google-analytics.com
-REMOVE PLEASE !!!!!!!!!!!!!!!!!!!!!!!!!!!
syndication.twitter.com0.0.0.0 syndication.twitter.com
-lightswitch,anudeep
Tteredo.ipv6.microsoft.com
-spy domain,present in @crasy-maxthepiratebay.org
-this one is blocked in ZeroDot1 lists.Just FYI.track.spiceworks.com
-wondering what does that break(it's not present anywhere though)@funilrys not much left and i'll finish them all tomorrow.When you have time just do the changes. š
Please report to mitchellkrogza/Ultimate.Hosts.Blacklist#400 for syndication.twitter.com (will check later
Please report to mitchellkrogza/Ultimate.Hosts.Blacklist#115 for teredo.ipv6.microsoft.com (will check later)
I'm not removing thepiratebay.org
@dnmTX :heavy_check_mark:
Please report to mitchellkrogza/Ultimate.Hosts.Blacklist#400 for syndication.twitter.com
@lightswitch05 you have it in your lists.Does it brake the player really?
0.0.0.0 v10.vortex-win.data.microsoft.com
-anudeep,stevenblackvideo-stats.l.google.com
-very suspicious(not present anywhere)
W
webcache.googleusercontent.com
0.0.0.0 webcache.googleusercontent.com.pathful.com
-lightswitch(possible match)
That's it.Done and done :smile:
I'll be monitoring the whitelisted.list
for the feature and if anything i'll report it here š
I use the twitter website daily with syndication.twitter.com
blocked and I have never had any issues. However, I don't ever log into my account - I just browser a couple public profiles. I also don't use the app, so maybe I'm not an average user - but blocking this doesn't affect me
Thanks @lightswitch05 . @funilrys :point_up:
@dnmTX @lightswitch05 @smed79 Thanks for the inputs :+1:
Keep up the good work :rocket: :tada:
Cheers, Nissar
:hugs:
@funilrys i understand that all the changes that we did here are not merged yet but i did check the
whitelisted.list
in justdomains_ to see what was removed from the clean.list
and these are the results:
google.maniyakat.cn
googlecentreservices.rockhillrealtytx.com
googledrivedocument.beechdrift.co.uk
googlegetmyphotos.pythonanywhere.com
googlegetmysyncphotos.pythonanywhere.com
googlegetphotos.pythonanywhere.com
None of those appear to be in the whitelist and you know,they're all malicious and should be blocked. Please inspect.Thank you.
P.S. looks to me that everything that starts with google...
has been whitelisted.
Just FYI. Once the changes got merged and due to this post here became very long,i'll be opening a new issue as a continuation of this one here to report further if any more changes/adjustments needs to be done.
@dnmTX Fixed. All next generation of the whitelisted.list
from now will do things the right way.
@funilrys there is something wrong in the lightswitch05 repo since the last filtering:
INVALID\hosts
-duplicates
INACTIVE\hosts
-lots and lots of duplicates...
...not sure what else but right now it's a mess.
same at @dead-hosts ? If not then it's because of the Travis :thinking:
Will check that later.
Well...at @dead-hosts is in filtering so i don't know.I'm kind of focused on things here at the moment.I don't even visit there as of late.
@lightswitch05 @smed79 @funilrys @anudeepND @quidsup
As i'm little hesitant to start removing domains on which i have no prior knowledge of i decided to compare each entry from the whitelist against various lists to see if there are any matches.Lists that i used for the comparison: @lightswitch05 @anudeepND @StevenBlack @justdomains @quidsup I'll be listing here the whitelist entry,domains found that matched and lists that were found in. Due to whitelist is kind of big(1000+) i'll be doing it in alphabetical order,once we done with one i''ll move to the next letter down.The point of this is everyone here to review it and either agree or disagree before it gets removed. Unless commenting is necessary use thumbs UP or DOWN just to keep it cleaner here. @funilrys you got the honor to remove or leave the entrie(s),once confirmed by you i''ll move to the next one. Thank you all for participating.Here you go: A ALL .digitaltrends.com
0.0.0.0 vstats.digitaltrends.com
-stevenblack ALL .gov -complicated/work in progress ALL .twimg.comoem.twimg.com
#Twitter possible Windows spying -quidsup ALL .ubuntu.compopcon.ubuntu.com
#Ubuntu unaproved tracking -quidsup