vokins / yhosts

AD hosts爱好群,群号:201973909;无限期暂停更新。劝君更尽一杯酒,西出阳关无故人。莫愁前路无知己,天下谁人不识君。
1.67k stars 513 forks source link

NXDOMAIN #41

Closed CNMan closed 7 years ago

CNMan commented 7 years ago

check with https://github.com/lennylxx/ipv6-hosts/blob/master/update_hosts.py

python update_hosts.py -s 8.8.8.8 -t a -c hosts
grep '^#' hosts.out > NXDOMAIN
python update_hosts.py -s 114.114.114.114 -t a -c NXDOMAIN
grep '^#' NXDOMAIN.out > NXDOMAIN
#127.0.0.1 hndnserror8.wo.com.cn
#127.0.0.1 jldnserror6.wo.com.cn
#127.0.0.1 jldnserror7.wo.com.cn
#127.0.0.1 jldnserror8.wo.com.cn
#127.0.0.1 lndnserror8.wo.com.cn
#127.0.0.1 bjdnserror8.wo.com.cn
#127.0.0.1 hbdnserror8.wo.com.cn
#127.0.0.1 ss.cnnic.cn
#127.0.0.1 admaster.mobi
#127.0.0.1 api2.adsmogo.com
#127.0.0.1 api2.adsmogo.mobi
#127.0.0.1 api2.adsmogo.net
#127.0.0.1 api2.adsmogo.org
#127.0.0.1 s.think.cn.maysunmedia.com
#127.0.0.1 cm.reachmax.cn
#127.0.0.1 1.wps.cn
#127.0.0.1 test.new.api.ad.xiaomi.com
#127.0.0.1 xlmc.sec.miui.com
#127.0.0.1 ad6.liverail.com
#127.0.0.1 ads-fa.spotify.com
#127.0.0.1 cq01-cm-et01.cq01.baidu.com
#127.0.0.1 adimg.dayoo.com
#127.0.0.1 dywg.dayoo.com
#127.0.0.1 icon.55bbs.com
#127.0.0.1 img.royaltianjin.com
#127.0.0.1 s.sucodb.com
#127.0.0.1 img.baotoushanghui.com
#127.0.0.1 i.rtkshealth.com
#127.0.0.1 img.geruishengwu.com
#127.0.0.1 ef363.263zw.com
#127.0.0.1 ef364.263zw.com
#127.0.0.1 ef365.263zw.com
#127.0.0.1 ef366.263zw.com
#127.0.0.1 fx.worldqueen.org.cn
#127.0.0.1 img.sanhaoyuan.cn
#127.0.0.1 hotav05.szjtcg.com
#127.0.0.1 img.tkguolugs.com
#127.0.0.1 ss.biquge.la
#127.0.0.1 tt.biquge.la
#127.0.0.1 t.91chs.com
#127.0.0.1 x.55peixun.com
#127.0.0.1 img.balaiyani.com
#127.0.0.1 edtp.shushu8.com
#127.0.0.1 edtz.shushu8.com
#127.0.0.1 ed.shushu8.com
#127.0.0.1 idxhy8s.xxbiquge.com
#127.0.0.1 img.jx.media8.cn
#127.0.0.1 s.mxmrt
#127.0.0.1 wx.8536818.com
#127.0.0.1 img.xinzhongwo.com.cn
#127.0.0.1 ns.apyuelang.com
#127.0.0.1 ss.apyuelang.com
#127.0.0.1 yl.apyuelang.com
#127.0.0.1 t.7jiajiao.com
#127.0.0.1 ima.xcyjzs.net
#127.0.0.1 imgcdn.qj06hfjkg.media8.cn
#127.0.0.1 oki.xcyjzs.net
#127.0.0.1 7.1qtv.com
#127.0.0.1 www.bncmf.com
#127.0.0.1 ii.bsshw.net
#127.0.0.1 img.khgj.cn
#127.0.0.1 il722.junxiaoniu.com
#127.0.0.1 i.bsshw.net
#127.0.0.1 ck.scltn.cn
#127.0.0.1 img.smart-idea.com.cn
#127.0.0.1 jump2.pszyzxh.org
#127.0.0.1 stats.liuliguo.com
#127.0.0.1 down.gotek.com.cn
#127.0.0.1 img.sxxjdz.com
#127.0.0.1 imgcdn.sxxjdz.com
#127.0.0.1 aa.scw0.com
#127.0.0.1 bb.78gua.com
#127.0.0.1 js.cccrir.com
#127.0.0.1 qj906nc.cdn.media8.cn
#127.0.0.1 tj.yule8.net
#127.0.0.1 img.yintengjiagu.com
#127.0.0.1 av.app583.com
#127.0.0.1 pic3.cstoa.com
#127.0.0.1 d2.maohetrade.com
#127.0.0.1 z.top267.com
#127.0.0.1 vip.alltone-textile.com
#127.0.0.1 qj01rd.cdn.media8.cn
#127.0.0.1 ad.fglighting.net
#127.0.0.1 link.jiduan.cc
#127.0.0.1 gm682.com
#127.0.0.1 img.shoubiao88.com.cn
#127.0.0.1 www1.boshipai.com
#127.0.0.1 pic.qjlm.com
#127.0.0.1 cc.yfi052.pw
#127.0.0.1 www.ychbgjg.cn
#127.0.0.1 img.78dxc.com
#127.0.0.1 m3z96.sanya1.com
#127.0.0.1 u349036.778669.com
#127.0.0.1 pylfi.flash.hxh999.com
#127.0.0.1 js.willads.pw
#127.0.0.1 zzad.com
#127.0.0.1 vr.mt145.com
#127.0.0.1 llqjs.gucheng.com
#127.0.0.1 yn.xsu.cc
#127.0.0.1 jsqcdn1124.0101122.com
#127.0.0.1 package01.com
#127.0.0.1 zgty365.com
#127.0.0.1 94776.zdfans.com
#127.0.0.1 ca.cbsi.com.cn
#127.0.0.1 data.cbsi.com.cn
#127.0.0.1 dmp.cbsi.com.cn
#127.0.0.1 dw.cbsi.com.cn
#127.0.0.1 stat.cbsi.com.cn
#127.0.0.1 f.kxbeta.com
#127.0.0.1 g.kxbeta.com
#127.0.0.1 r.71sem.com
#127.0.0.1 news.b9377h.com
#127.0.0.1 rayfilead.roxcdn.com
#127.0.0.1 k.zrwhcy.com
#127.0.0.1 a.stka8.com
#127.0.0.1 58.xgo.com.cn
#127.0.0.1 cpro.7791.com.cn
#127.0.0.1 imgcdn.mazaibang.com
#127.0.0.1 u.qoxyo.cn
#127.0.0.1 img.qj.media8.cn
#127.0.0.1 biz5.sandai.net
#127.0.0.1 float.sandai.net
#127.0.0.1 logic.cpm.cm.sandai.net
#127.0.0.1 analytics.xunlei.com
#127.0.0.1 kkpgv2.xunlei.com
#127.0.0.1 pv.xmp.stat.xunlei.com
#127.0.0.1 thunderqtypv.union.xunlei.com
#127.0.0.1 log.elk.hunantv.com
#127.0.0.1 push.dopool.com
#127.0.0.1 vid.atm.youku.com
#127.0.0.1 c.jk2h.com
#127.0.0.1 js1194.cnbole.net
#127.0.0.1 al.myrice.com
#127.0.0.1 img.nsywb.com
#127.0.0.1 ji.dazhantai.com
#127.0.0.1 sjkmio.812920.top
#127.0.0.1 down.by8974.com
#127.0.0.1 down.xzwd.360eus.cn
#127.0.0.1 js.ne08.com
#127.0.0.1 muban.dodounion.com
#127.0.0.1 aa-oo.top06.cn
#127.0.0.1 js.sanlitang.cn
#127.0.0.1 tj.itlily.com
#127.0.0.1 adweb.test.ximalaya.com
#127.0.0.1 slides.discovery.tom.com
#127.0.0.1 publish.ad.youth.cn
#127.0.0.1 img.jiayuannongye.cn
#127.0.0.1 ads.cndandi.cn
#127.0.0.1 jp.2015nian.com
#127.0.0.1 m3h126.85626666.com
#127.0.0.1 www-360se-cn.newtech-beauty.com
#127.0.0.1 img.0563d.com
#127.0.0.1 tonglan.nuoke88.cn
#127.0.0.1 t22.3sdaigou.com
#127.0.0.1 code2.huimee.net
#127.0.0.1 tz.lhzly.com
#127.0.0.1 c1.qshxc.com
#127.0.0.1 s.67yes.com
#127.0.0.1 www.ads8.cc
#127.0.0.1 i2.wljnj.com
#127.0.0.1 bdaz.adsfactor.net
#127.0.0.1 hotav30.szjtcg.com
#127.0.0.1 sdtu.sanyjx.com
#127.0.0.1 upload.jxsjmg.com
#127.0.0.1 sd.mmfile.net
#127.0.0.1 simg.52zbz.com
#127.0.0.1 imgcdn.ynwqls.com
#127.0.0.1 vip.yule8.net
#127.0.0.1 img.52tyd.com
#127.0.0.1 img.yinhemining.cn
#127.0.0.1 hc.wstztt.com
#127.0.0.1 s1.ynwqls.com
#127.0.0.1 img.baidu9635.com
#127.0.0.1 il728.caijuedian.com
#127.0.0.1 il82.tlywgs.cn
#127.0.0.1 il83.hrbfjddb.cn
#127.0.0.1 img.ynwqls.com
#127.0.0.1 iiad.liuliguo.com
#127.0.0.1 jx01bv.cdn.media8.cn
#127.0.0.1 ms.liuliguo.com
#127.0.0.1 j.mainbx.com
#127.0.0.1 amazon-adsystem.com
#127.0.0.1 g.doubleclick.net
#127.0.0.1 q.track-cn.imrworldwide.com
#127.0.0.1 www.imrworldwide.com
#127.0.0.1 y.analytics.yahoo.com
#127.0.0.1 users.51.la
#127.0.0.1 tga.acs86.com
#127.0.0.1 js.adm.cnzz.net
#127.0.0.1 tui.cnzz.net
#127.0.0.1 ca.cnzz.com
#127.0.0.1 opendsp.tanx.com
#127.0.0.1 agoodm.wapa.taobao.com
#127.0.0.1 ff.win.taobao.com
#127.0.0.1 intl.wapa.taobao.com
#127.0.0.1 intl.waptest.taobao.com
#127.0.0.1 vs6.gzct.uc.ucweb.com
#127.0.0.1 au.umeng.co
#127.0.0.1 alog.umeng.co
#127.0.0.1 oc.umeng.co
#127.0.0.1 www.umeng.co
#127.0.0.1 au.apk.umeng.com
#127.0.0.1 fb.umeng.com
#127.0.0.1 feedback.umeng.com
#127.0.0.1 dt.tongji.linezing.com
#127.0.0.1 img.tongji.linezing.com
#127.0.0.1 creatim.allyes.com.cn
#127.0.0.1 cbjslog.baidu.com
#127.0.0.1 mobads-speed.baidu.com
#127.0.0.1 ji.cnbole.net
#127.0.0.1 js1116.cnbole.net
#127.0.0.1 js1210.cnbole.net
#127.0.0.1 js1315.cnbole.net
#127.0.0.1 js1597.cnbole.net
#127.0.0.1 js2294.cnbole.net
#127.0.0.1 js3077.cnbole.net
#127.0.0.1 js3636.cnbole.net
#127.0.0.1 js3698.cnbole.net
#127.0.0.1 js4110.cnbole.net
#127.0.0.1 js436.cnbole.net
#127.0.0.1 js453.cnbole.net
#127.0.0.1 js883.cnbole.net
#127.0.0.1 js986.cnbole.net
#127.0.0.1 stat1.cnbole.net
#127.0.0.1 easou.irs01.com
#127.0.0.1 g.miaozhen.com
#127.0.0.1 csdnim.qtmojo.com
#127.0.0.1 msn.qtmojo.com
#127.0.0.1 rack.ra.icast.cn
#127.0.0.1 adnet.sohu.com
#127.0.0.1 osupdate.aliyun.com
#127.0.0.1 cf.gdatecube.net
#127.0.0.1 s.gdatecube.net
#127.0.0.1 ark.letv-epg.wasu.tv
#127.0.0.1 ark.vmoters.com
#127.0.0.1 dc.letv-epg.wasu.tv
#127.0.0.1 aiseet.atianqi.com
#127.0.0.1 qchannel01.cn
vokins commented 7 years ago

第一个没看明白。具体是什么意思?ipv6解析吗?

这个hosts是我之前维护的hostsx的数据,比较旧了,很多域名已不存在, 比如127.0.0.1 spcode.baidu.com kkpgv.xunlei.com等 再比如:127.0.0.1 .p2v.tudou.com这种数据在hosts里面是无法生效的。 https://github.com/koala0529/adhost/blob/master/adhosts 现在暂时不接受任何其他国外hosts数据和我之前维护过的所有旧数据的合并请求。 因为yhosts的设计原则与之前的hostsx,simpleu都是不相同的。 我们现在的数据加入,必须要有具体的重现页面,环境,网络分析,截图。

对于那个conf我想问一下 比如:代码里面的

local-data: "img.ujian.cc A 127.0.0.1" local-data: "api.ujian.cc A 127.0.0.1" local-data: "www.ujian.cc A 127.0.0.1"

可否写成: local-data: "*.ujian.cc A 127.0.0.1" 或者写成: local-data: "ujian.cc A 127.0.0.1"

这样呢? 我现在针对ipad设备使用的是这个数据 https://vokins.github.io/ad.js/ad.js

vokins commented 7 years ago

@CNMan 您好,我查询了一下,http://tool.chinaz.com/pagestatus/?url=practivate.adobe.com 这个域名显示确实不存在了,请问有没有比较好的方式可以批量检查,方便以后清理数据。

有些域名类似于

127.0.0.1 js1116.cnbole.net

127.0.0.1 easou.irs01.com

127.0.0.1 g.miaozhen.com

127.0.0.1 csdnim.qtmojo.com

可能此域名不存在了,但是这个广告联盟还会出现新的广告域名, 比如cnbole.net下有很多网站wap版本的弹窗广告会比较多。 irs01.com miaozhen.com qtmojo.com 都是比较常见的广告联盟。

如果可以这样简写 local-zone: "ujian.cc" deny 感觉效率会高很多,文件也会小很多。

我看您的dnscrypt-blacklist-domains.txt数据比较庞大大概2m多。比如像208.185.87.134.liveadvert.com这样的可以用丢弃域名和子域名的方式来简化吗? 还有个问题,比如像这个域名下liveadvert.com除了www.liveadvert.com这个域名我想正常访问,其他我都不想正常访问,有没有解决的好办法吗?

vokins commented 7 years ago

@CNMan 还有,像类似于这样的域名

127.0.0.1 hndnserror8.wo.com.cn

127.0.0.1 jldnserror6.wo.com.cn

127.0.0.1 jldnserror7.wo.com.cn

127.0.0.1 jldnserror8.wo.com.cn

127.0.0.1 lndnserror8.wo.com.cn

127.0.0.1 bjdnserror8.wo.com.cn

127.0.0.1 hbdnserror8.wo.com.cn

这些域名是联通网络劫持无法访问域名的时候自动跳转过去的域名, 是不是只有联通用户的网络才可以检测到?还是不管任何运营商都可以检测到?

类似这样的域名的意义是什么呢? ad.doubleclick.net.43700.9335.302br.net ad.doubleclick.net.43703.9335.302br.net 如果您访问的话他又会跳转到: http://static.adsafeprotected.com/skeleton.gif 那么是否意味着我只要屏蔽static.adsafeprotected.com此域名就可以呢?

vokins commented 7 years ago

我还发现一个问题,比如img.tongji.linezing.com这个域名返回状态是200,说明域名时存在的,这样的话就不能从hosts里面删除了。

有没有win或者mac下的脚本可以批量检测? 要求是域名不存在的才可以删除域名。 比如:a.7baike.com http://tool.chinaz.com/pagestatus/?url=a.7baike.com 查询结果为:域名未解析,或服务器没有响应!则可以删除。 返回任意状态值比如2xx,3xx,4xx均不可删除。

vokins commented 7 years ago

今天我再次访问 http://tool.chinaz.com/pagestatus/?url=img.tongji.linezing.com 又提示 域名未解析,或服务器没有响应!了。 有没有什么脚本可以批量获取状态呢?

vokins commented 7 years ago

这个脚本我配置了半天依旧无法运行。 能否帮忙跑一下,看看那些域名已经不存在了呢?谢谢

vokins commented 7 years ago

http://pan.baidu.com/s/1nva4N6x

CNMan commented 7 years ago

现在可以用https://github.com/missdeer/blocklist/blob/master/update.go 这个golang脚本检测 编译了可用于win 64位的exe update.zip

cd /d d:\download
update.exe

再从屏显log里找出non-exist就行了

2017/04/11 13:54:52 google dns reports as non-exist: a.wlmnt.com
2017/04/11 13:54:52 google dns reports as non-exist: a.ydlnt.com
2017/04/11 13:54:53 google dns reports as non-exist: pagead2.lxms2004.com
2017/04/11 13:54:53 google dns reports as non-exist: img.nsywb.com
2017/04/11 13:54:54 google dns reports as non-exist: a.tnlhy.com
2017/04/11 13:54:55 google dns reports as non-exist: down.bossbas.com
2017/04/11 13:54:56 google dns reports as non-exist: publish.ad.youth.cn
2017/04/11 13:54:57 google dns reports as non-exist: a.tnlfk.com
2017/04/11 13:54:59 google dns reports as non-exist: jake.80s.tw
2017/04/11 13:54:59 google dns reports as non-exist: a.xhnlf.com
2017/04/11 13:54:59 google dns reports as non-exist: aa-oo.top06.cn
2017/04/11 13:54:59 google dns reports as non-exist: 5.nord-raum.com
2017/04/11 13:55:02 google dns reports as non-exist: i2.wljnj.com
2017/04/11 13:55:02 google dns reports as non-exist: a.wlfng.com
2017/04/11 13:55:05 google dns reports as non-exist: j.qihaoqu.com
2017/04/11 13:55:08 google dns reports as non-exist: a.xlwnx.com
2017/04/11 13:55:08 google dns reports as non-exist: a.wlcnd.com
2017/04/11 13:55:10 google dns reports as non-exist: www.lxms2004.com
2017/04/11 13:55:12 google dns reports as non-exist: a.tnlby.com
2017/04/11 13:55:13 google dns reports as non-exist: a.tnlbd.com
2017/04/11 13:55:13 google dns reports as non-exist: i.qihaoqu.com
2017/04/11 13:55:14 google dns reports as non-exist: a.xflnk.com
2017/04/11 13:55:14 google dns reports as non-exist: activate.lumion.net
2017/04/11 13:55:14 google dns reports as non-exist: link.yida558.com
2017/04/11 13:55:14 google dns reports as non-exist: img.21kas.com
2017/04/11 13:55:15 google dns reports as non-exist: m1.dx1200.com
2017/04/11 13:55:15 google dns reports as non-exist: qaz.afjlb.com
2017/04/11 13:55:15 google dns reports as non-exist: wsx.4ggww.com
2017/04/11 13:55:15 google dns reports as non-exist: t.qihaoqu.com
2017/04/11 13:55:16 google dns reports as non-exist: ig.wlknb.com
2017/04/11 13:55:17 google dns reports as non-exist: qd.80s.tw
2017/04/11 13:55:17 google dns reports as non-exist: code.wantaico.com
2017/04/11 13:55:20 google dns reports as non-exist: a.tnldt.com
vokins commented 7 years ago

我在mac下运行成功了,谢谢,原来要一行一行的运行就可以。现在处理了一大部分数据了,非常感谢

vokins commented 6 years ago

2018/09/30 09:38:37 google dns reports as non-exist: m78n.me 感谢,数据已处理。