AdguardTeam / AdguardFilters

AdGuard Content Blocking Filters
https://adguard.com/
GNU General Public License v3.0
2.93k stars 613 forks source link

Chinese ad servers for the mobile ads filters #6703

Closed ameshkov closed 6 years ago

ameshkov commented 6 years ago

@AdamWr please take a look at this comment: https://github.com/AdguardTeam/AdguardForiOS/issues/494#issuecomment-327797523

There are some ad servers listed there, which fit perfectly the mobile ads filter.

AdamWr commented 6 years ago
  1. api.tinyhoneybee.com I'm not sure if I should block this totally. If I understand correctly it is some kind of framework and can be used not only for ads. There are these rules in mobile filter for this domain: ||api.tinyhoneybee.com/api/getADConfig ||api.tinyhoneybee.com/api/getADResource

  2. ad.ettoday.net Added to English filter: ||ad.ettoday.net^ ettoday.net##div[id^="abp"] ettoday.net##.etmall-shopping-guide

  3. ad1.udn.com Added to English filter: ||ad1.udn.com^ ||ad2.udn.com^ ||ad3.udn.com^ ||ad4.udn.com^ ||ad5.udn.com^ ||udn.com.tw/upf/*_banners/BD/$third-party udn.com##.mag_topic > a[href^="https://goo.gl/"][target="_blank"] > img[src*="_banners/BD/"]

  4. graph.facebook.com From what I know, I shouldn't block this, because it will break Facebook auth.

I added these rules to English filter, because these ad servers are also used in desktop version of websites, so I think it is better place for them.

lancelot-moon commented 6 years ago

@AdamWr @ameshkov

Yes, afterwards I found that facebook sometimes is unable to normally load pages if blocking graph.facebook.com. But sometimes facebook's content is normally loaded if blocking graph.facebook.com. There is a risk.

Can you find other detailed ad domains in Erudite dictionary app, except for graph.facebook.com? https://goo.gl/RnEcT8

Screenshots: ![image](http://i.imgur.com/1zCjCMI.jpg) ![image](http://i.imgur.com/jJtNAHO.png)

I report the results here after I put them in my blacklist and enable Adguard DNS (No Simplified domain names filter in my blacklist).
Screenshot: ![image](http://i.imgur.com/oaAhAaG.png)

1. In IQQI, it still has ads after blocking api.tinyhoneybee.com/api/getADConfig and api.tinyhoneybee.com/api/getADResource. You need to block the whole api.tinyhoneybee.com. Otherwise, we need to find if there are other ads domains.
Screenshots: ![image](http://i.imgur.com/KlmCUES.png) ![image](http://i.imgur.com/tPsWJX2.png)

2. Via facebook app, read news on ETNEWS新聞雲 fan page. The ad blocking is almost perfect. But I see ads when I read the under news.
Screenshot: ![image](http://i.imgur.com/tYPHMDC.jpg)

https://www.ettoday.net/dalemon/post/29790?from=fb_et_news If you read the news by a browser, no ad. But there are ads under this news by facebook app. In facebook app, the 1st ad is blocked. A space on this website.
Screenshot: ![image](http://i.imgur.com/PIH4LG6.png)

But the 2nd and 3rd ad aren't blocked. 2nd ad:
Screenshot: ![image](http://i.imgur.com/XJ5Z1JA.jpg)

3rd ad:
Screenshot: ![image](http://i.imgur.com/tohejgt.jpg)

3. Via facebook app, read news on udn.com 聯合新聞網 fan page.
Screenshot: ![image](http://i.imgur.com/YvuT1tZ.png)

May you solve the half-transparent pop-ups? A net friend recommended Adguard to hide page elements. Is the method available? Finally, hope that Adguard establishes a Chinese filter. Please put a Chinese filter on the website. https://kb.adguard.com/en/general/adguard-ad-filters Maybe the Chinese filter can be based on EasyList China + EasyList filter. And then you can put the above Chinese ad domains in Chinese filter, not in English filter. Thanks.
ameshkov commented 6 years ago

You need to block the whole api.tinyhoneybee.com.

This is okay to have this rule in the user filter, but we cannot afford to block it ourselves -- this domain is not just for loading ads, so blocking it will lead to breaking other apps/websites.

Otherwise, we need to find if there are other ads domains.

@AdamWr could you plz look at that app and see what ads are loaded after this API request?

But I see ads when I read the under news. In facebook app, the 1st ad is blocked. A space on this website. May you solve the half-transparent pop-ups?

This is actually the same issue -- AG cannot use "cosmetic" rules for in-app ad blocking (and opening a website inside of the FB app is "in-app"). That's why the ads are blocked partly in this case.

A net friend recommended Adguard to hide page elements.

AG is capable of doing it in Safari only.

lancelot-moon commented 6 years ago

For Erudite dictionary app, if we also can not block graph.facebook.com, may you find other ads domains? Thanks.

ameshkov commented 6 years ago

For Erudite dictionary app, if we also can not block graph.facebook.com, may you find other ads domains?

I've seen this before in Android apps -- FB loads ads from their own domain.

There's a hope that we'll find something for the app that use api.tinyhoneybee.com, but I doubt we'll crack this one.

lancelot-moon commented 6 years ago

Via facebook app, read news on ETNEWS新聞雲 fan page. 1st, 2nd and 3rd ad are on the same page. The same news. But the 1st ad is blocked. I think 2nd and 3rd ads aren't blocked because those ad domains are not in your filter.

Via facebook app, read news on udn.com 聯合新聞網 fan page. I don't understand. Adguard can remove ad pop-ups on ETNEWS via facebook app. Why can it not remove half-transparent pop-ups on udn.com 聯合新聞網 fan page via facebook app?

AdamWr commented 6 years ago

Ads in Erudite dictionary app are blocked on my iPad when I have AdGuard DNS enabled. I can reproduce these ads only when I disable AdGuard DNS, it seems that these are Google ads.

Screenshot AdGuard with enabled DNS ![erudite_dns](https://user-images.githubusercontent.com/29142494/30271139-7637310c-96ef-11e7-9371-b2da365421b3.png)
Screenshot AdGuard without DNS ![erudite_no_dns](https://user-images.githubusercontent.com/29142494/30271147-800b1982-96ef-11e7-821d-f9291b671c5e.png)


In IQQI, it still has ads [...] You need to block the whole api.tinyhoneybee.com. Otherwise, we need to find if there are other ads domains.

Yes, and unfortunately I didn't found other way to block these ads than blocking whole api.tinyhoneybee.com. There was request from i.api.ad-locus.com, but blocking it doesn't help. I adde this to Mobile filter - https://adlocus.com/main/index.php/en/advertiser/ad-inappbanner

Via facebook app, read news on ETNEWS新聞雲 fan page. The ad blocking is almost perfect. But I see ads when I read the under news.

I can't find these ads on my iPad, maybe because I have this website in desktop mode, not mobile. But I also tried to find them in mobile view on Windows without any adblocker and also can't find these ads from your screenshot or similar.

Adguard can remove ad pop-ups on ETNEWS via facebook app. Why can it not remove half-transparent pop-ups on udn.com 聯合新聞網 fan page via facebook app?

Because they are provided in different way. On ETNEWS this popup can be blocked by blocking ad server. On udn.com this popup is provided by this script: https://s.udn.com.tw/static/js/dist/bundle.mobile.js?2017081701 so to block this in Facebook app we would have to block s.udn.com.tw what would break the page. Only proper way to hide this, it is using "cosmetic" rules which are impossible to use in-app.

lancelot-moon commented 6 years ago

Why can you block ads in Erudite Dictionary app with AG DNS? It's strange! I didn't use wifi. I changed to use cellular network. Even Adguard for Android is unable to block ads in Erudite app. https://imgur.com/a/bk8yk Do AG DNS servers include Russian filter? Maybe you have a good Russian filter which has those Russian ad domains even if not blocking graph.facebook.com. But AG has no Chinese filter.

For ads in ETNEWS新聞雲 fan page, you run facebook app and then search "ETNEWS新聞雲", not a browser app. You need to find the news about the dogs in ETNEWS新聞雲 fan page via facebook app. The location of this news: You can see the photo about the white dogs.

Screenshot: ![image](http://i.imgur.com/tYPHMDC.jpg)

When opening this news, you can see the 1st ad is blocked. There is a space. But there are 2nd and 3rd ad which are not blocked in the below. I guess that 2nd and 3rd ad domains aren't in your filter.
AdamWr commented 6 years ago

I did it, as I said I checked it on iPad (I don't have iPhone) and I have this website with desktop version. What should be blocked?

Screenshots ![1](https://user-images.githubusercontent.com/29142494/30276703-f32b7ee6-9705-11e7-9a6c-d616f01296a5.png) ![2](https://user-images.githubusercontent.com/29142494/30276698-f0edaee2-9705-11e7-9daf-cb7cf1c25283.png) ![3](https://user-images.githubusercontent.com/29142494/30276696-f0e742a0-9705-11e7-8dd9-daa697719d2b.png) ![4](https://user-images.githubusercontent.com/29142494/30276699-f1025216-9705-11e7-88ec-0c085c52bd00.png) ![5](https://user-images.githubusercontent.com/29142494/30276697-f0eaaf4e-9705-11e7-8cd6-6ca032c4c4af.png) ![6](https://user-images.githubusercontent.com/29142494/30276700-f13dcb2a-9705-11e7-9cc1-c971357e58fd.png) ![7](https://user-images.githubusercontent.com/29142494/30276695-f0e27be4-9705-11e7-83f0-ff5593267337.png)


Even Adguard for Android is unable to block ads in Erudite app.

I will check it later on Android, but could you please tell me, do you have HTTPS filtering enabled in AdGuard?

lancelot-moon commented 6 years ago
  1. Unfortunately, you don't have an iphone to try. But you can use an Android phone.

I used my Android phone for opening the same news on ETNEWS via facebook app. AG for Android only blocks 1st ad and doesn't block 2nd ~ 6th ad.

You can directly search "https://www.ettoday.net/dalemon/post/29790?from=fb_et_news" on facebook app. Don't need to go to ETNEWS新聞雲 fan page. It's faster to find the news.

Screenshot: ![image](https://i.imgur.com/lK2M1tU.jpg)

In the below, the 1st ad is blocked. A space on this website.
Screenshot: ![image](https://i.imgur.com/bKUdVVi.jpg)

The 2nd ad isn't blocked.
Screenshot: ![Iimage](https://i.imgur.com/ghta45q.jpg)

The 3rd ad isn't blocked.
Screenshot: ![image](https://i.imgur.com/6LSpxIA.jpg)

The 4th ad isn't blocked.
Screenshot: ![image](http://i.imgur.com/xV2e6RV.jpg)

The 5th and 6th ad aren't blocked.
Screenshot: ![image](https://i.imgur.com/2mUDASs.jpg)

I didn't notice yesterday. There are ads in other 5 places on this website, except for the blocked 1st ad. The conclusion is no Chinese filter in AG, which results in unblocked Chinese ads. 2. Yes, I already enabled HTTPS filtering in AG for Android version.
Screenshot: ![image](https://i.imgur.com/behbcXc.jpg)

The ads in Erudite app are not blocked. I guess graph.facebook.com loads different ad domains in different countries. Your Russian filter is better, but AG has no Chinese filter. So Chinese ads appear in Erudite app. I have a thought. In ios, you can use local VPN and real VPN at the same time. You can first use VPN for getting Taiwan ip and then use AG pro with AG DNS. Or change your ios language setting from Russian to Traditional Chinese. I don't sure what language of ads you see is according to your system's language setting or country location. Check if there are Chinese ads in Erudite app.
ameshkov commented 6 years ago

@lancelot-moon Adam is not russian:)

ameshkov commented 6 years ago

@lancelot-moon there're some Chinese-specific filters available in AG. None of them helps?

lancelot-moon commented 6 years ago
  1. I already enabled EasyList China filter in AG for Android. But it's useless.

  2. In AG, I can see EasyList China filter. Is that filter simplified by AG or not? May you put EasyList China filter on your filters website? Or is it the same with what I download from the under website? https://filterlists.com/ I ever copied the content of EasyList China + EasyList into my blacklist. But AG pro for ios became abnormal, which caused the disconnection of internet when I enabled DNS status. Anyway, you put a Japanese filter on your website. https://kb.adguard.com/en/general/adguard-ad-filters Why can you not put a Chinese filter there?

  3. Maybe those ad domains are known for your language's filter, but unknown for Chinese filter. Or Adam should read that news in mobile website mode, not a desktop mode. Perhaps he will find ads. Well, he still doesn't give a feedback. He can use an android phone. Directly search "https://www.ettoday.net/dalemon/post/29790?from=fb_et_news" on facebook app. Don't need to go to ETNEWS新聞雲 fan page. It's faster to find the news.

AdamWr commented 6 years ago

Sorry for the delay.

I checked it on Android phone and I was able to reproduce ads. If I'm not wrong, Facebook use own built in app web browser and it seems that these ads are from Facebook and looks like "sponsored posts". Website also looks different than "normal" mobile version of ettoday.net and they probably use something like amp.

At this moment I didn't notice any "ad server/domain" related to this ads which is not blocked and only way I find to block these ads is blocking graph.facebook.com (there will be still empty gray boxes), but is not good, because it breaks Facebook...

Screenshots ![screenshot_2017-09-18-11-26-05](https://user-images.githubusercontent.com/29142494/30539330-34121ce6-9c72-11e7-8e38-0a1b41a78a18.png) ![screenshot_2017-09-18-12-35-15](https://user-images.githubusercontent.com/29142494/30539329-3411c7aa-9c72-11e7-9146-b4ba51fa89b9.png)
lancelot-moon commented 6 years ago

Blocking "graph.facebook.com" is useless for me. Total are 6 sponsored ads. In that news, the 1st ad is originally blocked by AG. The 2nd and 3rd ads are in the article. The 4th~6th ads are below the article.

AdamWr commented 6 years ago

I don't see any way to block these ads. Not only on this website, but also on others which I checked, it is impossible to block some type of ads in web browser built in Facebook app.

InquiM commented 6 years ago

Hey guys. With all these fake apps around, I have to ask. is this yours or a face? https://play.google.com/store/apps/details?id=com.adguard.android.contentblocker Many thanks