NanoAdblocker / NanoFilters

Script snippets and filters
GNU General Public License v3.0
125 stars 28 forks source link

[Anti-adblock] derstandard.at #45

Closed ghost closed 6 years ago

ghost commented 6 years ago

URLs where this issue occurs

https://derstandard.at/Web

Screenshots

screen shot 2018-03-14 at 13 31 42

Screenshot of console

screen shot 2018-03-14 at 13 32 53

Describe the issue

Anti adblock cannot be circumvented. It was working before, when refreshing the page 2-3 times. (maybe the Chrome 65 update is to blame or derstandard.at changed their blocking system)

Anything else you believe to be useful

In case that it works, wait some time as the implemented anti adblock does not block your access to the website at first.

Environment

Your filter lists

My filters​​​​​ 0 used out of 0​​​​ Nano Defender Integration​​​​​ 2,719 used out of 2,757​​​​ Nano filters​​​​​ 59 used out of 59​​​​ Nano filters – Timer bypassing (sometimes works)​​​​​ 0 used out of 0​​​​ Nano filters – Whitelist​​​​​ 0 used out of 0​​​​ uBlock filters​​​​​ 4,401 used out of 4,401​​​​ uBlock filters – Badware risks​​​​​ 8 used out of 8​​​​ uBlock filters – Privacy​​​​​ 65 used out of 65​​​​ uBlock filters – Resource abuse​​​​​ 216 used out of 218​​​​ uBlock filters – Unbreak​​​​​ 321 used out of 321​​​​

Adblock Warning Removal List​​​​​ 393 used out of 409​​​​ EasyList​​​​​ 81,571 used out of 81,894​​​​

EasyPrivacy​​​​​ 14,347 used out of 14,516​​​​

Malware Domain List​​​​​ 1,128 used out of 1,143​​​​ Malware domains​​​​​ 17,644 used out of 17,644​​​​

Peter Lowe’s Ad and tracking server list​​​​​ 2,596 used out of 2,600​​​​

DEU: EasyList Germany​​​​​ 9,225 used out of 9,245​​​​

jspenguin2017 commented 6 years ago

@okiehsch Kind of hard to reproduce, need to refresh 6~7 times and wait a bit.

okiehsch commented 6 years ago

This is quite tricky, it is related to https://github.com/uBlockOrigin/uAssets/issues/1334

||imagesrv.adition.com/js/srp.js$script,redirect=noopjs,domain=derstandard.at
@@||derstandard.at/MetaAdServer/MAS.aspx$xmlhttprequest,first-party
@@||derstandard.at/s/*$image,first-party

fixed that issue, but the redirection does not work anymore.

@@||imagesrv.adition.com/js/srp.js$script,domain=derstandard.at fixes the issue on my end, you have to delete the existing cookies if you already get the anti-adblock warning.

Blocking cookies outright also fixes the issue.

I am open for any other possible solutions.

jspenguin2017 commented 6 years ago

image Seems like the GUID is responsible for the block screen. I'm worried that if I change it login will break... I guess blocking cookies is no-go.

jspenguin2017 commented 6 years ago

What's your concern about whitelisting that script?

okiehsch commented 6 years ago

Yes, fixing without breaking the login is probably not possible, unless you allow the script.

I have no particular concern as every ad related stuff pulled through that script is blocked, I was just curious if there was a better way, i.e. without allowing a 3rd-party script, to fix the issue,

okiehsch commented 6 years ago

https://github.com/uBlockOrigin/uAssets/commit/23aa500afcd91f8272378cc9d4796326bd1c23d5

ghost commented 6 years ago

It worked for a while, and now it starts blocking again. Here is a screenshot of the console:

screen shot 2018-03-15 at 20 27 20

URLs that are blocked by the client are: https://at.staticfiles.at/sites/mainweb/css/adv.css?v=528 https://derstandard.nuggad.net/rc?nuggn=1072017547&nuggsid=1660588788&nuggtg=community%2Fautorisierung%2Fpurschranke&nuggrid=https%3A%2F%2Fderstandard.at%2F https://ad1.adfarm1.adition.com/s?t=iqGLuSZ&v=1&w=278694888&a=1&b=11&f=0&o=5&r=11&p=https(1)purwallhit(1)lad(0)artexkl(0&s=3213868

Concerning the login, it would be ok if the login is blocked with your method. The login is just there to leave comments on articles. One can do that through incognito mode (with the extension disabled) or within another browser.

jspenguin2017 commented 6 years ago

We can't globally apply a solution that breaks login.

okiehsch commented 6 years ago

I tried to reproduce using Chrome, I got the anti adblock message one time after I started a new browser session, I then deleted the existing cookies and I now can't reproduce even if I again start a new browser session.

With Firefox I can't reproduce.

ghost commented 6 years ago

Deleting the cookies works on Chrome for a certain time. After a time it starts blocking again (on Chrome at least).

jspenguin2017 commented 6 years ago

Yea, they seem to have changed something.

okiehsch commented 6 years ago

https://github.com/uBlockOrigin/uAssets/commit/02605791540990f758f81c14c6fd4dfac0971ff2

ghost commented 6 years ago

I am sorry to reopen the ticket, but the issue is not fixed on my end. I have purged all caches and updated the filter lists again (adblocker v1.0.0.37), as well as having deleted the website's cookies. The paywall reappears after a while. (The only effective method right now is blocking all cookies for that website within the chrome settings) screen shot 2018-03-22 at 20 55 03

jspenguin2017 commented 6 years ago

Huh, they might be watching us...

okiehsch commented 6 years ago

@jspenguin2017 so you can reproduce?

I can't, I have clicked countless articles with Chrome and Firefox without issue. If I disable the filter I added yesterday, I can consistently reproduce after clicking two or three articles.

jspenguin2017 commented 6 years ago

Eh, I can't reproduce anymore... Hum...

jspenguin2017 commented 6 years ago

I thought I updated the filters... I'll put a whitelist in Nano Defender just in case.

ghost commented 6 years ago

I still get this

screen shot 2018-03-24 at 12 39 42

The links that are blocked: https://at.staticfiles.at/sites/mainweb/css/adv.css?v=529 oewa.js -> https://dispatcher.oewabox.at/oewa.js rc -> https://derstandard.nuggad.net/rc?nuggn=1072017547&nuggsid=1660588788&nuggtg=community%2Fautorisierung%2Fpurschranke&nuggrid=https%3A%2F%2Fderstandard.at%2FWeb s -> https://ad1.adfarm1.adition.com/s?t=irgdzGj&v=1&w=1372691478&a=1&b=11&f=0&o=5&r=11&p=https(1)purwallhit(1&s=3213868

And this blocked link changes the t and w values: https://ad1.adfarm1.adition.com/s?t=irgdAO1&v=1&w=73084297&a=1&b=11&f=0&o=5&r=11&p=https(1)purwallhit(1)lad(0)artexkl(0&s=3213868

Maybe there is an issue with updating the nano filter list. 64 of 64, after purging all caches and reloading: https://privatebin.net/?d49193799b3734d7#X499Aj8yUuO3GxQvE1h1C7ZlnJRWUnvvN0e65dyl+P8=

jspenguin2017 commented 6 years ago

From Quick Issue Reporter, reported from Austria.

Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36 Nano Defender 14.10

[anti-adblock] https://derstandard.at/

jspenguin2017 commented 6 years ago

I can't reproduce the problem. Tried over 20 pages.

ghost commented 6 years ago

I am sorry to reopen it again and again, but the problem is still reproducible on my end on multiple machines.

derstandard

No other extensions for Google Chrome Version 66.0.3359.181 than: Nano Adblocker 1.0.0.50 Nano Defender 14.37

With filter list: My filters ​​​​​0 used out of 0​​​​ Built-in (9/12) Nano Defender Integration​​​​​ 2,711 used out of 2,755​​​​ Nano filters ​​​​​79 used out of 81​​​​ Nano filters – Timer bypassing (sometimes works)​​​​​ 0 used out of 0​​​​ Nano filters – Whitelist ​​​​​0 used out of 0​​​​ uBlock filters ​​​​​5,802 used out of 5,804​​​​ uBlock filters – Badware risks​​​​​ 11 used out of 11​​​​ uBlock filters – Privacy ​​​​​96 used out of 97​​​​ uBlock filters – Resource abuse​​​​​ 256 used out of 258​​​​ uBlock filters – Unbreak​​​​​ 407 used out of 407​​​​ Ads (2/4) Adblock Warning Removal List ​​​​​419 used out of 435​​​​ EasyList ​​​​​81,955 used out of 82,137​​​​ Privacy (1/3) EasyPrivacy​​​​​ 14,842 used out of 15,024​​​​ Malware domains (2/4) Malware Domain List ​​​​​1,116 used out of 1,131​​​​ Malware domains ​​​​​21,693 used out of 21,693​​​​ Annoyances (0/5) Multipurpose (1/5) Peter Lowe’s Ad and tracking server list ​​​​​2,640 used out of 2,644​​​​ Regions, languages (1/36) DEU: EasyList Germany ​​​​​9,412 used out of 9,433​​​​

jspenguin2017 commented 6 years ago

Try clearing your cookies.

ghost commented 6 years ago

I have been blocking any derstandard.at cookie for a long time now. It worked until today. So there were no cookies. But even after deleting all cookies, the adblocking starts after a while.

Edit: After deleting all cookies it took 3 clicks for the adblocker warning to pop up. Editx2: Now the adblocker warning disappeared. What kind of voodoo is going on here.

jspenguin2017 commented 6 years ago

You aren't blocking cookies in your screenshot.

ghost commented 6 years ago

No, I was not. The method of blocking cookies on that site did not work anymore. However, it seems like it is working now with nano defender. When not visiting the site for some time, it takes three clicks to activate the adblocker warning. But when clicking on another page the adblocker warning is gone. So, thank you. Hopefully closed now forever. Thank you again for your help!

ghost commented 6 years ago

Ok, I think I have figured it out... or maybe I am getting crazy. Windows 10 with Chrome 66 and nano defender/adblocker works fine. But on macOS 10.13.4 and the same Chrome and nano defender/adblocker version (same filter list) does NOT work and the adblock warning stays. This could be the reason why you cannot reproduce it.

jspenguin2017 commented 6 years ago

Seriously? Well, I can't access that setup so I can't test.

okiehsch commented 6 years ago

@jspenguin2017 do you need to block cookies for it to work?

It works on my end with a default setup, I also can't test with a system running macOS.

ghost commented 6 years ago

Right now, no cookies are blocked. Just nano defender/adblocker is running. It works on Windows but not on macOS. If it is not too much to ask, you could run macOS inside a Virtual Machine.

jspenguin2017 commented 6 years ago

@kajover I thought macOS only run on Apple's proprietary hardware.

@okiehsch It works perfectly fine for me as well, all on default.

ghost commented 6 years ago

@jspenguin2017 It runs in vmware and virtualbox https://www.youtube.com/watch?v=9QNlz9FPGI8 https://www.youtube.com/watch?v=szHn5GunojU https://techsviewer.com/install-macos-high-sierra-vmware-windows/

Just as another idea, alternatively, there are Apple Stores in your country maybe.

jspenguin2017 commented 6 years ago

It runs in vmware and virtualbox

Ah, looks like a lot of work, I might try it later. @okiehsch Do you have access to an Apple device?

there are Apple Stores in your country maybe

Apple's devices are too overpriced, for $1700 I'd rather get a XPS 13 developer edition which has i7 processor instead of a m3. And the touchscreen is also a bonus.

Beside, I heard Apple was sued again for hardware design failure.

okiehsch commented 6 years ago

Do you have access to an Apple device?

Only to mobile devices running iOS.

ghost commented 6 years ago

Bad news, it started popping up again on Windows. (I will personally browse the website in incognito mode only, or regularly delete cookies)

jspenguin2017 commented 6 years ago

Works perfectly fine for me, even in incognito mode. What extensions have you installed? image

ghost commented 6 years ago

I do not have any special extensions... chrome chrome3 extensions settings1 settings2 chrome2

jspenguin2017 commented 6 years ago

I have no idea why it's not working for you... Is there anything on your side that could interfere with network connection (Pi Hole, Adguard native app, etc.)?

ghost commented 6 years ago

No pihole, no special apps, no VPN, no anything. The PC (Windows 10 Pro Version 1803) is my gaming PC, so it is a fairly basic installation with some games. I am also using the MacBook on two locations. First one has a Huawei LTE router and the other one a regular router from the ISP. Edit: Macbook has Google's 8.8.8.8 as DNS.

okiehsch commented 6 years ago

Reading through https://www.lteforum.at/mobilfunk/derstandard-nur-mehr-kostenpflichtig-mit-adblocker.9763 everybody who used uBO had no issues with standard.at

This is the last post that mentions uBlock https://www.lteforum.at/mobilfunk/derstandard-nur-mehr-kostenpflichtig-mit-adblocker.9763/seite-9.html#post-172493

Esokrates commented 6 years ago

@kajover is right. I can reproduce. Until the last few days I blocked cookies on that site, now they force you to have cookies enabled so I removed my cookie blocking policy and started browsing in incognito mode. After clicking through a few articles and waiting a bit I was blocked from further accessing the page.

ghost commented 6 years ago

@okiehsch I have tried ublock origin, and it does not work. Is it possible that they block Austrian IPs? Could you try through a VPN? I really cannot understand why it works for you guys. I have deleted all extensions and reinstalled nanodefender/adblocker.. Takes 30-40 clicks to get the adblock wall. @Esokrates Yep, I did the same before.. just blocking all cookies on that site. But this workaround does not work anymore.

Esokrates commented 6 years ago

It's kinda hard to reproduce though. Happened to me in one incognito session pretty instantly. Closed browser tried again and it took some time to trigger it again. So testing with only a very few click is not enough.

okiehsch commented 6 years ago

I did not mean to say that it should work with uBlock, as a rule of thumb ,if it works with Nano it works with uBlock and vice versa. "Takes 30-40 clicks to get the adblock wall." Well that does really sound strange, which anti-adblock script would only fire after the 30th click, that defeats it's purpose.

okiehsch commented 6 years ago

Is it possible that they block Austrian IPs?

I get blocked if I disable uBO-filters and I do not have an austrian IP at the moment.

I have now clicked atleast 50 articles and I still can't reproduce, there is really nothing more I can do, sorry.

ghost commented 6 years ago

@okiehsch I know, I just tried it anyway. Maybe they want you to have some kind of demo period where you can read and decide if you want to pay for it. But after clicking around the website, as I said around 30-40 clicks, you get a permanent adblock wall. What works are extensions that delete cookies after a while or you manually delete them quite often. Can I help somehow buy running some debugging or logs?

Esokrates commented 6 years ago

@okiehsch Open the page, click around, leave the tab with the page open for a while and return a few minutes later to click a few articles. Triggers for me.

okiehsch commented 6 years ago

Their adblocking script is not trivial, I would have to reproduce myself to find a working fix.

If you really only get blocked after 40 articles just delete cookies regularly, it is the best advice I can give you.

okiehsch commented 6 years ago

@Esokrates the tab of https://github.com/NanoAdblocker/NanoFilters/issues/45#issuecomment-391892923 is still open and that was 10 minutes ago. unbenannt

ghost commented 6 years ago

Here a screenshot of the cookies after deleting them and running the nano extensions, if that helps. cookies

okiehsch commented 6 years ago

As you can see from my screenshot I only run uBO, now I have said all there is to say from my end. I can't reproduce, I can't help.