jspenguin2017 / uBlockProtector

An anti-adblock defuser for Nano Adblocker and uBlock Origin
GNU General Public License v3.0
744 stars 82 forks source link

Cookies security policy and network filter bypassing (a.k.a. g00 Exploit) by Instart Logic #95

Closed uBlock-user closed 7 years ago

uBlock-user commented 7 years ago

Detects adblocking and starts inserting g00 third party cookies as first party and inserts /g00 content. Upon opening console, it removes any trace of its nefarious activities and trolls the user by changing the URL until the page can no longer be loaded correctly.

The script is inline, you can review the script by opening the source - view-source:www.spin.com

For ref - https://github.com/uBlockOrigin/uAssets/issues/227

Old updates Update from X01X012013: Hi guys, in case you are affected and don't want to break comment section, just open DevTools once and the website will stop showing ads. You might still be tracked in the background, but at least you won't be getting ads. You can do this by pressing `F12` or `Ctrl+Shift+I`. We are still working on this, there are a few solutions that are semi-functional, but the race conditions is stopping us from rolling out a patch. ----- Another update from X01X012013: Here is a summary of what happened until today, March 16th: ~~We found a solution to their anti-console system: https://github.com/X01X012013/AdBlockProtector/issues/95#issuecomment-286934592~~ The websites are starting to act weird, we are not sure if they found a new detection method or they recorded our IPs and just stop serving us the `g00` page. **Update:** They found a new way to detect console: https://github.com/X01X012013/AdBlockProtector/issues/95#issuecomment-287625572 We were able to analyze what's going on in the background, unfortunately, there isn't much interesting stuff to see there. We found a few possible solution against `g00`, including patching `Promise`, `navigator`, or `console`, however, appending the fix to Script does not work due to race condition. We are still working on this. ----- Update from X01X012013: New anti-console detection code: https://github.com/X01X012013/AdBlockProtector/issues/95#issuecomment-287633888

Hi, it's X01X012013 again. Looks like uBO-Extra found a solution. For technical reasons, we couldn't implement the fix in AdBlock Protector yet. If you do not want to use uBO-Extra, you can try this Userscript form @uBlock-user : https://github.com/X01X012013/AdBlockProtector/issues/95#issuecomment-289503216 Another workaround is to open the console with F12 or Ctrl+Shift+I and g00 will stop showing ads.

jspenguin2017 commented 7 years ago
  1. What do you mean by et al.? Can you list all the domains?
  2. There is no way for a website to know if the console is open. It can guess it with window resize, but if you have the console detached and open before loading the page, there is no way for it to detect an open console. Actually, it can... https://bugs.chromium.org/p/chromium/issues/detail?id=672625 (thanks to @ameshkov for the link)
  3. I don't see anything wrong with the website, except that it does refresh once when loading the page.
![image](https://cloud.githubusercontent.com/assets/7283682/23907118/b9b9be24-0895-11e7-81e4-1a5e6777f9a7.png)
uBlock-user commented 7 years ago

What do you mean by et al.? Can you list all the domains?

It means spin.com and the entire group of domains.

There is no way for a website to know if the console is open.

Possible in Chromium/Chrome, not in Firefox, unfortunately they're detecting console being opened. Read - https://github.com/uBlockOrigin/uAssets/issues/227#issuecomment-268023489

I don't see anything wrong with the website, except that it does refresh once when loading the page.

Run the logger and the check the amount of cookies being sent and read the uBO thread I linked for better understanding of the situation.

uBlock-user commented 7 years ago

This is the script - https://gist.githubusercontent.com/uBlock-user/8b732091dd25fa4d6c046a4cbb2dbfed/raw/bfe5325f203ada81324cb58a4406c131e1a3c98c/gistfile1.txt

jspenguin2017 commented 7 years ago

Eh... Yea, I didn't word my response properly. It is possible to guess if the console is open, but if you detach the console and open the console before loading the page, there is no way to detect it. https://bugs.chromium.org/p/chromium/issues/detail?id=672625

jspenguin2017 commented 7 years ago

It's at the beginning of the page too... might have some race conditions...

Can you explain a bit more what do you want me to do with this website? It works fine for me...

uBlock-user commented 7 years ago

Did you read the read the uBO issue I linked ? It's a serious issue and uBO-Extra has been bypassed three times already, this is why I posted here believing maybe you can patch it for good.

Can you explain a bit more what do you want me to do with this website?

I want you to kill that script - https://gist.githubusercontent.com/uBlock-user/8b732091dd25fa4d6c046a4cbb2dbfed/raw/bfe5325f203ada81324cb58a4406c131e1a3c98c/gistfile1.txt It loads inline.

jspenguin2017 commented 7 years ago

That thread is so long, can you summarize it?

ghajini commented 7 years ago

@uBlock-user ,can't reproduce issue on ublock origin. possibly try in new chrome profile with ubo &uboextra

uBlock-user commented 7 years ago

Summary from gorhill - _"Instart Logic's technology used to disguise third-party network requests as first-party network requests, including the writing/reading of third-party cookies as first-party cookies. I consider this to be extremely hostile to users, even those not using a content blocker, as it allows third-party servers to read/write cookies even if a user chose to block 3rd-party cookies through your browser setting.

The company behind the technology understand how hostile its technology is to users, and thus tries to hide what is being done by making it difficult to investigate by detecting whether the browser's developer console is opened, and when it detects it is opened, it ceases completely to make use of the obfuscation mechanism. The developer console-detecting code works only for Chromium-based browsers however, and therefore the obfuscation technology is not used when using Firefox (a different web page is served for Firefox)."_

uBlock-user commented 7 years ago

@ghajini , clear all cache and site data for that domain and try again, I can reproduce it.

jspenguin2017 commented 7 years ago

OK, I see the problem. I'll investigate it.

uBlock-user commented 7 years ago

It detects adblock but won't send you any notice or anything like that or block any images or videos, it reloads the page with /g00 content and fetches 3rd party adserver cookies and sets them as first party basically facilitating tracking by the ad-servers and tracking networks.

jspenguin2017 commented 7 years ago

Notes for devs: http://pastebin.com/8cPJfpt7

ghajini commented 7 years ago

@uBlock-user can't reproduce after clearing cookies/data/cache http://prnt.sc/ejvf09 http://prnt.sc/ejvfso http://prnt.sc/ejvg7j http://prnt.sc/ejvgfk

might be network level filtring/antivirus/other adblocker interfering...try new chrome profile

uBlock-user commented 7 years ago

and it also inserts ads with that script - https://i.gyazo.com/ad0d50a1ed3308b64b45923e9ffd7cef.png

@ghajini, you're still at Chrome 56 ? I'm using Chromium 59, so that could be the case.

network level filtring/antivirus/other adblocker interfering

Nothing of that sort is present here. I don't need a new profile to know when a script is bypassing the extension on my end.

jspenguin2017 commented 7 years ago

I don't see inserted ads neither...

uBlock-user commented 7 years ago

I see it on that article.

ghajini commented 7 years ago

@X01X012013 bro you needuBO extra(for g00 thing) as this thing is better handled in ublock origin.... sorry @uBlock-user and @X01X012013 for bumping the topic edit : i have official stable chrome installedhttp://prnt.sc/ejvpke, i will try chrome 59 portable

jspenguin2017 commented 7 years ago

How about just mock as FireFox when visiting these sites?

uBlock-user commented 7 years ago

Can you do that via the userscript ? Otherwise I have no intension of installing another extension for that purpose.

jspenguin2017 commented 7 years ago

Yes... but I don't think I can do it for the main HTML...

uBlock-user commented 7 years ago

or else I could just kill all the inline-scripts with ||spin.com^$inline-script although it would break videos from playing and thumbnails of images from loading as a consequence.

Also I'm not the only one as it's been reported today on Easylist forum too - https://forums.lanik.us/viewtopic.php?p=114695#p114695

jspenguin2017 commented 7 years ago

That's not really a solution...

uBlock-user commented 7 years ago

Do you have any other ideas in mind ?

jspenguin2017 commented 7 years ago

Like ask Chromium devs to fix detectable console? I don't see how can we debug this thing without console.

Update: We found a way to make console undetectable! https://github.com/X01X012013/AdBlockProtector/issues/95#issuecomment-286934592

uBlock-user commented 7 years ago

That's not the core problem here at all, that script should be killed before successful execution, can you do something about that ?

jspenguin2017 commented 7 years ago

The bootstrapper? I need to find a weak point in the closure in order to break it, if someone found one let me know...

The problem is even if we found one they can easily fix it...

uBlock-user commented 7 years ago

IT makes use of lots of window properties, can we kill it by blocking access to all or one specific properties ?

["stop","open","alert","confirm","prompt","print","requestAnimationFrame","cancelAnimationFrame","requestIdleCallback","cancelIdleCallback","captureEvents","releaseEvents","getComputedStyle","matchMedia","moveTo","moveBy","resizeTo","resizeBy","getSelection","find","getMatchedCSSRules","webkitRequestAnimationFrame","webkitCancelAnimationFrame","btoa","atob","setTimeout","postMessage","clearTimeout","blur","focus","setInterval","close","clearInterval","createImageBitmap","scroll","scrollTo","scrollBy","fetch","webkitRequestFileSystem","webkitResolveLocalFileSystemURL","openDatabase","chrome","console","$","jQuery","_","smg","piCheckIsMobile","piDetectMobile","getCookie","setCookie","deleteCookie","toggleMobileSite","ImgTouchCanvas","wp","scInterrupters","_gaq","CLARITY","Refresher","addURLParam","sharrre_track","campaign_monitor_define","CampaignMonitor","disqus_shortname","jQuery111305645092176691662","$toggleEls","$searchQuery","frames","self","window","parent","opener","top","length","closed","location","document","origin","name","history","locationbar","menubar","personalbar","scrollbars","statusbar","toolbar","status","frameElement","navigator","applicationCache","external","screen","innerWidth","innerHeight","scrollX","pageXOffset","scrollY","pageYOffset","screenX","screenY","outerWidth","outerHeight","devicePixelRatio","clientInformation","screenLeft","screenTop","defaultStatus","defaultstatus","styleMedia","onanimationend","onanimationiteration","onanimationstart","onsearch","ontransitionend","onwebkitanimationend","onwebkitanimationiteration","onwebkitanimationstart","onwebkittransitionend","onwheel","isSecureContext","onabort","onblur","oncancel","oncanplay","oncanplaythrough","onchange","onclick","onclose","oncontextmenu","oncuechange","ondblclick","ondrag","ondragend","ondragenter","ondragleave","ondragover","ondragstart","ondrop","ondurationchange","onemptied","onended","onerror","onfocus","oninput","oninvalid","onkeydown","onkeypress","onkeyup","onload","onloadeddata","onloadedmetadata","onloadstart","onmousedown","onmouseenter","onmouseleave","onmousemove","onmouseout","onmouseover","onmouseup","onmousewheel","onpause","onplay","onplaying","onprogress","onratechange","onreset","onresize","onscroll","onseeked","onseeking","onselect","onshow","onstalled","onsubmit","onsuspend","ontimeupdate","ontoggle","onvolumechange","onwaiting","onbeforeunload","onhashchange","onlanguagechange","onmessage","onoffline","ononline","onpagehide","onpageshow","onpopstate","onrejectionhandled","onstorage","onunhandledrejection","onunload","performance","onauxclick","customElements","ongotpointercapture","onlostpointercapture","onpointercancel","onpointerdown","onpointerenter","onpointerleave","onpointermove","onpointerout","onpointerover","onpointerup","crypto","ondevicemotion","ondeviceorientation","ondeviceorientationabsolute","indexedDB","webkitStorageInfo","sessionStorage","localStorage","caches","speechSynthesis"]
jspenguin2017 commented 7 years ago

We can't block access to all window properties, that would break everything. We can try specific ones... I know blocking I10C_70_19419036859033012 would work, but probably not for long.

uBlock-user commented 7 years ago

What about chrome ? That property is probably to detect Chrome.

jspenguin2017 commented 7 years ago

I think it uses navigator.userAgent to detect browser...

var ffUA = "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0";
Object.defineProperty(window.navigator, "userAgent", {get: function() { return ffUA; }});
jspenguin2017 commented 7 years ago

OK, ticking preserve log I can see what it is trying to do...

![image](https://cloud.githubusercontent.com/assets/7283682/23914393/71d76dfc-08ab-11e7-8cc2-7b0d097b6353.png)
uBlock-user commented 7 years ago

came up with this

if (a.domCmp(["spin.com"])) {
    a.readOnly("navigator", function () { });
}

Needs to be tested throughly though..

It's working on my end.

uBlock-user commented 7 years ago

List of websites infested with this g00 adware -

``` baltimoresun.com, boston.com, capitalgazette.com, carrollcountytimes.com, celebuzz.com, celebslam.com, chicagotribune.com, computershopper.com, courant.com, dailypress.com, deathandtaxesmag.com, extremetech.com, gamerevolution.com, geek.com, gofugyourself.com, hearthhead.com, infinitiev.com, lolking.net, mcall.com, mmo-champion.com, nasdaq.com, orlandosentinel.com, pcmag.com, ranker.com, sandiegouniontribune.com, saveur.com, sherdog.com, spin.com, sporcle.com, stereogum.com, sun-sentinel.com, thefrisky.com, thesuperficial.com, timeanddate.com, tmn.today, twincities.com, vancouversun.com, vibe.com, weather.com, wowhead.com, legacy.com ```
uBlock-user commented 7 years ago

As per the behavior I have noticed, they first detect the browser type by fetching the user agent and since majority of the users use Chromium/Chrome clones, then they follow up with anti adblock check and start loading their /g00 upon successful detection. So by killing the window.navigator property, it fails and it's not a random variable which can be changed at will either!

jspenguin2017 commented 7 years ago

Removing navigator seems to have a pretty big race condition... And breaks comment section. We could lock INSTART_TARGET_NAME, but there's an obvious fix to this solution...

uBlock-user commented 7 years ago

What kind of race condition ? Can you elaborate ? and what's the obvious fix ?

jspenguin2017 commented 7 years ago

The in-line script is very early in their code, so there is no guarantee that our code can go in fast enough to nuke the property. This is one of the problem with Chrome, it's too fast.

The obvious fix is rename INSTART_TARGET_NAME by adding some random stuff at the end. Nuking navigator is harder to bypass, but it breaks comment section.

And is this the company behind this exploit? https://www.instartlogic.com/solutions/appshield/ad-integrity

uBlock-user commented 7 years ago

Works fine on my end, It always executes successfully on my end. For the comment section, that might be a little sacrifice as I believe only registered users get to post a comment and I don't think most users bother to register, it's a newspaper website, you can just browse, open an article and read it, there's no restriction of any kind for guests.

Secondly, renaming INSTART_TARGET_NAME would stick ? Can they change it and troll us again ? Yes, that's the company behind this horrendous exploit.

jspenguin2017 commented 7 years ago

No, the comment section is powered by Disqus. Since their documentation is not open to public, I can't really tell if INSTART_TARGET_NAME is part of their API. If it is, it'll be a huge pain for them to change since they will need to inform every developer using their service, and we can change it again with a few keystrokes. I can't really test on my end since the page works without any rules. You may want to use uBO to lock INSTART_TARGET_NAME or navigator since I haven't put it in Script yet, and when I update Script, your changes will be reversed. (abort-on-property-read or -write should do the trick)

uBlock-user commented 7 years ago

Locking INSTART_TARGET_NAME has no effect in uBO. Locking navigator in uBO breaks more than comment section. Nuking it in the script is a preferable option in my opinion.

Also have you tried in loading it in chrome ? I'm sure you will able to reproduce there if not in Opera. Also so what're you going to use for the fix?

jspenguin2017 commented 7 years ago

Well, looks like you are right. The page doesn't work in Chrome... Works on Opera though... I see ads when in Chrome and the page doesn't respond... I'm not sure what to do... I don't really want to break comment section, which is quite important for a news page...

![image](https://cloud.githubusercontent.com/assets/7283682/23928115/93fa94b2-08e3-11e7-959b-b37dd2b403b9.png)

 

Maybe mock as Opera as a temporary solution?

jspenguin2017 commented 7 years ago

ImpressiveDisgusting... These guys were able to bypass uBO... This is a big problem. I need to reverse engineer their exploit when I have time. They are using example.com for something, that is not in the scope of intended use for that domain, maybe we can report this to IANA...

And anything related to this issue on those adblock forums?

jspenguin2017 commented 7 years ago

WOW it looks like it can actually detect DevTools in Chrome!

lukemulks commented 7 years ago

Hi @X01X012013 - first off, many thanks for all you do. I am on the team at Brave, and have been working on reversing this on Boston.com.

It detects Dev Tools It detects Wireshark It detects Charles Proxy

I have been tracking in this issue if this helps: https://github.com/brave/browser-laptop/issues/7363

I have merged a commit to block the RPC tracking at the window level (one C request, one M request) in our blocking engine, tracked from this issue: https://github.com/brave/adblock-lists/issues/17

I am likely going to expand the issue 17 above to apply globally, because these folks are slimy as hell.

I have one Wireshark capture where I caught the refresh, which should help yield some info.

Basically, IL acts as a CDN, and has a separate ad layer for blocking that hooks into the publisher Doubleclick server. The GPT slot definitions, viewport breakpoint mappings, etc are called from a 1p domain string that is prepended with some obfuscation.

Anyway, appreciate what you do here. Hope we can ally on many of these cases, feel free to drop a line whenever. (I spent 5yrs in ad ops and product integration prior to joining Brave, so I know where some bodies are buried) ;-)

lukemulks commented 7 years ago

Coincidentally, the Easylist appears to have been updated with a ton of exceptions in the specific blocking section of the primary Easylist file, and a ton of them look fast and loose and are listed under IL, which I couldn't figure out until I saw InstartLogic..(!IL)...Damn, I hope this isn't wha...

Tracking that here, as part of a larger Easylist audit - fourth post from the top: https://github.com/brave/adblock-lists/issues/14 I am going to cross check some of the !IL domains and see if there are any patterns. Will let u know if anything interesting comes from it.

uBlock-user commented 7 years ago

We don't have much of a choice here, they have bypassed uBO-Extra three times already as of now, so defensive solutions against them are pointless, we have to go for the offensive one !!

maybe we can report this to IANA...

and how would that stop them ? They don't give a fuck about regulations of IANA or any other committee as such, neither does the rest of anti adblocking websites on the Internet.

uBlock-user commented 7 years ago

The only thing the Easylist forums have is this - http://forums.lanik.us/viewtopic.php?f=62&t=34244&sid=c02c4740fdfedddda1a6bb2147fbda46&p=108538&view=show#p108538

szymon1118 commented 7 years ago

Probably I didn't bring anything which may fix this issue but I found something which may be helpful: https://gist.github.com/kakumar/d86a8e19548f412fbce08d9458f5c710

jspenguin2017 commented 7 years ago

@lukemulks Thanks, we need as many people as possible to reverse their logic. I hope we'll soon find a solution.