jspenguin2017 / uBlockProtector

An anti-adblock defuser for Nano Adblocker and uBlock Origin
GNU General Public License v3.0
740 stars 83 forks source link

viasatsport.fi (iframe of is.fi) #88

Closed arttuberg closed 7 years ago

arttuberg commented 7 years ago

The site shows blurry version of videos if adblock is detected.
I added my own whitelisting filters on uBlock origin so I can see the videos, but only with forced commercials before the video. When I added AdBlockProtector, it started again complaining about adblock and showed blurred videos only, unless I disable ABProtector.

I'm not at all skilled with writing filters or analyzing what could help, is there a solution someone could help me with? I'm using Firefox 52.0 atm.

Example: http://www.is.fi/mestarienliiga/art-2000005119802.html

jspenguin2017 commented 7 years ago

I'll have a look.

Another test link: http://www.viasport.fi/urheilu/j%C3%A4%C3%A4kiekko/uutiset/kooste-nashville-pittsburgh-0-2-pittsburgh-mestariksi-voitoin-4-2

uBlock-user commented 7 years ago

    (function() {
        // Load detector on a sample of page loads
        // Ratio should be a number between 0 and 1
        //     - 0 means 'never load'
        //     - 0.333 means 'load on every three page loads'
        //     - 1 means 'always load'
        var sabdetect_sample_ratio = 10/100;

        window.Sabdetect_load = Math.random() < sabdetect_sample_ratio;

        if (window.Sabdetect_load) {
            var sabd = document.createElement('script');
            sabd.type = 'text/javascript';
            sabd.async = true;
            sabd.src = '//sn.sanoma.fi/js/sabdetect/Sabdetect-is-async.min.js';
            var s = document.getElementsByTagName('script')[0];
            s.parentNode.insertBefore(sabd, s);
        }
    })();
</script>
<script type="text/javascript">
    if (window.Sabdetect_load) {
        window.setTimeout(function() {
            if (typeof window.Sabdetect == 'undefined' && dataLayer) {
                dataLayer.push({'event':'eventinfo',
                    'eventCategory':'Adblock',
                    'eventAction':'Blocked',
                    'eventLabel': dataLayer[0].page.category.categoryid});
            }
        }, 5000);
    }
uBlock-user commented 7 years ago

Killing the setTimeout call should do the job I suppose.

jspenguin2017 commented 7 years ago

Hum... yea. Or lock Sabdetect. I'll try that. Thanks. Update: Actually, let's just lock Sabdetect_load to false. Looks like that code isn't for videos...

jspenguin2017 commented 7 years ago

vjs.getPlayers() returns an interesting object, but doesn't seem to contain URL to the media file... Examining vjs namespace, it looks like AdBlock detection is integrated into VideoJS. The player seems to have detected a generic protector... And seems that the video is Geo Locked.

@szymon1118 Can you have a look please?

szymon1118 commented 7 years ago

@X01X012013 You are right and it's hard for me to grab the url without being able to play a video. I found a method to grab url but it doesn't work because there is an error in API response (I'm posting a link to Python script I had found connected with this site: http://programtalk.com/vs2/python/12272/svtplay-dl/lib/svtplay_dl/service/viasatsport.py/). I tried looking into this script: https://viasport-assets.mtg-api.com/2017030808/prod/3-f2a6e6b66f51f7838b39.js and seems that it contains a method to form an API url: getStreamLinks but as I said before API doesn't work for me.

jspenguin2017 commented 7 years ago

I disabled generic protectors on this iframe for now, your rules should now work properly.

if (a.domCmp(["is.fi", "viasatsport.fi"])) {
    //Issue: https://github.com/X01X012013/AdBlockProtector/issues/88
    a.readOnly("Sabdetect_load", false);
    if (a.domCmp(["viasatsport.fi"], true)) {
        a.config.allowGeneric = false;
    }
}
jspenguin2017 commented 7 years ago

@szymon1118 Thanks a lot. Do you know which country is not Geo Locked? Maybe we can Tunnel Bear over... How do you find all these, you are REALLY good at finding media URL grabbing methods...

Notes for devs: License: http://programtalk.com/vs2/?source=python/12272/svtplay-dl/LICENSE

from __future__ import absolute_import
import re
import json

from svtplay_dl.service import Service, OpenGraphThumbMixin
from svtplay_dl.fetcher.hls import hlsparse
from svtplay_dl.error import ServiceError

class Viasatsport(Service, OpenGraphThumbMixin):
    supported_domains_re = ["www.viasatsport.se"]

    def get(self):
        match = re.search("__STATE__']=({.*});</script><script>window", self.get_urldata())
        if not match:
            yield ServiceError("Cant find video info")
            return

        dataj = json.loads(match.group(1))
        vid = dataj["dataSources"]["article"][0]["videos"][0]["data"]["mediaGuid"]

        url = "https://viasport.mtg-api.com/stream-links/viasport/web/se/clear-media-guids/%s/streams" % vid
        data = self.http.get(url)
        dataj = data.json()
        hls = dataj["embedded"]["prioritizedStreams"][0]["links"]["stream"]["href"]
        if re.search("/live/", hls):
            self.options.live = True
        streams = hlsparse(self.options, self.http.request("get", hls), hls)
        if streams:
            for n in list(streams.keys()):
                yield streams[n]
arttuberg commented 7 years ago

Thank you for quick response! Looks like we got rid of the blurring for now. Will definitely keep using this protector and recommend to friends.

jspenguin2017 commented 7 years ago

Thank you for your support. We'll keep trying to find a better solution so the video will play without the need to watch ads.

@szymon1118 Maybe we can just patch VideoJS? Block their VideoJS and use our own? Ours can be activated using a.VideoJS.init().

szymon1118 commented 7 years ago

@X01X012013 Website has a domain .fi so probably Finland isn't Geo Locked (unfortunately I haven't found any free proxy or VPN for Finland which would be working). You can try to patch their player but I don't know what do you mean by patching because their player has almost 40000 lines of code: https://viasport-assets.mtg-api.com/2017030808/prod/avod-player-f2a6e6b66f51f7838b39.js but I'm not sure if it's the main player's script. I also found yesterday but I forgot about it that maybe we can disable adblock detection using this: videoPlayers["video-ceef42dc-bd89-478c-a9ee-95dd1c46039c"]._player.adBlockerBlocker._isShowing. Let me know if it's worth something :smile: or maybe you will be able to find something more interesting in videoPlayers.

jspenguin2017 commented 7 years ago

@szymon1118 Thanks for the info. I'll have a look.

Update: Hum, this looks interesting: view-source:http://www.viasatsport.fi/embed/fd15002d-91a3-4c21-b418-212d2968a72f/ceef42dc-bd89-478c-a9ee-95dd1c46039c

jspenguin2017 commented 7 years ago

I am a bit busy lately and their code is quite complex. I'll work on this when I have time, if someone can help me pin down the offending function, that'll help a lot.

ghajini commented 7 years ago

@jspenguin2017 can you check wirh finland vpn (Tunnelbear) If these works from easylist

@@||fwmrm.net/p/MTG_Brightcove_HTML5/AdManager.js$domain=viasport.fi

@@||viasport.fi^$xmlhttprequest

@@_Play_Html5_Live$script,domain=viasport.fi

Edit=on my end it worked.....though whitelisting

jspenguin2017 commented 7 years ago

@ghajini TunnelBear doesn't have Finland... I will have another look at that Python algorithm, and see if I can find something.

jspenguin2017 commented 7 years ago

@ghajini Those rules doesn't work, with them, you will have to watch both preroll ads and blurred video. The blurring is done in CSS, but it has solid mutation observer so the blurring can't be overridden...

ghajini commented 7 years ago

Ok that was working months ago....Disregard what I suggested.....

jspenguin2017 commented 7 years ago

Oh, that Python script seems to be still valid, got the JSON URL: https://viasport.mtg-api.com/stream-links/viasport/web/se/clear-media-guids/f1YVbVOh9KCZ05UykzbCdMnGVypKvRCG/streams

And media file URL: https://viasatsport-jic-vod-hls.secure.footprint.net/hls/Viasat_Sport_-_Production/995/808/index.m3u8?nvb=1497290588&nva=1497297818&token=095f412fc170922f9abed

jspenguin2017 commented 7 years ago

OK, I was able to get this to work... Kind of... Some React(?) code is preventing the document from being mutated... I had to nuke the whole webpage. image

jspenguin2017 commented 7 years ago

I think that's pretty horrible user experience... But I'll still mark it as workaround...

@adreporter Thanks for the VPN.

jspenguin2017 commented 7 years ago

It's been a while and I still couldn't find a solution, closing as no solution. Please let me know if you have found something interesting.

jspenguin2017 commented 7 years ago

The debug switch is now hard coded into the extension, you will have to load the unpacked extension to enable it...

jspenguin2017 commented 7 years ago

Fixed in https://github.com/jspenguin2017/uBlockProtector/commit/8020c2610b21a7269a943bb45467b1db0f7c2a78