wp.pl and related - Githubissues

xxcriticxx commented 7 years ago

website again made couple changes @X01X012013 can you please check

jspenguin2017 commented 7 years ago

Thank you for your report, I will look into this when I have time.

xxcriticxx commented 7 years ago

thanks for help

jspenguin2017 commented 7 years ago

Test URL: http://opinie.wp.pl/andrzej-zupanski-ukradl-miliony-zostal-bohaterem-6083990028674177a http://www.money.pl/gospodarka/wiadomosci/artykul/marki-wlasne-biedronka-lidl-produkty-markowe,55,0,2241335.html

xxcriticxx commented 7 years ago

@X01X012013 did you release new ver of your software so i can test?

jspenguin2017 commented 7 years ago

Looks like they put the detector inside the video player. All the functions are closured, pretty well done. If someone figured out how to grab the link to the video file, please let me know. For now, I don't have more tricks in my bag. I'll have another look later when I have time.

xxcriticxx commented 7 years ago

@mikhoul or @szymon1118 any input on this?

szymon1118 commented 7 years ago

@X01X012013 Link to the video could be grabbed by splitting and trimming (formatting) what is inside <meta name="keywords" content=" more info here: http://wp.tv/jsapi/article_related.js . Then you can access playlist of videos eg. http://wp.tv/api/article_related.json?tags=marki+w%C5%82asne%3Bbiedronka%3Bsklep%3Bsie%C4%87%3Bproducenci+%C5%BCywno%C5%9Bci&domain=www.money.pl&type=default. Inside JSON you can get mid (probably movie id) and from this point there are two ways to grab video url (first link probably contains urls for whole playlist):

xxcriticxx commented 7 years ago

but still we have to change the player with our own player?

jspenguin2017 commented 7 years ago

@xxcriticxx Yes, we would need to do that.

jspenguin2017 commented 7 years ago

@szymon1118 I am able to grab the first JSON, but the url array is empty when I try to get "http://wp.tv/player/mid," + mid + ",embed.json". I noticed there are 30 elements in the first JSON, the first one has title matching the video player's title, is that the right one?

UPDATE: It looks like the method works for money.pl but not for wp.pl.

    if(Domain.endsWith("wp.pl") || Domain.endsWith("money.pl")){
        onEvent("load", function(){
            const tag = $("meta[name=keywords]").attr("content");
            $.get("http://wp.tv/api/article_related.json", {
                tags: tag,
                domain: Domain,
                type: "default"
            }, function(data){
                let mid;
                try{
                    mid = data.clips[0].mid;
                }
                catch(err){
                    console.error("AdBlock Protector failed to find media ID! ");
                }
                if(mid){
                    console.log("http://wp.tv/player/mid," + mid + ",embed.json");
                    $.get("http://wp.tv/player/mid," + mid + ",embed.json", function(data){console.log(data);});
                }
            });
        });
    }

mikhoul commented 7 years ago

Sadly I can't test the video from Canada.

szymon1118 commented 7 years ago

@X01X012013 I have changed your code from:

if(mid){
    console.log("http://wp.tv/player/mid," + mid + ",embed.json");
    $.get("http://wp.tv/player/mid," + mid + ",embed.json", function(data){console.log(data);});
}

to:

if(mid){
    console.log("http://wp.tv/player/mid," + mid + ",embed.json");
    $.get("http://wp.tv/player/mid," + mid + ",embed.json", function(data){
        window.open(data.clip.url[1].url, '_blank').focus();
    });
}

and for me it works for both money.pl and opinie.wp.pl websites so I don't really know where is the problem :smile:. Thanks for your help.

jspenguin2017 commented 7 years ago

@szymon1118 Hi, that is not the issue there, url[1] is undefined for the first test link. I am able to get the first video of second test page working, working on others.

jspenguin2017 commented 7 years ago

Hi guys, I was able to make the first video of some pages working! Please have a look and submit links to pages where the first video is not working.

@szymon1118 Thanks for your help, I need clarification on something: article_related.json contains video in the correct order? So the first video is clip[0] and clip[1] for the second one? Why is there 30 items in the JSON? I bet there won't be 30 videos on the same page...

I am not able to get it to work on the first test URL, maybe it's Geo Lock?

xxcriticxx commented 7 years ago

@X01X012013 what player did you use? looks like you substituted the original player

@mikhoul i am in usa most stuff wont work but some will

paging @MajkiIT and @F4z to test

jspenguin2017 commented 7 years ago

@xxcriticxx The browser built-in player, like last time. One thing I need you guys to test is the correct video is loaded. I'm assuming clips array is in the correct order...

Things to test:

Is the first video working?
Is the correct video loaded?

Also, I need to know, how many video can be on the same page? Should I add a button on the page for "load next video"?

xxcriticxx commented 7 years ago

@X01X012013 what do we need script + list? or script only?

xxcriticxx commented 7 years ago

here is page with multiple video's

http://wiadomosci.wp.pl/kat,1329,title,Austria-12-latek-czlonkiem-terrorystycznej-siatki-w-Wiedniu,wid,18685214,wiadomosc.html?ticaid=11883e

xxcriticxx commented 7 years ago

at this page http://kobieta.wp.pl/najmodniejsze-fryzury-na-nadchodzacy-sezon-6084375065597057a correct video plays for second then freezes and new one starts

correct vid:

capture

new vid:

capture2

jspenguin2017 commented 7 years ago

@xxcriticxx Hi, both List and Script are updated. The new implementation is moved from List to Script.

For your reply above, what should be the correct video? Just a black screen?

xxcriticxx commented 7 years ago

@X01X012013 vid about makeup and hair by title Najmodniejsze fryzury na nadchodzący sezon maybe ask @szymon1118 for help

jspenguin2017 commented 7 years ago

I'm a little bit busy these days, but I promise that I will be back. If someone can sort out how to grab the correct media ID, then I will be back immediately.

xxcriticxx commented 7 years ago

@X01X012013 i tested this on different computer and correct vid plays maybe was website error or maybe something with the computer i will test so more and report.

szymon1118 commented 7 years ago

@X01X012013 Referring to your question about clips array order i think that it is in the correct order. Regarding to 30 videos in JSON i think there is a playlist like on youtube to watch other video thematically related with the first one or something :smile: so it depends on demand for function to load new video (if anyone would like to watch new video and is it worth your time). For me every previously tested video works fine. I found a solution in AAK script to grab mid directly from player in website source. Here is the code: http://pastebin.com/NvrWwKFR . It is not clean, I know but I had to use setTimeout to wait until the mid variable will not be undefined. According to the playlist using for example this JSON: http://wp.tv/player/related.json?domain=opinie.wp.pl&mid=1996794&type=video we can grab if I am not mistaken every related video with the first one so there is no need now to use keywords. About Geo Lock problem I have not looked into it yet.

jspenguin2017 commented 7 years ago

@szymon1118 Wait, I'm now confused, so what is the correct way to grab mid? And what to do if there are more than one video on the same page?

xxcriticxx commented 7 years ago

@X01X012013 the correct way is the way you have it now.

xxcriticxx commented 7 years ago

@X01X012013 is wp.pl fix in dropper version?

jspenguin2017 commented 7 years ago

@xxcriticxx All the dropper does is to load the main script, it does not add anything. Currently wp.pl is partially fixed, so expect similar behavior with dropper version.

xxcriticxx commented 7 years ago

@X01X012013 roger

@szymon1118 can you look at geo lock part please?

jspenguin2017 commented 7 years ago

I'll add multi-video support when I have time. Thank you for your patience.

szymon1118 commented 7 years ago

Sorry for the long absence but I had exams. Studies, you know :smile:.

@xxcriticxx I tested two websites through VPN and i have not any Geolock protection so maybe It could be bypassed by replacing a player. Anyway I hadn't noticed any geo lock myself connecting via VPN. I also found that https://github.com/MajkiIT/polish-ads-filter found a solution: http://www.certyficate.it/pomijanie-wykrywania-adblocka-portalach-wp/ but I think when WP change their code the script will not work.

@X01X012013 The newest logic of grabbing mid I found in AAK code works for every site that @xxcriticxx posted so I think we can stay with this method. The problem is with replacing multiple videos on website. I did some tests and for me it is hard to find a good method to replace many films on single website because eg. there is a website: http://wiadomosci.wp.pl/kat,1329,title,Austria-12-latek-czlonkiem-terrorystycznej-siatki-w-Wiedniu,wid,18685214,wiadomosc.html?ticaid=11883e and without scrolling down while loading only one video is loaded.

What I observed that they are loading dynamically new articles with videos attached by AJAX while you scrolling down a page. So we should dynamically replace new videos or scroll very fast to the very down of website and then up like I did (I think scrolling down with JS is a bad practice but I hadn't found any better solution). It it my code for now: http://pastebin.com/EQHrRUAb.

jspenguin2017 commented 7 years ago

@szymon1118 I implemented multi-video support to 3.0.20, try it out and let me know how it goes.

szymon1118 commented 7 years ago

@X01X012013 Userscript works (good job) but the old method with keywords is bad in my opinion because the title upper video doesn't match video content (for the first video it is good but for the next video title is different). To my mind the best option I have found till now will be to grab mid directly from the player as I posted in a code above: var mid = $( player ).find(".titlecont a.title").attr("href").match(/mid[=,]([0-9]+)/);

jspenguin2017 commented 7 years ago

@szymon1118 @xxcriticxx 3.0.21 is out, try it out and let me know if the videos are correct this time 😄

szymon1118 commented 7 years ago

@X01X012013 Great job :smile: It seems to work for every site that had been posted. For http://www.certyficate.it/pomijanie-wykrywania-adblocka-portalach-wp/ I know that they asked for help on adguard forum: https://forum.adguard.com/index.php?threads/resolved-kobieta-wp-pl.18565/page-2#post-123248 so maybe you can check more informations from here. I tested this script and I remember that I couldn't run it in chrome. Tried in firefox with Greasemonkey and it worked so maybe script has more issues.

jspenguin2017 commented 7 years ago

@szymon1118 They have a pretty long list of domains, should all of them be matched or just wp.pl and money.pl?

szymon1118 commented 7 years ago

@X01X012013 I think script should work similarly for other domains but i'm not sure. Maybe @xxcriticxx make further tests because I rarely visit these websites. If I found any issues I will report.

jspenguin2017 commented 7 years ago

@szymon1118 I'll add all the domains in, the player patching routine shouldn't break anything. Let me know if it does.

xxcriticxx commented 7 years ago

@szymon1118 i am in usa so i get geo lock on some videos

@X01X012013 can you fix or patch that code that @MajkiIT put on his website? also wp provides videos for many different websites.

jspenguin2017 commented 7 years ago

@xxcriticxx I don't want to add Geo Lock breaking into AdBlock Protector. And I have no clue what is the thought process behind that Userscript. I mean I know what it does, but I don't know why it tried to do those things.

Check this list for domains related to wp.pl. https://github.com/X01X012013/AdBlockProtector/blob/gh-pages/Related%20Domains.md If you mean videos on "external" websites, then I will need to know which website does that and list them one by one. I can't make it generic since chances are it will break more than it fixes.

xxcriticxx commented 7 years ago

@X01X012013 good start would be this list of domains from @F4z

http://biztok.pl
http://wp.tv
http://kafeteria.tv
http://kafeteria.pl
http://wrzuta.pl
http://pudelek.tv
http://komediowo.pl
http://sfora.pl
http://autokrata.pl
http://wawalove.pl
http://hotmoney.pl
http://snobka.pl
http://money.pl
http://abczdrowie.pl
http://gadzetomania.pl
http://autokult.pl
http://komorkomania.pl

jspenguin2017 commented 7 years ago

@xxcriticxx I think all those are already included.

xxcriticxx commented 7 years ago

@X01X012013 i will dig around maybe i Find some more

MajkiIT commented 7 years ago

All wp sites.

abczdrowie.pl
autokrata.pl
autokult.pl
biztok.pl
gadzetomania.pl
hotmoney.pl
kafeteria.pl
kafeteria.tv
komediowo.pl
komorkomania.pl
money.pl
pudelek.tv
sfora.pl
snobka.pl
wawalove.pl
*.wp.pl
wp.tv
wrzuta.pl
pudelek.pl

jspenguin2017 commented 7 years ago

@MajkiIT Yes, all 19 are already included. Please check here for the list: https://github.com/X01X012013/AdBlockProtector/blob/gh-pages/Related%20Domains.md

xxcriticxx commented 7 years ago

got couple ad block today can someone test

http://komorkomania.pl/34194,jak-wydluzyc-czas-pracy-telefonu-na-baterii http://sportowefakty.wp.pl/sportowybar/664944/szpilka-trenowal-jak-rocky-balboa-jego-film-wzbudza-jednak-kontrowersje http://wawalove.pl/Jest-sledztwo-ws-przyjecia-korzysci-majatkowej-przez-Hanne-Gronkiewicz-Waltz-lub-jej-zastepce-a25560

jspenguin2017 commented 7 years ago

@xxcriticxx I'll fix it

Note: load jQuery + hide block screen with List

MajkiIT commented 7 years ago

@X01X012013 all domains are OK.

xxcriticxx commented 7 years ago

@X01X012013 i have both list and script and i still get ad block msg

jspenguin2017 commented 7 years ago

@xxcriticxx I'll fix it, soon... Not done yet...

jspenguin2017 / uBlockProtector

wp.pl and related #40