android ver? - Githubissues

xxcriticxx commented 7 years ago

@X01X012013 how long till we have android version?

jspenguin2017 commented 7 years ago

That's a good question... Considering there isn't a good script host on Android yet, we might need to wait a bit... It should work fine on Android right now, but I'm not sure.

xxcriticxx commented 7 years ago

there was something about android on reek's resp before am going try to find it

jspenguin2017 commented 7 years ago

Try FireFox + USI on Android. I use this configuration right now, but I'm not sure if it works or just crashes, since I can't easily see the console...

xxcriticxx commented 7 years ago

@X01X012013 https://github.com/reek/anti-adblock-killer/issues/1215

xxcriticxx commented 7 years ago

according to that there is TM for android

uBlock-user commented 7 years ago

I tried TM on Android, doesn't work as expected, not the same as the extension anyways.

jspenguin2017 commented 7 years ago

TM for Android is a browser, it would be tricky to get uBO working on it. You can install uBO on FireFox for Android.

uBlock-user commented 7 years ago

That's just bad, it should have been a proper script host app rather than browser app.

There's another way. Install FF on Android and install TM from the AMO.

xxcriticxx commented 7 years ago

or try that USI

jspenguin2017 commented 7 years ago

Is TM available on FireFox for Android? It wasn't there last time I checked...

xxcriticxx commented 7 years ago

this don't work?

jspenguin2017 commented 7 years ago

That is a browser, you can't install uBO in that browser. Since there is no point to have AdBlock Protector if you don't use any ad blocker, I'd say that doesn't work.

jspenguin2017 commented 7 years ago

You might be able to get a system wide ads filter, but none I tried really works without root.

xxcriticxx commented 7 years ago

i dotn have phone to try this on someone will have to test this for us

jspenguin2017 commented 7 years ago

What? Then how did you take the picture above?

uBlock-user commented 7 years ago

I use Adguard Premium on Android.

xxcriticxx commented 7 years ago

i borrow my coworkers cell phone i have iphone

jspenguin2017 commented 7 years ago

Adguard isn't free right? That's not an option here, if we are going to support Android, we'll have at least one recommended setup that is free.

xxcriticxx commented 7 years ago

i am using adguard free on my iphone

xxcriticxx commented 7 years ago

i tested some more and i dont think its working it always shows 1 on all the pages i go to

jspenguin2017 commented 7 years ago

If you don't have an ad blocker, Script alone won't be doing anything... I have a way to see the console of FireFox for Android, it's a bit complex so it might be better to wait for me to properly test things out. If you have any setup recommendation (browser + ad blocker + script host "combo") please post it below.

xxcriticxx commented 7 years ago

according to google store adguard android only works on yandex browser and samsung internet browser. have to confirm this with @ameshkov.

uBlock-user commented 7 years ago

That's Adguard Content Blocker, that's different. I'm talking about this - https://adguard.com/en/adguard-android/overview.html Also this one is not available on Play Store, so don't search there.

ameshkov commented 7 years ago

@uBlock-user is right, ContentBlocker is just a browser extension, while the full version cannot be in Play store as Google does not tolerate system-wide ad blockers. Full version is free for blocking ads in browsers, the only paid option is blocking ads in apps. Anyways, it does not yet support userscripts.

FF with greasemonkey is a viable option if you're okay with its performance and such, but it'd be hard to maintain.

@X01X012013 didn't you think about compiling AdBlockProtector into a WebExtensions-based add-on as well? It does not mean that you have to drop the userscript, what I mean is providing a browser extension as an alternative option.

Benefits:

You won't need to think about script hosts anymore. I know two Android browsers supporting WebExtensions add-ons: Firefox and Yandex browser.
It will be much easier to debug it as TamperMonkey makes it process very painful.

It should be relatively easy.

Wrap all the code into a function:

var applyProtector = function() {
// Here goes the original userscript code
}

Inside of a content script do the following:

// This is pseudo-code, but you'll get the idea.
var scriptText = applyProtector.toString();
var script = document.createElement('script');
script.innerText = '(function() { try { ' + scriptText + ' } finally { document.currentScript.parentNode.removeChild(document.currentScript); }';
document.rootElement.appendChild(script);

The only problem is GM_ functions. However, you currently use them only for configuring userscript, and with your own extension you can do it elsewhere.

jspenguin2017 commented 7 years ago

@ameshkov If we will rely on extensions, why don't we just go uBlock Origin + USI? Our List is specifically designed for uBlock Origin and uBO works on FireFox for Android.

ameshkov commented 7 years ago

USI?

// Ah, nevermind, found it.

ameshkov commented 7 years ago

Anyways, not trying to convince you, just throwing ideas in response to

If you have any setup recommendation (browser + ad blocker + script host "combo") please post it below.

xxcriticxx commented 7 years ago

@X01X012013 tested on android tv box with ff and ubo and usi. atominik.com works, titulky.com works, pudelekx.pl does not work, miniminiplus.pl does not work. i think you have to adopt greasemonkey routines. i will test on chrome later today.

jspenguin2017 commented 7 years ago

@xxcriticxx GreaseMonkey has some weird syntax and restrictions, I don't really want to get into that. Script is built ground up for TamperMonkey, we can't switch unless someone wants to help me port it to GreaseMonkey.

@ameshkov Sorry I think you misunderstood, our situation is like so:

Use FireFox for Android, obvious choice for ad blocker would be uBO, and script host USI (or make our own as you recommended). The problem is FireFox for Android isn't really a good browser, it's slow beyond belief and can't really handle modern JavaScript, even jQuery 3.x crashes on it.
Use another browser, now I need the community for recommendation on the "combo".

Your combo requires FireFox, but for ad blocker you chose Adguard, where there is an obvious better alternative, so I wanted your rational on your choice.

jspenguin2017 commented 7 years ago

OK, to sum up, here are the possible combos we got so far:

FireFox + uBO + USI¹ / Native extension²
TM Browser + Adguard¹ / Pi Hole² / Port uBO to Android³
Custom Chromium³

¹ May require some redesign of AdBlock Protector ² May have compatibility or other issues ³ May be too difficult or require too much time

ameshkov commented 7 years ago

Your combo requires FireFox, but for ad blocker you chose Adguard, where there is an obvious better alternative, so I wanted your rational on your choice.

Not at all, the whole point was that with WebExtensions approach you don't need to stick to a single browser. For instance, I find FF on Android just too slow with or without ad blocker (IMO). Also, it will guarantee that userscript is executed before page scripts. Even TM does it in an async manner, waits for a message from the background page.

However, that there's only one Chromium-based browser supporting extensions, I hope one day there will be more.

jspenguin2017 commented 7 years ago

@ameshkov I'm not very familiar with extensions, why do I need to add an element to the page? Can't I just use unsafeWindow.eval? Also, I don't think many people would like to side load extensions, does the store has certification time after the first release? Or should I just make a loader instead?

ameshkov commented 7 years ago

@X01X012013

Can't I just use unsafeWindow.eval?

There is no unsafeWindow, so the only way to execute script in a page context is to inject the script tag.

When I get a free minute, I'll try to build an example extension for you.

jspenguin2017 commented 7 years ago

@ameshkov I guess I should go read the documentation... I have no idea how the whole extension thing works...

ameshkov commented 7 years ago

@X01X012013 here it is: https://github.com/ameshkov/sample-extension

How to test it in Chrome:

Clone the repo
Open chrome://extensions and switch it to developer mode
Load unpacked extension -> load sample-extension

Inspect extension's background page to see what's going on inside of the main process. Use dev tools on any webpage and note the sample-extension.js file executed everywhere (it just prints something in console).

jspenguin2017 commented 7 years ago

@ameshkov Thanks a lot, this definitely helps to get started, but those GM functions are still a problem... Looking at Violent Monkey's source code, it seems that VM is able to get a reference to the content window? https://github.com/violentmonkey/violentmonkey/blob/12bbd191d6b0e8819876e17988752caf0995e73c/src/injected.js#L343

ameshkov commented 7 years ago

but those GM functions are still a problem...

What if you move the configuration part to a separate javascript file?

The second part could've use the configuration passed as a parameter:

var applyUserscript = function(configuration) {
}

Looking at Violent Monkey's source code, it seems that VM is able to get a reference to the content window?

Nope, it is not a real window. It shares DOM though, but not javascript.

jspenguin2017 commented 7 years ago

@ameshkov I need more than just read and write configurations with GM functions. I also need privileged close and cross origin request.

ameshkov commented 7 years ago

@ameshkov I need more than just read and write configurations with GM functions. I also need privileged close and cross origin request. (Other functionalities can be done by the extension code)

In this case, you'll have to implement some sort of a messaging between the page code and the content script. Again, I'll show you a simple example of how it's usually done once I have some free time.

jspenguin2017 commented 7 years ago

@ameshkov But messaging would be asynchronous, ~~I sometimes need a synchronous cross origin request...~~ actually, they are not synchronous and I didn't know it 😶

ameshkov commented 7 years ago

I wonder how did they implement it in TM as it uses the very same script injections approach.

jspenguin2017 commented 7 years ago

It's getting quite complex, and we are heading from a simple wrapper to re-implementing a script host... There are a few open source script hosts already, there is no reason to re-implement that. However, for open source system wide ad blocker, we have:

Adblock Plus: unmaintained, single filter list, and overall pretty dysfunctional. https://github.com/adblockplus/adblockplusandroid
AdAway: Edit host file, no run-on process, speed up connection, but needs root, and can't handle more complex filtering. https://github.com/AdAway/AdAway
NetGuard: Can configure for individual application, but only support domain based ad blocking. https://github.com/M66B/NetGuard (This one is interesting, it's not fully free out of the box, but it's open source, which means you can add a negate operator to the purchase check and unlock the features... This is the first time that I see an open source project that isn't fully free.)

So nothing open source has all the functionalities we need, if we are going to make more than a simple wrapper, we might as well make an app that matches our needs. Yes, it will be significantly more complex, but we would be doing something useful for us and probably many other people and not be re-implementing a script host that is already done by someone else.

jspenguin2017 commented 7 years ago

@ameshkov I need clarification on something... Can Adguard for Android handle these rules (or equivalent with different syntax)?

Element hiding ###like-us-adblock-modal Advanced element hiding animmex.*##a[href^="https://forum.animmex.com/viewtopic.php"] Domain condition ||flakyfeast.com^$domain=gry.pl Redirect ||xmovies8.org/ads_*.js$xmlhttprequest,first-party,redirect=noopjs (Domain is NSFW) Predefined-script injection ps4news.com##script:inject(noeval.js)

jspenguin2017 commented 7 years ago

@ameshkov Oh wait, actually, Tampermonkey can't do synchronous cross origin request... The synchronous flag is ignored... But GM_getValue is synchronous... How is that done...

More I look into these script hosts more I realize that script hosts are using pretty horrible hacks to get around browser restrictions... Why browsers won't give extensions the raw reference to the real window object? It's not safer the current way since, as you see, Userscript in TM can still perform cross origin requests to steal your password. I understand that web pages must not be able to touch extensions, but why extensions can't touch web pages directly?

jspenguin2017 commented 7 years ago

This might be useful: USI source code: https://bitbucket.org/usi-dev/usi/src Unfortunately the comments are not in English...

ameshkov commented 7 years ago

@X01X012013

I need clarification on something... Can Adguard for Android handle these rules (or equivalent with different syntax)?

Disclaimer: these are just my thoughts on the topic, please don't take it too seriously.

I'd just like to point out, that I don't offer using AG as a default Android solution. Even though I find it the best available Android ad blocker ( obviously:) ), but you are right -- it is not a FOSS, so it'd be weird to advise it by default.

On the other hand, there're some "Content Blockers" out there (for Samsung and Yandex browsers), which are FOSS and which support core ABP filters syntax.

Core ABP filters syntax is somewhat limited comparing to what uBO or Adguard can do, but on the other hand it opens more capabilities for you, eventually, you'll be able to support Safari and iOS.

Element hiding ###like-us-adblock-modal

Yes

Advanced element hiding animmex.*##a[href^="https://forum.animmex.com/viewtopic.php"]

It is not an advanced hiding, just a wildcard TLD matching, we don't support it, just exact domains (animmex.net##....). We might add it in future, though. Do you find it useful?

What we do support is extended CSS syntax: https://github.com/AdguardTeam/ExtendedCss

We've discussed it with Raymond once, so uBO supports both types of syntax, backwards-compatible ([-ext-xxx=""]) and the new one (:has()).

Domain condition ||flakyfeast.com^$domain=gry.pl

Yes

Redirect ||xmovies8.org/ads_*.js$xmlhttprequest,first-party,redirect=noopjs (Domain is NSFW)

Nope. Redirect is just a workaround as browsers extensions cannot modify response content. In Adguard we have $replace modifier, which is quite more powerful: https://kb.adguard.com/en/general/how-to-create-your-own-ad-filters#replace-modifier

However, we have the $redirect modifier on our roadmap, it'll be a bit different (we'll try to make it compatible with ubo, though).

Predefined-script injection ps4news.com##script:inject(noeval.js)

We use a different way for scripts injections. However, I like what uBO does very much (named scriptlets collection which every filter author can use).

ameshkov commented 7 years ago

@ameshkov Oh wait, actually, Tampermonkey can't do synchronous cross origin request... The synchronous flag is ignored... But GM_getValue is synchronous... How is that done...

This I can tell you. It preloads current GM values state along with the userscript code.

More I look into these script hosts more I realize that script hosts are using pretty horrible hacks to get around browser restrictions...

Yeah:) Mostly due to the Greasemonkey legacy syntax. It was quite okay to have synchronous methods back there when Firefox was single-process. Now with all the inter-process-communication involved having anything synchronous may significantly slow down your browser.

Why browsers won't give extensions the raw reference to the real window object? It's not safer the current way since, as you see, Userscript in TM can still perform cross origin requests to steal your password. I understand that web pages must not be able to touch extensions, but why extensions can't touch web pages directly?

Because in this case web pages may easily interfere with the content script's privileged context. Even in the current "safe" approach, there are some very interesting examples. Look at this gem: https://bugs.chromium.org/p/project-zero/issues/detail?id=1225

jspenguin2017 commented 7 years ago

@ameshkov Thanks for the reply, it definitely helps clean up some questions I have.

It is not an advanced hiding, just a wildcard TLD matching, we don't support it, just exact domains (animmex.net##....). We might add it in future, though. Do you find it useful?

For this website, the wildcard is kind of required, since they have like 30 different domains.

In Adguard we have $replace modifier

I'll consider it to be able to handle redirect (just different syntax).

Adguard for Android looks pretty good, I agree that it is one of the best system wide ad filter, but to qualify for our recommended setup, the software needs to be either open source or free (or both).

Because in this case web pages may easily interfere with the content script's privileged context.

I don't mean the privileged script should share the same window, just in the privileged context, window.unsafeWindow points to the window of the web page. I guess it is still possible to leverage an attack through prototype of something... Yea, I guess there is a reason for completely isolated contexts...

ameshkov commented 7 years ago

For this website, the wildcard is kind of required, since they have like 30 different domains.

I guess it makes sense, will file an issue to our tracker: https://github.com/AdguardTeam/AdguardForAndroid/issues/1169

Anyways, in AG filters we're trying to stick to the core syntax when it's possible just to make filters compatible with any ad blocker available.

I agree that it is one of the best system wide ad filter, but to qualify for our recommended setup, the software needs to be either open source or free (or both).

I completely agree with this.

I hope one day AG Premium will be good (and popular:)) enough for us to make a FOSS "light" version. At the moment, we're just too afraid of people cloning the app. We'll see how it goes on iOS though (AG for iOS Pro is paid software, but it is open source at the same time).

ghajini commented 7 years ago

@jspenguin2017 ....his do you update script using USI on Firefox android..... I click check for updates in using options and it doesn't update adprotector scrip to latest......so I have to reinstall script every time to update it to latest......

jspenguin2017 / uBlockProtector

android ver? #136