Functionality of uMatrix/RequestPolicy/uBlock/Policeman

[CrashNBurn71]

Has anyone taken a look at GorHill's uMatix?

It's what I wound up using in Opera (as it beat the pants offa its closest unmaintained competitor).

So I've been testing it in FireFox for the last few months. I really like the fine-grained control - wherein you can allow/deny different types of content from domains. As opposed to blocking/allowing ALL from a given domain.

[Bfgeshka]

There's Policeman, which can do it in the same way, but much closer to RPC.

[nodiscc]

I was going to open discussion on this, you beat me to it. There is a lot of overlap in functionality between RP, Policeman, uMatrix, uBlock...

I find RP more user-friendly than uMatrix, while uMatrix has more fine-grained control in some areas (blocking resource types - issue for this is https://github.com/RequestPolicyContinued/requestpolicy/issues/547). Also keep in mind that resource types can just be guessed, see https://github.com/RequestPolicyContinued/requestpolicy/issues/256#issuecomment-65745840

It was also suggested to import common adblock format domain blocking lists to subscriptions (see https://github.com/RequestPolicyContinued/subscriptions/issues/28), which may or may not be a good idea.

Overall I think the overlap "problem" can not be fixed as this is something inherent to Free Software projects (different software for different use cases).

Input welcome.

[CrashNBurn71]

RPc is definitely more accessible, and obvious as to what means what. It took me a while on Opera to figure out how to actually use uMatrix. uBlock gets "talked about" on slashdot when AdBlock et al come up... but it is functionally crippled compared to pretty much all its other brethren, as its also mentioned how to use uB with uM.

RPc also has the ability to allow|deny an IP-address|Domain that is a secondary or tertiary call... e.g. 128.155.8.1 --- allow if it comes from foobar.com which is allowed on primary domain foo.com. Whereas uMatrix "loses" that information. You would need to allow "foobar.com" and 128... for foo.com --- and you would see how many "types" of things those things might allow --- but you would also be allowing 128.155.8.1 for ANY request from foo.com or any of it's secondary|tertiary+ resource|html requests... I don't know if thats a significant difference, or a possible security hole or not. It's definitely amusing to me (at least) that uMatrix doesn't actually "matrix" the domain request|calls... just the resources requested.

Personally my biggest (maybe only) problem with RPc is trying to manage rules from the config - in that case all the "fall-through" logic|mechanics of RPc is lost, and the rules are a random unrelated mess.

I really like uMatrix's ability to allow|deny resources regardless of origin (CSS especially). Or block iFrames - unless force overridden (even when an origin is allowed).

[nodiscc]

you can allow/deny different types of content from domains. As opposed to blocking/allowing ALL from a given domain.

Would this be solved/related to #688? (see comment https://github.com/RequestPolicyContinued/requestpolicy/issues/688#issuecomment-128519307 - UI options to block from full/base domain/full path)

[myrdd]

@CrashNBurn71

RPc also has the ability to allow|deny an IP-address|Domain that is a secondary or tertiary call

Yes, thanks for mentioning that. It's difficult if not impossible to put all the information in one matrix.

Personally my biggest (maybe only) problem with RPc is trying to manage rules from the config - in that case all the "fall-through" logic|mechanics of RPc is lost, and the rules are a random unrelated mess.

I don't get what you mean – do you want to manage the rules directly in the config file? Suggestions are always welcome.

I really like uMatrix's ability to allow|deny resources regardless of origin (CSS especially).

This is possible in RP. Add a rule * --> destination.

Or block iFrames - unless force overridden (even when an origin is allowed).

Yes, I think iFrames are a good point – but iirc NoScript can do that as well, and I'd recommend to always use NoScript.

---

@nodiscc

Would this be solved/related to #688? (see comment #688 (comment) - UI options to block from full/base domain/full path)

No, #688 is nothing about content types. They are used by policeman and uMatrix. Regarding RP see https://github.com/RequestPolicyContinued/requestpolicy/issues/547.

---

@nodiscc I see you gave this issue the „unconfirmed“ label. I am confused – how do you understand this label? I'd say „unconfirmed“ is for bugs which are not reproducible.

[nodiscc]

@myrdd I added the unconfirmed label because this issue is not about a particular item that needs to be changed/improved so I was hoping we could find actionable tasks in this discussion, and report them as separate issues. I guess the discussion label would be fine as well.

[nodiscc]

Hi, I've been experimenting with different addon combinations. I was previously using RequestPolicy Continued + µBlock Origin. It seems that with appropriate settings, uBlock alone can reproduce RP's behavior (let the user control cross-site requests):

For the equivalent of Default Deny in RP, the trick is to set Block (red) for 3rd-party in the middle column in uBlock.

• Left column is the request destination
• Middle column is the global/default policy:Allow/Block all requests TO otherdomain.com (globally)
• Right column is the policy for the current domain: Allow Block requests from currentsite.com to otherdomain.com

I believe this blocks all 3rd-party resources. Legitimate destinations have to be whitelisted on a per-site basis in the right column (here shown in green), like in RPc.

The UI is not very clear (missing column headers, missing tooltips when hovering a rule, and I think you need to click the padlock icon after editing if you want to make your changes permanent), so this is not a full replacement for RP (very straightforward to use), but it allows running only one addon, which simplifies usage (what addon filters first? what exactly is filtered? need to check both addons to adjust filtering, etc.).

The docs are confusing, but I could find some info at

• https://github.com/gorhill/uBlock/wiki/Blocking-mode:-hard-mode
• https://github.com/gorhill/uBlock/wiki/Dynamic-filtering:-default-deny

What is the difference between these? I'm still experimenting, can someone confirm my findings? @myrdd @gorhill does uBlock actually have equivalence with RequestPolicy when set up this way?

Update: https://github.com/gorhill/uBlock/wiki/Dynamic-filtering:-quick-guide is the main resource. It seems that destination domains should be whitelisted using the noop (grey) setting instead of the allow (green) setting, because allow also bypasses static (list based) filters. You want to allow requests to this domain, but not to known ads on it.

[myrdd]

Thank you nodiscc for your description! I did not know uBlock's dynamic filtering. I like it more than the UI of uMatrix.

The UI is not very clear (missing column headers, missing tooltips when hovering a rule

Yes, you need to read the documentation first. However, after reading the docs, I find the deny/noop/allow colors intuitive.

I think you need to click the padlock icon after editing if you want to make your changes permanent

That's correct, but you can directly create a permenent rule by holding CTRL. The padlock is documented here.

[uBlock] is not a full replacement for RP (very straightforward to use), but it allows running only one addon, which simplifies usage (what addon filters first? what exactly is filtered? need to check both addons to adjust filtering, etc.).

The main, inherent difference between RP and all other blockers I know is the "other origins" feature. [tech details] As far as I can tell only extensions to Gecko-based browsers are able to have an "other origins" feature. Unfortunately, Mozilla didn't add the "origin URI" to the WebRequest API yet. [/tech details]

Besides that, RequestPolicy aims at being intuitive and easy to learn for what it does. Since the focus of uBlock and uMatrix is elsewhere, RequestPolicy will continue to be a separate add-on. However, it would be nice if RP communicated with other blocker add-ons like uBlock so that using both at the same time gets more convenient.

---

So, if you want to use uBlock in the way RequestPolicy works, you should block anything third-party and then set a noop (!) rule on any domain you want to allow. Use noop instead of allow so that uBlock's static rules still apply.

[nodiscc]

Thanks, added https://github.com/RequestPolicyContinued/requestpolicy/wiki/FAQ#using-ublock-like-requestpolicy.

uBlock does not block server or client redirects (https://github.com/gorhill/uBlock/issues/226), whereas RPc does.

[nodiscc]

uBlock's rule storage file is formatted like:

* amazon-adsystem.com * block
* scorecardresearch.com * block
* sharethis.com * block
* skimresources.com * block
* smartadserver.com * block
* statcounter.com * block
en.wikipedia.org wikimedia.org * noop
github.com avatars0.githubusercontent.com * noop
github.com avatars1.githubusercontent.com * noop
github.com avatars2.githubusercontent.com * noop
github.com avatars3.githubusercontent.com * noop
github.com camo.githubusercontent.com * noop
github.com cloud.githubusercontent.com * noop
github.com github-cloud.s3.amazonaws.com * noop
github.com raw.githubusercontent.com * noop
lazerhawk.bandcamp.com bcbits.com * noop
lovelybacon.deviantart.com deviantart.net * noop
makezine.com netdna-cdn.com * noop
news.softpedia.com softpedia-static.com * noop

@myrdd Moving to this storage format for import/export/subscriptions would be great (as well as supporting noop operation. I think you intended to rewrite the subscriptions module (https://github.com/RequestPolicyContinued/requestpolicy/issues/597)?

It would largely decrease maintenance of "official" subscriptions (if RP is made able to download custom lists). uBlock and RPc lists/subscriptions would not need to be maintained separately anymore (both block and allow (noop) lists). Would completely solve subscriptions/#28, https://github.com/RequestPolicyContinued/requestpolicy/issues/491, https://github.com/RequestPolicyContinued/requestpolicy/issues/645, https://github.com/RequestPolicyContinued/requestpolicy/issues/717, https://github.com/RequestPolicyContinued/requestpolicy/issues/185, ...

At this point I'm only using uBlock and custom Firefox preferences, and am slowly rebuilding whitelists. Only 1 blocker with only 1 control panel, better temp/permanent rules management, priorities, strictness controls... Only things missing are redirect controls, request count, image placeholders and other origins.

I will keep maintaining the subscriptions repo and get the website fixed, but am not currently using RPc (actively) anymore.

[myrdd]

Moving to this storage format for import/export/subscriptions would be great (as well as supporting noop operation.

This would be a breaking change, so something for v2.0. However, we could add support for uBlock lists / AdBlock lists in a 1.0 version. Feel free to create issues.

Having ternary rules (allow/block/noop) would be as well something for 2.0.

I think you intended to rewrite the subscriptions module (#597)?

True. After that, it'll be easier to improve the subscription capabilities.

It would largely decrease maintenance of "official" subscriptions

An intermediate step could be a script that converts the lists into lists readable by RP, on a daily or hourly basis.

Thanks @nodiscc for your continuous help!

[layus]

I may be a bit late to the party, but it seems that uMatrix is fundamentally different from RP in the sense that rules are always interpreted in the context of the main website, while RP allows to define individual rules for third parties.

For example, I do not want to enable youtube.com everywhere, but when I do, I want youtube.com to be able to reach googlevideo.com. With RP, I just add the permanent setting that youtube can access googlevideo. Whenever I temporarily enable youtube for a given website, it works immediately because it can access googlevideo.

With umatrix, you need to enable youtube and googlevideo on every website where you want youtube videos. RP has a notion of hierarchy among third parties, while umatrix has none.

In that context, while RP rules can be exported in the same format as umatrix, but they do not mean the same thing. The rule youtube.com googlevideo.com * allow, which corresponds to the above use case, is interpreted differently by RP and umatrix.

[drzraf]

To dump it: jq -r .dynamicFilteringString < ~/.mozilla/firefox/*.default/browser-extension-data/uBlock0@raymondhill.net/storage.js

[myrdd]

from https://github.com/RequestPolicyContinued/requestpolicy/issues/704#issuecomment-357897329:

[uBlock Origin and uMatrix] do not have the concept of "other origins".

At least uBlock Origin has.

There's a toggle for resources from third-party sites.

@YtvwlD Do you mean the "3rd-party" line?

As far as I understand, it's not the same as in RP:

[YtvwlD]

Okay, what's the difference? This seems to have the same effect as enabling the GitHub domains in RequestPolicy.

[CrashNBurn71]

It's not the same. Yet I can't think of a single instance where I would need the RPC version

• Main Site causes Site2 to attempt to load resources from Site3. -- RPC: Allow Site3 coming from Site2 coming from Primary Domain. -- uMatrix: Allow|Deny Site2. Allow|Deny Site3 within the context of the Primary Domain.

In the latter case, you still get the Primary Domain's context, and allowing "Site3" doesn't whitelist it for every other site, unless you explicitly do so.

These days on the desktop I find uMatrix easiest to manage, with a handful of glob|regexp♰ exceptions done with uBlock. While on mobile, I just use Firefox Focus.

♰ Which RPC was not inclined to add.

[Uristqwerty]

Consider these cases:

Sites may embed youtube, and youtube can pull in required resources from other google domains, but sites cannot normally access anything from google.

Same as above, but you only allow youtube temporarily on a case-by-case basis, but because youtube's dependencies are grouped with youtube, you only need to enable one domain and the rest come with it.

[CrashNBurn71]

I don't allow sites to "embed Youtube", that's an iFrame, and they are hard blocked by default. I'll just open the video in question in it's own tab, if I really need to see it,

Such videos show as a empty png box with the video-link-text at the top, e.g. https://www.youtube.com/embed/K00ljNsLp7E?start=0&wmode=transparent

[YtvwlD]

I don't allow sites to "embed Youtube", that's an iFrame, and they are hard blocked by default.

I allow YouTube for certain sites.