Websocket Exception Rules Being Ignored?

[GuardianMajor]

I tried looking for past discussion on this, what I sparsely found about websockets didn't really apply to my case; so, here it goes.

Issue

In the presence of * * 3p block adding a site exception for websocket such as xyz.com * websocket noop gets discarded as soon as the rules are saved. Why is this?

Example

Any site that establishes ws:// websocket connection to any external site, for example you can test this on beta.speedtest.net

Steps

Pretty self explanatory, but:

1. Set the global rule * * 3p block
2. Try to set the site rule beta.speedtest.net * websocket noop
3. Save (it discards it)

Your settings

Mostly default, global 3p rule and advanced setting of suspendTabsUntilReady true

• OS/version: Windows 10 Pro x64
• Browser/version: Firefox 52.0.1 (32-bit)
• uBlock Origin version: 1.12.1

Am I missing something, because otherwise it seems like a bug. Anything else I should look at? I kept this particular profile and configuration fairly stock to avoid "other" factors. Eventually once resolved, the configuration will get deployed more broadly.

Thanks.

[gorhill]

beta.speedtest.net * websocket noop

Not supported. Type-based rules work only for these types: image, inline-script, 1p-script, 3p, 3p-frame, 3p-script.

[GuardianMajor]

To add some additional information, I kept trying with this and found that this rule seems to work but still don't know why.

beta.speedtest.net ws:// websocket allow

but the question remains why the redundancy, isn't telling it about websocket enough, without the need to add ws:// for the destination?

Edit: In hindsight of what you said, it seems that it is not redundant and probably just ignoring the websocket type keyword.

[GuardianMajor]

@gorhill you beat me to it trying to provide more information :) I noticed it was missing from the list of types on the documentation but it also came with the notice that you are not done with the documentation and will get to it later, so wasn't sure if that was the reason.

So why isn't websocket a type? I mean it is clearly present for everything else, why not for this portion? Just trying to understand the logic behind the exclusion. For the filters it is present, why not for the rules?

[gorhill]

So why isn't websocket a type?

This was designed as it is currently simply because these are the types which are supported by the dynamic filtering panel UI, I didn't want the overhead cost of supporting any type when there are no way to easily toggle these types in the UI.

[Atavic]

@GuardianMajor There are two URI schemes: Web Socket ws:// and Secure Web Socket wss:// for the Websocket Protocol.

[GuardianMajor]

I see, you are right, they are not available in the GUI but since the rules editor has a "manual" option, meaning you can do it by hand, wouldn't it be simple enough, without too much overhead to allow it there - in the came vain as CLI and not add the GUI element which would satisfy the lower overhead cost, no?

You already done the heavy lifting of having the types available using the "rules wizard" to create them (such as $websocket, etc) from say the logger, so this would hardly be that much more system heavy to be available as a manual option. Just speaking to the robustness of the solution, this would certainly add that without burdening the interface itself, just a though. Down the line, if it tests well, might even be able to add a GUI interface for it without too much overhead as well.

Just something to consider. Thanks.

@atavic, yes I am aware of that thank you but has really no bearing I can see on this particular discussion, does it? You know you can subscribe to a topic without actually saying something right? :) Unless participation and contribution is not the goal, in which case, carry on.

[GuardianMajor]

@Atavic yes thank you, it was already explained by the developer, repeating it adds no value.

[gorhill]

The worry is not the heavy-lifting, the worry is the added overhead. Currently this is what happens when uBO has to test for dynamic filtering of a websocket request type, all these combination needs to be tested, in order:

beta.speedtest.net  www.example.com          *
     speedtest.net  www.example.com          *
               net  www.example.com          *
                 *  www.example.com          *
beta.speedtest.net      example.com          *
     speedtest.net      example.com          *
               net      example.com          *
                 *      example.com          *
beta.speedtest.net              com          *
     speedtest.net              com          *
               net              com          *
                 *              com          *
beta.speedtest.net                *          *
     speedtest.net                *          *
               net                *          *
                 *                *          *

If uBO was to support to dynamic filtering rules for any type, it would also need to evaluate the following set of combinations:

beta.speedtest.net  www.example.com  websocket
     speedtest.net  www.example.com  websocket
               net  www.example.com  websocket
                 *  www.example.com  websocket
beta.speedtest.net      example.com  websocket
     speedtest.net      example.com  websocket
               net      example.com  websocket
                 *      example.com  websocket
beta.speedtest.net              com  websocket
     speedtest.net              com  websocket
               net              com  websocket
                 *              com  websocket
beta.speedtest.net                *  websocket
     speedtest.net                *  websocket
               net                *  websocket
                 *                *  websocket

So what appears as a trivial enough thing from outside, is something which double the overhead incurred for every single network request. Since the UI does not support all sort of request types, I did not want all users of dynamic filtering pane to incur the overhead cost which would be worth to just a minority of users (compare the number of users of uBO versus uMatrix).

This said, I may -- as time allow -- consider investigating different representation internally to lower overhead and thus eliminate the worry.

[GuardianMajor]

@gorhill not sure if I can ask here while we are in the middle of it or if I should take it over to the Mozillazine discussion; so bear with me and let me know - it is not intended to go against repository guidelines.

Is there a way to have * * 3p block enabled (which I assume will also include 3p scripts/frames/etc) and yet have extensions such as DownThemAll, PushBullet, etc to accomplish their tasks? I have been playing with behind-the-scene and moz-extension-scheme to try and exempt them.

However, the result has been inconsistent or often contrary to what is expected. I was hoping it is just something wrong with the way the rules are written; hoping you can weigh in to elimination the possibility of a parsing bug? Thanks in advance.

PPS. Here is what I tried to do to mitigate this (example), no success: add moz-extension-scheme * * noop (changing to allow has same effect) and you get below in logger:

13:37:39|[eye]| 3p block|--|other|https//www.bing.com/az/hprichbg/rb/GlacierBay_EN-US149389825061920x1080.jpg (the : was removed from the above link to prevent link generation)_

PPPS. I am aware that putting those two in the whitelist would "resolve" it but what I need is a bit more of a scalpel than machete. Meaning, I rather not allow ALL behind the scene or extension activity, just the ones that have been allowed. Wanted to clarify that in case it didn't come through. Thanks.

PS. I appreciate the explanation and I hope that in time you can find a way to mitigate that. Thank you. In the meantime, any chance you can introduce an advanced setting toggle for it (something like - enableAllTypes = true/false) so that those ("minority" of users) who already deviate from the default to enable/have it as long as they accept the "overhead"? Just a thought on how to introduce it without affecting everyone. Thanks.

[h1z1]

Problem I have with this is it's not clear to end users (advanced or not), that not all types are possible let alone supported. Twitter for example generates csp-response callbacks. I'd never heard of those before nor know why a site needs to know.

ie

https//twitter.com/i/csp_report?a=NVQW&ro=false

[Atavic]

@h1z1 Look at: report-uri It is listed halfway on CSP in the Wild Section.

Also here it says that what your are blocking is sent through a JSON-formatted report of the violation.

Twitter's CSP Report Collector.

[gorhill]

Duplicate of #230.