FeatReq: Rewrite target hostnames in DOM

[mk-pmb]

When I try to read this blog, it tries to load Angular.js from ajax.googleapis.com. I'd have to allow that in order for my Firefox to be served my cache's redirect to ajax.google.webcache.infra, and then allow ajax.google.webcache.infra. Could there be an easy way to teach RPC about my redirects and have it replace them on the page? That would kill two birds with one stone, save a lot of clicks in RPC, and save quite some convincing of Firefox to not protect me against my trusted cache's "attack" on those websites whenever they invent a new defense.

I found

• decentraleyes in #267 "How to deal with ajax.googleapis.com?", which does have some overlap as it tries to solve a use case similar to the example.
• attempts at software bundles in #338 and #166 that could solve namespace unification aspects, which we also discussed in #417.

In comparison, rewriting domains in the usual [1] DOM locations seems a smaller task with broader range of application, as my web cache can do some other cool tricks as well. Another benefit of the redirect approach over the CDN emulation addons is to help NoScript discern the cached and live versions. Once RPC translates domains from a user-defined dictionary (wildcards are a bonus), solutions for some of the namespace unification issues can be supplied by 3rd party tools easily as a side effect.

• [1] For the most urgent cases, <script src=>, <iframe src=> and <link href=> probably will do, no need to detect the 5th level of nested stylesheet @imports. ;-)

If the domain rewriting is extended to whatever part of RPC checks and asks me "This webpage has asked to redirect to…", it could even harmonize URLs that I click in mail or forums.

[mk-pmb]

Next stages of feature creep:

• Hostname translation can include a path and protocol selection on the custom side, to help using simple webspace/ webservers like python's SimpleHTTPServer.
  • static.example.net -> http://localhost:8080/stexa
  • static.example.com -> http://freehost.example.net/e/eve2016/eve-cdn
• Templating: If the new target starts with a question mark, use the remainder, replace %hn with the current lookup's hostname, and replace %HN with %hn. It it the result starts with a question mark, repeat.
  • i.imgur.com -> undel.alias/%hn
  • i42.tinypic.com -> undel.alias/%hn
  • i43.tinypic.com -> undel.alias/%hn
  • i44.tinypic.com -> undel.alias/%hn
  • undel.alias -> https://undelete.example.net/img/auth=TOKEN/host=
  • Now I want <img src=> translated, too.
• When rewriting <link href=>, kill that <link integrity=>.
  • That's probably what breaks GitHub's CSS on the WayBack Machine – Firefox: "None of the "sha256" hashes in the integrity attribute match the content of the subresource."
  • Could probably be fixed as a side effect, by having an entry that translates web.archive.org to itself (better: to empty, and treat that as unchanged) and has the "kill integrity" flag set. Could have a flag to limit it to <link> tags for performance.
  • Option to provide a custom value for the integrity attribute.

[myrdd]

I'd be cautious with changing the URLs in the DOM. There might be multiple side effects; even more than I'll mention now.

• Fx handles Mixed Content, i.e. https -> http requests, differently; so your web cache probably needs to support https.
• The page you're requesting might notice that the URL has changed. You maybe do not want that.
• I'm not sure you really "want" RP/Noscript to think that a script comes from "*.webcache.infra", where it actually comes from another site; you'd allow all scripts (or requests) to your webcache, regardless of its real origin.
• NoScript's ABE could block the requests from some-internet-domain.com to your webcache located at your local network.

I'd have to allow that in order for my Firefox to be served my cache's redirect to ajax.google.webcache.infra, and then allow ajax.google.webcache.infra.

Really? I'd suppose RP doesn't even recognize the content finally came from the webcache, no? I'm assuming your webcache is implemented like a proxy.

In any case, I won't implement "DOM URL rewrites" in RP. If you choose to go that path, there should be another add-on (which seems not to exist yet) for it.

Maybe HTTPS-Everywhere is another approach you could try. It supports custom rulesets. However, RP still will see the requests to ajax.googleapis.com.

[mk-pmb]

Thanks for reminding me about script safety on my webcache! My NoScript can trust ajax.google.webcache.infra since it will serve only a reviewed subset of Google's real CDN. As for RPC, I shall search the issues for how to get more precise rules than *.webcache.infra, although I can't currently think of a risk that isn't blocked by either my NoScript or my webcache. (It won't easily GET tracking pixels.) ABE rules are configured to allow injection and click-stealing "attacks" from my cache.

In any case, I won't implement "DOM URL rewrites" in RP. If you choose to go that path, there should be another add-on (which seems not to exist yet) for it.

Ok, then I'll start developing one. :-)

[myrdd]

As for RPC, I shall search the issues for how to get more precise rules than *.webcache.infra,

You can do it manually. As for the menu, the issue is https://github.com/RequestPolicyContinued/requestpolicy/issues/474.

Ok, then I'll start developing one. :-)

Good luck :)

[mk-pmb]

Thanks for that hint! I thus discovered path-based rules and it looks like they might help a lot.