myrdd commented 10 years ago

This issue is for general discussion and any kind of short comments or questions. Anything that doesn't need a separate issue can be written here. Off-topic allowed.

other useful discussions:

documentation (and wiki) discussion: https://github.com/RequestPolicyContinued/requestpolicy/issues/902
developers' discussion: https://github.com/RequestPolicyContinued/requestpolicy/issues/826

SkySkimmer commented 10 years ago

Why is the website in a branch as opposed to a separate repository?

myrdd commented 10 years ago

@SkySkimmer gh-pages is an orphan branch, this means it doesn't share history with another branch. Github makes the content of that branch available on https://requestpolicycontinued.github.io/requestpolicy/. See https://github.com/RequestPolicyContinued/requestpolicy/issues/445#issuecomment-52476299.

myrdd commented 10 years ago

FYI: 446e54bfae856edf1a01c12a8a1698ddfa90332e

[menu] show numRequests also for origin & otherOrigins

this feature is now complete

also, now the flag will be shown in front of the origin(s) that flag is specific to that origin, so it might be that the "main" origin has no red flag, but an other origin.

There is now a new file: GUILocation.jsm. It contains several classes which are used in menu.js. The "Destination" class has been renamed to "GUIDestination" which is now located in GUILocation.jsm.

A nice possibility to test the display of other origins is Google Image search. Here's an exemplary screenshot: the menu

SkySkimmer commented 10 years ago

What does the flag do?

myrdd commented 10 years ago

What does the flag do?

it shows red if anything has been blocked for that origin. Suppose you have a rule allowing everything which google.de requests, the flag for google in the above image would turn gray.

nodiscc commented 10 years ago

Nice changes @myrdd

@SkySkimmer The idea was to have everything in the same Github repo for portability/accountability/integrity/authentication (code, issues, website, doc, releases one git clone away). You still have to run a backup script for issues, but this should be optional as long as the doc, FAQ, etc. is up-to date. You can use git hooks to do it automatically.

The released XPIs are attached to a git tag, so the committer can add a checksum to the commit for users to verify the integrity of the XPI. Unfortunately the XPIs on github can't be used for public releases (single-click installation not working, addon auto-update etc.) so it's not perfect, but once we have a signed XPI we could host the XPI anywhere (the original sslsites.de proposed by @myrdd would be ok IMHO, as long as the XPI is signed and matches the checksum in github tags).

The only thing still outside the repo is the wiki, and we could add a submodule to it in the main repo (git submodule add https://github.com/RequestPolicyContinued/requestpolicy.wiki.git), so you'd get it withgit clone --recursive`.

myrdd commented 10 years ago

once we have a signed XPI we could host the XPI anywhere (…) as long as the XPI is signed and matches the checksum in github tags

How do you mean this? So you would sign the XPI with the CA certificate and with GPG, with the gpg-signed being for github? We could upload the certificate-signed xpi on both places, no?

we could add a submodule to it in the main repo

adding the wiki to the repo sounds interesting – will this be for everyone who clones the main repo? Will the submodule be in all branches or just in one? (ok, master branch would be enough in general – as soon as 1.0 is released we will switch to master again.)

nodiscc commented 10 years ago

How do you mean this?

The release process would go like:

Changes are made to the source until you think it's releasable (say the latest commit is ed6c435405)
You build the XPI and sign it with the CA certificate, then do an sha512sum of it
Then git tag -u [gpg-key id] v1.0.0b8 ed6c435405 with the tag message This is release v1.0.0b8 with lot of bugfixes. SHA-512 sum of the attached XPI is eab1851a48ff5a27268ac6e786f3af8188fb23b86e94e16b3db06da819c4d2d82846747530137a97f060486a1a3bc3cde7ef402ef4affd7239d79b04c5947797
You create a Github release for the tag and attach the XPI to it.

By then:

The GPG signing proves the source has not been tampered with (you signed it)
The SHA 512 sum provides integrity verification for the XPI on github (in case someone wants to download it from here, or to check XPIs from other sources against it - this is really the XPI that's built from this tag)
The CA signing provides a secure auto-update mechanism when we host the XPI elsewhere (AMO doesn't support signed XPIs?? But we could host it anywhere as long as it's signed and can be checked against the GPG-signed SHA512 sum. For example you originally used https://sslsites.de/requestpolicy.256k.de/ which would be ok)

I hope I made it clear :/ Please tell if something doesn't make sense ...

will this be for everyone who clones the main repo? Will the submodule be in all branches or just in one?

It will be for everyone who uses git clone --recursive or does a git submodule update --init in their copy. Yes it's only in the branch you add the submodule to.

myrdd commented 10 years ago

Oh, thanks a lot @nodiscc for the detailed explanation. I think this is the way I'll do it. Would you like to see pre-releases signed, too? Well, I guess it's just a few clicks, so I could do it ;)

Yes it's only in the branch you add the submodule to.

alright, so go ahead and do that, if you'd like to @nodiscc :) we can do this either on master or on dev-1.0.

nodiscc commented 10 years ago

Pull request for the wiki as a submodule at https://github.com/RequestPolicyContinued/requestpolicy/pull/490, feel free to merge it if it's ok.

Would you like to see pre-releases signed, too?

If you could sign every future tag that has an XPI attached (and add the SHA to the tag message) this would be great! Thanks!

ldgbc commented 10 years ago

Found out about an add-on's call "Policeman" (https://addons.mozilla.org/en-US/firefox/addon/policeman) through an "ghacks.net" article. From the look of it, it purpose is similar what NoScript and RequestPolicy, however it look rather new and probably in rapid development, given a year it might be a contestant.

It already is restartless/jetpack, which NoScript is trying to achieve with it next major version, and also one of the "Issue" planned for RequestPolicy.

It also have a few other feature that RequestPolicy planned to have. Anyway, enough diagnosis, I wanted to ask what the contributor of RequestPolicyContinued think of it? It have potential if the developer continue to work on it. Currently is lack the ease of use but offer a strictness that surpass RequestPolicy current state.

From the look of the commit ever since the RequestPolicyContinued started, it seem like "myrdd" does the major of the work. If Policeman become "better", would you consider stop working on RequestPolicy and perhaps contribute to Policeman instead? It would lessen the work that need to be put in this project.

I would type more but this is already too long.

myrdd commented 10 years ago

hi @ldgbc thanks for your comment, I also stumbled on the ghacks.net article yesterday. I already recognized @futpib's work several months ago as he had made some changes to RequestPolicy (see here). As you can see from the commits there, his first step was to differentiate between different content types, which is also afaik the main feature of policeman.

However, I still don't know the roadmap for policeman, so it would be great to hear @futpib's comment on this. It would be great if the two projects could work together in any form. In fact I already thought about some kind of framework/library which could be used by RP, NoScript etc. but this shouldn't be within the scope of this discussion.

By the way, policeman is mainly written in coffeescript, but as it compiles to javascript, there's absolutely no problem. So working together and being in contact between projects would be great. However, having both RP and policeman could be ok or even good, i.e. it could be possible that @futpib has different plans than RP has, and users might be happy to choose between different addons. Once again, we need to hear what @futpib will say.

PS: By the way, most of the work I've done so far is refactoring. The current internal structure is not good for maintaining. I already did a lot, but there's still a lot more to do. Possibly @futpib started his own addon/rewrite because refactoring is a lot of work (?).

PPS: I'm currently working on restartlessness, and it looks good :) but it's indeed a bigger change.

@ldgbc go ahead and write more about your thoughts :)

futpib commented 10 years ago

Well, honestly, I don't have a definite roadmap (does RP have one?), but the addon is like an improved RP for me, so it will stay fairly similar to it anyhow. I think we can't get around duplicating effort here.

I have tried working on RP, but changes I had in mind were hard (at least for me) to incorporate into existing code base (you guessed it @myrdd ). It's actually more then filtering on content-type, it's a mini-language for writing kind of rules. On the other hand, RP has a huge user base that is happy with it. It's not like RP is bad or dead or something, so I wouldn't expect people to just leave it.

myrdd commented 10 years ago

FYI: I've updated RP's readme and [wiki](). I think it's a little better structured now.

Now my answer to @futpib

I don't have a definite roadmap (does RP have one?)

It's not written down :) But you could extract it from the issues that exist.

I have tried working on RP, but changes I had in mind were hard (at least for me) to incorporate into existing code base

I agree with you that RP currently is still bad for extending. Nevertheless I would be very pleased if we could merge the two projects again some day or at least share the code basis. As for now, I will continue maintaining RP and refactoring its codebase. You're welcome to join at any time! :)

myrdd commented 10 years ago

Version 1.0.beta8 is released. Already installed versions >= 1.0.0b4 will update automatically.

The release provides a SHA-512 sum and the tag itself is signed with a GnuPG key. I've just uploaded the key to a keyserver (hkp://keys.gnupg.net). The key ID is AE78FDFE, the user ID is Martin Kimmerle <dev@256k.de>.

nodiscc commented 10 years ago

congrats @myrdd thanks thanks thanks! What happened to the versioning number schema? 1.0.0b7 -> 1.0.beta8

myrdd commented 10 years ago

What happened to the versioning number schema? 1.0.0b7 -> 1.0.beta8

I wanted to have the word "beta" in the version string. Also I wanted to remove the last zero. Sadly "1.0beta8" was not possible, so it's now 1.0.betaN.

nodiscc commented 10 years ago

Thank you @myrdd for the very quick release for the fix of #514 I had tracked the problem down to yourpolicy.js but was unable to find what was going on (there are soooo many vars and no comments, and the firefox debugger doesn't seem to work on RP's chrome pages). If you can give some tips later on how to debug it would be nice.

myrdd commented 10 years ago

@nodiscc nice to hear you tried to find the problem. Yes, the code could have more comments.

To debug chrome code, you have to use the browser toolbox, see here: https://github.com/RequestPolicyContinued/requestpolicy/wiki/Working-with-the-Source-Code Feel free to extend the wiki or ask questions.

myrdd commented 9 years ago

FYI, I've changed from Apache Ant to GNU Make.

Now you can also run MozMill tests via make mozmill if you have set up everything correctly.

nodiscc commented 9 years ago

My GPG key id is 24D103D6. The fingerprint is BAFE 2502 80B2 8106 7D31 FA01 20EC 53B1 24D1 03D6. The user id is nodiscc <nodiscc@gmail.com>

myrdd commented 9 years ago

I did some work on the labels (recolouring). Also I created the page inspection label as I see potential in how RP could provide help to users in the question "Which of those domains need to be allowed/denied?". Maybe there will be some dedicated site inspection mode some time (after 1.0 of course). Maybe that mode could be even integrated into some of Firefox' developer tools (?).

Btw @nodiscc when you work on the issues, please remove a milestone from an issue if you close it as duplicate, wontfix etc. – for example like https://github.com/RequestPolicyContinued/requestpolicy/issues/141

myrdd commented 9 years ago

I've got a question to the community. Currently I've got a quite list of issues on my personal computer which don't exist on github. It includes bugs, features that I'm planning and ToDo's in general (e.g. refactoring). I'm not sure whether I should create an issue for each one of them on github. It would probably be a little more work than managing them on my pc, but on the other hand if it's online you can review it. What do you think?

Btw fyi, in Mozilla's source code repo every commit contains a link to a bug report, but I think that's not necessary for RP.

nodiscc commented 9 years ago

@myrdd you could paste your todo on https://gist.github.com/ and link it from here. gists allow comments and you can update them (they are just git repositories, you'll get a clone URL when you save the gist). And yes it would be interesting.

Ok for removing milestone for wontfix, duplicates, etc.

myrdd commented 9 years ago

That's a really nice idea @nodiscc, a public online scratchpad. :) Btw I'll translate/cleanup everything before creating the gist.

myrdd commented 9 years ago

In view of the current situation (Fx36 compatibility) IMHO it's a good time to add "donate" buttons. My idea was to add one button per contributor, meaning that there won't be a common account, but the user needs to decide to whom to donate.

Currently me and @nodiscc are candidates for the donation list, but anyone who seems to do any kind of significant and continuous contribution is welcome to add his/her donation button. What do you think @nodiscc?

Donations have originally been discussed here: https://github.com/RequestPolicy/requestpolicy/issues/407#issuecomment-52426453

nodiscc commented 9 years ago

I personnally don't expect donations, but ok. I'd start by adding a "Contribute to RequestPolicy Continued" paragraph in https://requestpolicycontinued.github.io/#about

You can help improving RequestPolicy Continued by testing, reporting bugs, improving code or documentation, or doing a small donation. How to contribute.

CONTRIBUTING.md should contain info on high priority bugs, how to report issues, debug, and donate to each individual contributor. For example you could add a homepage link to your github profile, and post a small text on how to donate specifically to you. This leaves the choice of the donation service to each contributor.

Another option is bug bounties (example). AMO also has a donation nag screen.

From experience don't expect too much from money donations :/

myrdd commented 9 years ago

From experience don't expect too much from money donations :/

No worries, I don't want to earn my living with that :) In fact I'd put donation buttons quite unobstrusively – no big images, not on top of any page, …

I agree on the contributing file, seems to be a de-facto standard. I've opened #542 for this.

IMHO bug bounties are too complicated, at least for the moment. If someone would like a feature being implemented, it's possible to bump a bug with the text +1 and that will be fully respected. A donation containing a comment like _„Please resolve issue #{issuenumber}“ is possible though.

nodiscc commented 9 years ago

A new open source browser fingerprinting tool: https://github.com/DIVERSIFY-project/amiunique (https://amiunique.org/). More up to date fingerprinting methods than panopticlick, which we link to in the doc, so I figured it might interest someone.

myrdd commented 9 years ago

Heya, this looks good! :) They list also some other sites on their faq. For example, I knew http://browserspy.dk already. IMHO panopticlick collects way too few data – much less than what is possible.

nodiscc commented 9 years ago

I've read the Disconnect addon (mentioned in https://github.com/RequestPolicyContinued/requestpolicy/issues/448#issuecomment-53158340) is now proprietary. I used to recommend/install disconnect for people who couldn't handle RP in default: block, but if we can

1) get blocklists from Disconnect into RPcontinued subscriptions 2) Have a simple first run page (https://github.com/RequestPolicyContinued/requestpolicy/issues/545) proposing default block/allow functionality vs default allow/block trackers modes

this would easily replace disconnect (and maybe ghostery)

Updating the website https://github.com/RequestPolicyContinued/requestpolicy/issues/445 is still on my list :/

myrdd commented 9 years ago

Disconnect is now proprietary? Hm, too bad! I've also installed it on some machines…

get blocklists from Disconnect into RPcontinued subscriptions

You want to do this? I'm focussing heavily on coding right now. RP 0.5 is marked as unstable with Fx 36, wich will be released in the late February…

Have a simple first run page (#545) proposing default block/allow functionality vs default allow/block trackers modes

Yes, I also consider a good initial setup page as important – however, my idea is to put the decision between default allow vs. default deny prominently into the menu, either in addition or instead of an "initial setup" page.

Updating the website #445 is still on my list :/

No worries, as long as the content of the current website is mostly up to date, I see no problem.

nodiscc commented 9 years ago

After looking at µBlock (a lightweight ad blocker), I think it implements some form of "local mirroring" of common js/css/font resources: https://github.com/gorhill/uBlock/blob/master/assets/ublock/mirror-candidates.txt. It seems to have been removed in a recent release though: https://github.com/gorhill/uBlock/releases

This (mirroring resources from common CDNs) was discussed at https://github.com/RequestPolicyContinued/requestpolicy/issues/267 for RequestPolicy

myrdd commented 9 years ago

That's interesting, I think it's a nice feature.

rezad1393 commented 9 years ago

hi i use both your addon and policeman addon together so that the requests get filtered first by RP addon then the allowed one get filtered by policeman addon. today when i was playing with RP(requestpolicy) addon policies something strange happened.from that moment on the requests go to policeman addon first then the allowed ones go to RP addon.but this was reverse before i somehow messed it up.

how can i fix this(reverse to old behavior?)

myrdd commented 9 years ago

Hi @rezad1393 if I remember correctly, the order is determined by the order of installation, so you couly try reinstalling the addons in the order you want, maybe that solves the problem?

Ah yes, I've read it there: https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Reference/Interface/nsIContentPolicy#shouldLoad%28%29

Note: When multiple content policies are used (for example through several extensions), if one of them rejects a request, the rest of the policies are not called. Reference: http://mxr.mozilla.org/mozilla-central/source/content/base/src/nsContentPolicy.cpp#146.

Note: The order of content policies in the above case depends on the precedence of installation.

rezad1393 commented 9 years ago

thanks man you were correct

rezad1393 commented 9 years ago

@myrdd it seems I reported too early. the workaround you said works at first but after a while it goes back to the reserve way(policeman first then RP addon)

what do I do? can I specify the order of addons that work on requests outside of installation order? cause that didn't last.

myrdd commented 9 years ago

@rezad1393 I could imagine that each time an addon gets updated it gets back to the top of the list. I guess this needs to be tested. Please open an issue, then I'll investigate when I have time for it. In the end I guess RP can't change anything at Firefox' behaviour; maybe this issue has been discussed in a Mozilla Bug – https://bugzilla.mozilla.org/

rezad1393 commented 9 years ago

maybe i didnt say this right.English is not my first language. i dont change these two addons after temporary fixes of uninstalling and reinstalling the policeman. but after using firefox for a while it goes to the the policaman->requestpolicy again.

this happens both 35 and 36 versions of firefox. i dont think its your addon bug (at least not in design) and i dont know what i did that made this happen. i just remember that it was after changing some of the requestpolicy setting this happened. i even wanted to revert to an old firefox profile from 3 weeks ago and somehow import my other changes of bookmarks and history from this period.

myrdd commented 9 years ago

Hi @rezad1393 ! To discuss this further, please open a new issue. It would be good to find out how to reproduce your issue. You could test whether you have the same problem when using a fresh new profile.

uninstalling and reinstalling the policeman

I found out that disabling and enabling policeman does the same regarding shouldLoad. This is at least a faster workaround.

rezad1393 commented 9 years ago

ok created an issue ticket here https://github.com/RequestPolicyContinued/requestpolicy/issues/603 and as i said there i created a new profile and just imported my bookmark and history and installed my enabled addons from old profile but that didnt fix it. i resume the discussion there if somebody is able to help me.

shirishag75 commented 9 years ago

First question, how does one create screenshot of the Request Policy as can be seen in https://github.com/RequestPolicyContinued/requestpolicy/issues/484#issuecomment-57916812 . I have an issue which I would like to explain using the screenshot for easier conversation.

I am on Debian GNU/Linux.

Till that one doesn't happen, I would give a starting picture of the issue at hand.

One of the sites I have trouble with is imgur.com . The site is good (as in needed and is useful) at vast of the sites where it is, but there is one entry in the Tools > RequestPolicy > Manage Issues which has me concerned . It says :-

Block       *.ads.imgur.com deny_trackers

Now I would like to have some more info. about this block but doesn't seem to be anyway to get that as on which site I saw and denied. Also once the deny_trackers is set, there doesn't seem to be a way to delete it, is there ?

Looking for some more info. so as to take a call.

rezad1393 commented 9 years ago

shirishag75

this are rules added by the addon itself that are called subscriptions. they are not added by you manually. think of them as groups of rules that serve a specific purpose. like in your provided example that is for blocking advertisement on imgur website. more info here

https://github.com/RequestPolicyContinued/subscriptions

shirishag75 commented 9 years ago

@rezad1393 thank you. That makes sense BUT this now puts me in a fix. I have a site (meta.unix.stackexchange.com) where the imgur.com is there in allowed destinations but with a question mark and I have no idea why the question mark is there.

Now that issue seems to be related to this issue which I raised on https://meta.stackexchange.com/questions/250599/unable-to-login-at-times-on-meta-unix-stackexchange-com which takes me to https://meta.stackexchange.com/questions/56161/which-browsers-are-officially-supported-and-what-else-do-i-need/71208#71208

Out of them, most of them work except for two, the first I have shared about imgur and the second one is the automated one about network logging at https://meta.stackoverflow.com/network-login-help :-

Messaging failed

Your browser does not support the HTML5 postMessage feature, or it is disabled. Please make sure that you have a reasonably modern browser (e.g. IE8+, Chrome 4+, Safari 4+, FireFox 3.6+, Opera 10.61+), and that you don't have cross-document messaging disabled in your browser settings.

nodiscc commented 9 years ago

how does one create screenshot of the Request Policy

Your linux distribution should have a screenshot tool installed; I use xfce4-screenshooter but this may be gnome-screenshot depending on your Desktop Environment. Capture the whole screen and crop the screenshot using an editor like GIMP

About the Imgur issue, you may want to attach a screenshot because it's not very clear to me.

About the Stackexchange login issue, disable RequestPolicy and see if it fixes it. If yes, please report back.

Sorry for my lack of activity on RPC for some time.

nodiscc commented 9 years ago

Also once the deny_trackers is set, there doesn't seem to be a way to delete it, is there ?

This is indeed blocked by the "Deny trackers" subscription you subscribed to. There is no way to disable a particular subscription item for now, it's a known issue: https://github.com/RequestPolicyContinued/requestpolicy/issues/491

The Block *.ads.imgur.com deny_trackers rule should not block your browsing since it only block images from Imgur's advertisement subdomain.

shirishag75 commented 9 years ago

About the Stackexchange login issue, disable RequestPolicy and see if it fixes it. If yes, please report back.

There seem to be quite a few issues there.

It turns out that the login issue was failing due to another addon HTTPS Everywhere. Apparently, the stackexchange folks haven't done full https re-direction (whatever the word means) . Once I disabled the two rules for Stackexchange I was able to login to meta.unix.stackexchange without an issue.

BUT as far as https://meta.stackoverflow.com/network-login-help is concerned, the messaging failure issue is still there.

Messaging failed

Your browser does not support the HTML5 postMessage feature, or it is disabled. Please make sure that you have a reasonably modern browser (e.g. IE8+, Chrome 4+, Safari 4+, FireFox 3.6+, Opera 10.61+), and that you don't have cross-document messaging disabled in your browser settings.

And I have no idea what this cross-document messaging bit is and where it is in my Firefox 36.0 on Debian.

nodiscc commented 9 years ago

@shirishag75 Disable RequestPolicy and see if it fixes it. If yes, please report back. If no, this is not a RequestPolicy bug. I don't have this message on Debian, iceweasel 36.0-2.

rezad1393 commented 9 years ago

it is related to websockets i think. if you open firefox browser console you will see this error at that page loading "The connection to wss://qa.sockets.stackexchange.com/ was interrupted while the page was loading." this has nothing to do with this addon . and as far as i know nothing to do with other addon too. i tested in empty firefox profile too and it's there too

RequestPolicyContinued / requestpolicy

general discussion #484

shirishag75