Add support for encryption

[Sp3ctr3]

This would involve fetching a public key from the server, generating an AES key encrypting the request body with it, encrypting the AES key with the public key and sending it to the server. We're still implementing the public key dispatch at the server side though.

[Sp3ctr3]

Currently we're planning to use RSA 4096 for key generation and AES 256 for content encryption. Here's a flowchart:

[Sp3ctr3]

I've implemented encryption at the server side and it goes like this: The moment the client connects to the proxy server it gets a public key (even before it starts to send anything). The server expects the client to generate a 128 bit AES key, encrypt the contents of the request, encrypt the AES key itself with the server's public key and append the encrypted AES key to the beginning of the message and send it to the server. Is this possible on the client side extension or do I have to change the implementation? Also you can send an initial bogus request so as to get the server's public key. This will ensure that the very first real request is not intercepted. @ZDroid Any comments?

[zlatanvasovic]

@Sp3ctr3 looks good to me. Will we have a key on end of connection, like on top of it?

[Sp3ctr3]

I didn't understand what you meant. The key will be exchanged only at the beginning of the connection.

[zlatanvasovic]

Aww, ok. :D

[juzerali]

I think JavaScript implementations of encryption and hashing algorithms do exist. However they will be blocking and slow due to JavaScript's inherent slowness. Pushing the encryption task to the worker will help a little. My 2 cents.

[darkowlzz]

@Sp3ctr3 are there any docs which talk about how to setup a local instance of the server and play around with it? Or could you create one?

[darkowlzz]

After going through a lot of reads and searches, I found this add-on, Tamper Data. I think we could learn something from it on how to grab the requests and encrypt the content before sending it out.

https://addons.mozilla.org/en-US/firefox/addon/tamper-data/

[nb333]

@darkowlzz thanks! @sp3ctr3 source code here: http://tamperdata.mozdev.org/source.html

[alexa-infra]

I don't think that overriding browser requests in extension code is a good idea.Because it requires separate code for each browser, also it involves many and many hacks, and IMO this is not very stable operation. I think about local proxy application. Browser extension will just setup a local address of proxy (like 127.0.0.1:8000) - it will be always the same, and this local proxy will dial with OpenFaux server. In this way it is guaranteed that a local proxy get 100% of browser requests. And these requests will be "raw" data (browser independent). So local proxy will encrypt and send this raw data to server and etc... Local proxy could be in python (as the server), and be packed by py2exe or analogues. It will decrease browser-and-platform dependent code, and simplify things in general. But it will complicate installation process for user. What do you think about this solution?

[zlatanvasovic]

Good point. Let's put ad-blocking in api (client-side) layer then.

2014/1/4 Alexey Vasilyev notifications@github.com

I don't think that overriding browser requests in extension code is a good idea.Because it requires separate code for each browser, also it involves many and many hacks, and IMO this is not very stable operation. I think about local proxy application. Browser extension will just setup a local address of proxy (like 127.0.0.1:8000) - it will be always the same, and this local proxy will dial with OpenFaux server. In this way it is guaranteed that a local proxy get 100% of browser requests. And these requests will be "raw" data (browser independent). So local proxy will encrypt and send this raw data to server and etc... Local proxy could be in python (as the server), and be packed by py2exe or analogues. It will decrease browser-and-platform dependent code, and simplify things in general. But it will complicate installation process for user. What do you think about this solution?

— Reply to this email directly or view it on GitHubhttps://github.com/openfaux/openfaux-client/issues/17#issuecomment-31578058 .

Zlatan Vasović - ZDroid

[darkowlzz]

@ZDroid ad-blocking ? Why are we dealing with ad-blocking?

[zlatanvasovic]

@darkowlzz I though about adblocking and encryption, uh. :D Why ad blocking? openfaux/openfaux-server#30.

[Sp3ctr3]

I thought about a local proxy server too. If you can accomplish that it'd be great. The current code can be run simply by running python server.py but it doesn't have support for encryption. I'll add a branch for encryption and post the code shortly.

[Sp3ctr3]

Here is the encryption enabled server: https://github.com/openfaux/openfaux-server/blob/enc/server.py

[darkowlzz]

If have a local proxy server, any idea how this server would get the OpenFaux remote server address? A central server with a list of all the nearby OpenFaux server address? Where people who would like to help expand the openFaux network are allowed to add their own OpenFaux server address too?

[Sp3ctr3]

Currently we are planning it in such a way that a user can host his own server OR he can utilize one of the multiple servers operated by OpenFaux.

[xykivo]

If we plan on the end user to use a local server, or a publicly available OpenFaux server, I think we will need to provide an extension that will direct all browser traffic through the proxy server. (Similar to the TOR extensions).

[darkowlzz]

@xykivo IIRC we already have a proxy switcher in our client add-on which directs the browser traffic through the proxy server.

[boxtown]

What I don't understand is, if the client machine is doing the encryption (in browser or not) then what is the purpose of the server?

[Sp3ctr3]

@boxtown The client encrypts the content for the requests which is then decrypted by the server. The server then encrypts responses which is then decrypted by the client.

[juzerali]

@alexa-infra Chrome/Chromium+Opera has a very well defined, and stable API for preventing requests. In fact they both have exactly the same API if I am not wrong. Firefox also provides an API for intercepting requests. The API is different and that is why we have the api branch. Here is where we will write an adapter that will provide a chrome like API for FireFox and jetpack too. I am not sure if Safari allows extensions to intercept requests. I have opened a question on StackOverflow. These APIs are not hacks. Also IMHO there is no reason to doubt the stability of these APIs.

As to running a server locally, it will be horrible from User Experience's perspective, although it will surely save a network round trip. Technically adept users who wants to do it can install it and create a proxy using the loopback IP address, but we should not burden those who don't want to do it.

[juzerali]

We can also encrypt the content using JavaScript implementations of encryption algorithms. We can even write them if they don't exist. We will just have to copy Java implementations shamelessly. But as I said it will be blocking and slow.

[alexa-infra]

@juzerali Yes, I know that it is possible. btw IE has the same ability to hook/rewrite web request (via OnBeforeNavigate event). Note that in this way we don't see full (final) http-request/http-response - not all headers are provided, no information about cache, etc.. So redirecting request in this way is not totally correct. Also there is a problem with conflicts between extensions: user could install several extensions for filtering/intercepting requests, and in other words - anyone could redirect request "after" our extension.

Tor uses local service as far I know, and it does not stop users. Moreover local proxy is easier in use - user doesn't need a browser extension if he knows how to set proxy in browser, Also he could set up this proxy even on iphone.

[xykivo]

Both Safari and IE have addons. These are similar to Chrome and Firefox extension. I'm not sure if these addons allow routing all traffic through a proxy (local or not).

See http://www.iegallery.com/Addons http://safariaddons.com/en-US/safari/

Note IE is notorious about addon backward compatibility. Older version of IE (8 and below if I remember correctly), do not have addon support.

[juzerali]

@alexa-infra What do you mean by not all headers are provided and no information about cache? For chrome at least we can use chrome.proxy api.

[alexa-infra]

@juzerali from Chrome docs http://developer.chrome.com/extensions/webRequest.html:

Note that the web request API presents an abstraction of the network stack to the extension.
Internally, one URL request can be split into several HTTP requests (for example to fetch individual byte ranges
from a large file) or can be handled by the network stack without communicating with the network.
For this reason, the API does not provide the final HTTP headers that are sent to the network.
For example, all headers that are related to caching are invisible to the extension.

The following headers are currently not provided to the onBeforeSendHeaders event.
This list is not guaranteed to be complete nor stable.

Authorization
Cache-Control
Connection
Content-Length
Host
If-Modified-Since
If-None-Match
If-Range
Partial-Data
Pragma
Proxy-Authorization
Proxy-Connection
Transfer-Encoding

As far I know, this process is the same for other browsers.

chrome.proxy also does not allow an interception of http requests (so it could not be used to rewrite/encrypt/etc http requests)

[juzerali]

Okay. That clears up things. FoxyProxy uses chrome.proxy in Google Chrome though, from their extensions page Uses Chrome Proxy API instead of changing system proxy settings.

What is the plan then?

On Mon, Jan 6, 2014 at 3:17 PM, Alexey Vasilyev notifications@github.comwrote:

@juzerali https://github.com/juzerali from Chrome docs http://developer.chrome.com/extensions/webRequest.html:

Note that the web request API presents an abstraction of the network stack to the extension. Internally, one URL request can be split into several HTTP requests (for example to fetch individual byte ranges from a large file) or can be handled by the network stack without communicating with the network. For this reason, the API does not provide the final HTTP headers that are sent to the network. For example, all headers that are related to caching are invisible to the extension.

The following headers are currently not provided to the onBeforeSendHeaders event. This list is not guaranteed to be complete nor stable.

Authorization Cache-Control Connection Content-Length Host If-Modified-Since If-None-Match If-Range Partial-Data Pragma Proxy-Authorization Proxy-Connection Transfer-Encoding

As far I know, this process is the same for other browsers.

chrome.proxy also does not allow an interception of http requests (so it could not be used to rewrite/encrypt/etc http requests)

— Reply to this email directly or view it on GitHubhttps://github.com/openfaux/openfaux-client/issues/17#issuecomment-31637001 .

Regards, Juzer Ali http://about.me/juzerali

[alexa-infra]

I'd like to choose a local proxy (because it's more solid solution in terms of development), but as you said, it could be very bad in terms of UX, I'm totally agreed. Interception of web requests in browser extension would be better for user, but we will have to deal with random bugs (for example: some website does not work, because we do not handle full request data).

Also local proxy solution will take a longer time to develop IMO (but it will handle all browsers in the same time). Interception of web requests could be developed faster for particular browser, and it could be good for quick demo. I don't know what is in priority here.

What do you think?

[juzerali]

@alexa-infra It will be interesting to know how FoxyProxy is doing it? I think it is a pretty stable proxy client. Or are you implying that it is not very stable?

[alexa-infra]

@juzerali FoxyProxy sets up a proxy server in browser (and this proxy server is an usual web proxy server) or vpn connection in system. It doesn't perform any kind of encryption to http request directly.

As far I understand the main goal of OpenFaux is an encryption of web browser traffic. It could not be solved only by an usual web proxy server (because it does just a blind redirection). Also we cannot hack default browser protocol with proxy servers (there is no way to force encryption on connection with proxy server).

To achieve this goal we need to get an outgoing http request somehow (by local proxy, or by request interception), encrypt it and send to OpenFaux server. The similar thing could be achieved by ssh-tunneling (local proxy + ssh tunnel to your server with real proxy), or by vpn connection. In these ways outgoing traffic from user machine will be 100% encrypted.

[juzerali]

So the our primary blocker is encryption inside the browser and not proxying. There are javascript implementations but that would be slow. If we need to do it inside browser then only option is NPAPI plugins.

If we are running a local server then we don't need extensions at all, we just need to proxy our browser to locally running OpenFaux server. I think this bug should be resolved as soon as possible to have clarity on the purpose of this project.

I meant OpenFaux-client repo.

[alexa-infra]

Yes, you're right. So there are 3 different options now:

• Browser extension (to intercept requests) + js implementation of encryption
• Browser extension (to intercept requests) + NPAPI plugin (native system code) for encryption
• Browser extension (to setup proxy) + local proxy server

There are pros & cons and very different development approaches for each option. But basically we should choose one item from the list: UX, performance, or development speed. @nb333 maybe you have some comments?

[juzerali]

The third option will work only in chrome since only chrome provides proxy api. We can even do away with browser extensions in that case.

On Tue, Jan 7, 2014 at 6:17 PM, Alexey Vasilyev notifications@github.comwrote:

Yes, you're right. So there are 3 different options now:

• Browser extension (to intercept requests) + js implementation of encryption
• Browser extension (to intercept requests) + NPAPI plugin (native system code) for encryption
• Browser extension (to setup proxy) + local proxy server

There are pros & cons and very different development approaches for each option. But basically we should choose one item from the list: UX, performance, or development speed. @nb333 https://github.com/nb333maybe you have some comments?

— Reply to this email directly or view it on GitHubhttps://github.com/openfaux/openfaux-client/issues/17#issuecomment-31734462 .

Regards, Juzer Ali http://about.me/juzerali

[boxtown]

It seems we are leaning away from running a local proxy server because it'll drive away people who don't know how to/are too lazy to set up a proxy on their browser?

Also, with regards to JS encryption slowness, that shouldn't be a factor. JS's inherent slowness is only with regards to compiled languages like C/C++. The speed difference is not going to be noticeable to humans. This is compounded by the fact that, if it's AES we are doing client side, AES was DESIGNED with speed in mind. Not knocking on NPAPI plugins, just saying, no need to shy away from JS.

[admwx7]

Would using NPAPI over JS give any other added benefit aside from speed? Making the switch would cost us the ability to reuse the code for multiple browsers since there is no standardized toolkit that's actually used by all browsers. I agree with @boxtown about the "slowness" of JS, it's actually rather responsive when you think about it, we can benchmark and see how big of an impact it will be on the user if we need to.

[juzerali]

This sounds like good news. If JS based encryption isn't noticeable then we will at least have a start with intercepting the requests.

[admwx7]

I've done some JS encryption and decryption before for creating connection calls to OAuth services for users in real-time without a noticeable impact, but that was meant to be working in the background while the user was able to navigate around the page so it never had to be benchmarked since it was never blocking...

[alexa-infra]

For small request/responses (like ajax) difference in performance should not be visible at all. For big data (like images, files etc) or big number of concurrent requests it could be noticeable by user. Also keep in mind that it could be quite tricky to optimize encryption algorithms in JS (possible, but not-trivial), and it may become a bottleneck on some stage of development.

[admwx7]

In that case I could see a possible issue in the future for large data groups. Maybe we can mitigate it by adding in a image downsizer since images will be our biggest culprit I'm sure. As for NPAPI (http://developer.chrome.com/extensions/npapi.html), it's not an option unfortunately. If the limiting factor comes to be JS processing client side then for the performance (and more techy) minded members we can always develop a proxy based client application as well. We're not restricted to only one client implementation, but JS will allow us rapid development for multiple browsers and will hit the UX goal for our less technical brethren, while (shamelessly) promoting our product on app stores.

[juzerali]

What about videos?

On Wed, Jan 8, 2014 at 9:49 PM, Austin Murdock notifications@github.comwrote:

In that case I could see a possible issue in the future for large data groups. Maybe we can mitigate it by adding in a image downsizer since images will be our biggest culprit I'm sure. As for NPAPI ( http://developer.chrome.com/extensions/npapi.html), it's not an option unfortunately. If the limiting factor comes to be JS processing client side then for the performance (and more techy) minded members we can always develop a proxy based client application as well. We're not restricted to only one client implementation, but JS will allow us rapid development for multiple browsers and will hit the UX goal for our less technical brethren, while (shamelessly) promoting our product on app stores.

— Reply to this email directly or view it on GitHubhttps://github.com/openfaux/openfaux-client/issues/17#issuecomment-31849274 .

Regards, Juzer Ali http://about.me/juzerali

[admwx7]

It will have the same issue as videos with a much more complex solution needed, either way it will be an issue we'll have to come up with a solution for at some point. In the case of videos we may have to resort to direct connections bypassing our service until we can find a suitable solution.

[xykivo]

I think that if there is a concern about performance for large data groups (video, images, audio) then proxy is our best option. I think JS optimizations will only carry us so far. On the other hand a proxy solution can be optimized by either moving to a native solution (C/C++) or using compiled python.

This might be a stupid question, but do we have a use case scenarios for a non techie user? (Installation, normal use, enable/disable, unisntall, different OSs)

Another stupid question - most web traffic these days comes on mobile. Do we have plans for mobile support?

[zlatanvasovic]

Don't worry about JS speed, I can speed it up. :D

2014/1/9 Dror Smolarsky notifications@github.com

I think that if there is a concern about performance for large data groups (video, images, audio) then proxy is our best option. I think JS optimizations will only carry us so far. On the other hand a proxy solution can be optimized by either moving to a native solution (C/C++) or using compiled python.

This might be a stupid question, but do we have a use case scenarios for a non techie user? (Installation, normal use, enable/disable, unisntall.)

— Reply to this email directly or view it on GitHubhttps://github.com/openfaux/openfaux-client/issues/17#issuecomment-31895916 .

Zlatan Vasović - ZDroid

[admwx7]

@ZDroid Sounds good.

@xykivo We want to avoid only having a proxy solution for our less techie users. The server will be designed so that we can create a proxy based client for those that want to do a full setup to optimize performance. Basing the client design as a browser plugin allows us to ignore their OS and just have to support their browser. Mobile support falls into the same category as the proxy design. We'll have the server setup and waiting for requests in a set format, any client can make those requests so it would just be a matter of designing a mobile app to redirect all traffic on a mobile device (probably just WiFi traffic? Or maybe have this configurable?) to our server and handle all responses for the mobile device. I haven't looked too into mobile data management apps so I'm not completely sure we can fully override how a mobile device handles it's data transmission but if all else fails we just need to get big enough to get native browser support for Chrome or build our own mobile browser with built-in support ^.^