implement https

[teusbenschop]

Implement secure connections on client and server.

[alerque]

This isn't the same issue, but I'm bringing it up here because it would be good to have in mind while implementing this: the current architecture that involves constant poling from the client needs to be replaced with long lived web socket connections. In choosing SSL libraries to deal with this issue, keep an eye out for how good their wss:// support is so that it will be easier to take care of this in the future.

[teusbenschop]

This would affect the server only (because all clients use browser widgets which Bibledit has no control over what SSL libraries they use). If there's a C++ SSL library that compiles across the full range of supported devices, and does not have wss:// support, a option would be to use a more standard SSL library for the server only, e.g. OpenSSL or what else. (I am writing this to not forget :) )

[teusbenschop]

Uhm, correction: It affects the clients also because they also have embedded servers... :)

[alerque]

The actual TLS/SSL negotiation involved in web sockets isn't any different that vanilla HTTP, so that bit really has as much to do with the web server libraries as much as it does encryption, but since they all play off of each other it's worth keeping in mind when picking libraries. And yes with the current architecture it will need to work across all platforms.

[alerque]

Cross-link related issue #94 and discussion on the user group.

[teusbenschop]

When all client ports have a server that works with plain http, on http://localhost:nnnn, as they do now, all works well. When all client port were to run secure http on localhost, like https://localhost:nnnn, then the client browser widgets likely throw up security warnings, as the browser component cannot verify the server certificate for localhost. I fail to find a way to obtain a server certificate for localhost. Yes, there is a way as explained on https://github.com/Daplie/localhost.daplie.com-certificates but that is just a hack. It is not a proper certificate for localhost. And in case the client works offline, then the client cannot even verify whether the certificate can be trusted, because it fails to look up the DNS for localhost.daplie.com. Since I am failing to see a proper solution for this, I am inclined to run a plain http server on localhost. But my point for writing this comment really is to ask for help since perhaps I am overlooking a few things.

[alerque]

Running on localhost over plain http is definitely a much smaller problem in the grand scale of things than the public server instances are. I would, however, default to binding the daemon to 127.0.0.1 rather than 0.0.0.0. If people want to change the configuration to bind to listen on their LAN without setting up a cert that's a choice they can make, but if they are just running what they think is offline client software on their own PC it should default to only talking to itself. Binding only to the loopback interface cuts most of the DNS spoofing and ARP poisening attacks you can do on a local network off an the knees. Its not perfect protection but the attack surface becomes pretty small and starts becoming dependend on a compromised local machine rather than any old network evesdropper.

There may be a way to do this better architecturally, but off the top of my head I'm not coming up with it. If we were actually writing out own client there are ways to do TLS handshakes without registered certs, but we're using the browser so we're limited by that ecosystem. Ther might be a way to do this with Electron, but I haven't looked into it yet. I was meaning to play around with compiling an Electron client after I got all the current per-platform builds working, but I'm not there yet.

[teusbenschop]

I'll make a note to bind to 127.0.0.1 for client configurations (while allowing the server to bind to 0.0.0.0).

Yes, this solution looks to be secure enough. And if the local device is compromised, then no network security of any type is of much use.

An advantage of using plain http on low power devices is that it requires less processing because the handshake iterations and the encryption and decryption routines are left out of the loop. While developing the secure sever and testing on https://localhost, the response of the server already feels slightly slower than the response of the plain http server.

[alerque]

Yes the round trip time will be a few ms slower on low power devices. But we can more than make up for it by not having the UI hang around waiting for such requests in the first place. For example the time saved by not having to load a full intermediate page to get to any menu item will be an order of magnitude more than anything lost in round trip time. Not polling every minute and using events instead will save an order of magnitude more battery than the SSL overhead. Using Websockets will eliminate both sets of issues—the handshake only happens once and the channel stays open for further communication (so no round trip delays even over a secure link) and becomes event driven (so no polling loops have to run at all).

[teusbenschop]

Yes, the elimination of the polling will improve the performance on low power devices. But the long lasting connections through Websockets would require some special configuration. The issue is that the server now listens for connections initiated by clients. Once a connection with a client has been established, there is a reading timeout on the socket of 60 seconds. The reason is that sockets do not remain open indefinitely in case the clients do not close their connections. This scenario may happen on poor network connections, I imagine, where not all packets sent by the client make it to the server, and so packets that close the connection may be lost. This means that the timeout on the server is needed to close connections, rather than leaving them open forever, which would exhaust server resources if there's enough of them.

[teusbenschop]

The difference in the timing between plain and secure http for fetching a resource is considerable, as measured five times on a 2010 Mac Pro with Google Chrome: 'http://localhost:8080/jquery/jquery-1.11.3.min.js: 45-65 ms 'https://localhost:8081/jquery/jquery-1.11.3.min.js: 60-80 ms

[teusbenschop]

The reason for the above difference likely is that the secure http library sends the data in chunks of max 16384 bytes a time, so the jquery resource took six chunks. The plain http server sends the whole lot in one go. Not sure why the library sends it in chunks, the library API indeed mentions this, so it looks normal behaviour by design.

[alerque]

This sounds like something is miss-firing—the difference shouldn't be quite that much. I don't have time to research it right now but off the top of my head that kind of chucking sounds like something you would do to optimize a pipeline that might have hardware accelerated encoding going on but maybe the acceleration part wasn't kicking in or isn't available on your test platform. In any case the chunking may or may not be related to the slowdown. Were you able to get a metric on response time for the request rather than just the overall download time? As in how long does the handshake part take before it starts transferring content?

Also something I meant to mention earlier, but I noticed while debuging something else that HTTP 302 headers were not getting sent at least some of the time when they should have been. Reducing a) the number of total requests and hence round trips and b) re-transfer of resources already sent are two easy ways to speed the whole thing up.

[teusbenschop]

This is an interesting one, and indeed worth looking into: How long do the individual stages of the transfer take. When I've got the whole secure server going, this is something that, when time permits, probably can be optimized in some way. It is now using the TLS library as it is, that is, as per the examples included in the library. Resources like jquery code is sent to the browser once, because browser caching is supported, so it is not re-transferred until the cache expires. Which means, in practise, that sending bits in chunks is not going to happen very often, and therefore perhaps this chunking does not have a noticeable impact.

[teusbenschop]

Secure browser communications is now possible, and can also be enforced through a server setting, see commit https://github.com/teusbenschop/bibledit/commit/df7dd5d51eb13c6680609d4abdd8c2096e6724b1

[teusbenschop]

While the Bibledit client now checks the common name of the server certificate, whether it matches with the expected common name, I am inclined to provide an option on the client to skip this check. When skipped, it still communicates securely (invisible to the outside world), but the client won't be 100% sure it communicates with the intended server. What do you think?

[teusbenschop]

Both cloud and server now support https, as of https://github.com/teusbenschop/bibledit/commit/32a1b6245897cbf606deb55fae86e10ece799bbe.

[teusbenschop]

This cloud edition does not yet have matching clients. Or to say the same thing in less flattering language: this cloud edition breaks compatibility with existing clients :)