HTTPS-protected web interface and fever when accessed remotely

[SkyCrawl]

Hello again,

in case anyone should wish to use Coldsweat remotely (over a non-secure or possibly snooping network/channel), I think HTTPS protection should be offered to the user. Since not every user will have an HTTPS certificate ready on their machine, it should likely be an optional feature triggered by the serve command. I'm thinking:

# offer both HTTP and HTTPS
python sweat.py serve -r --ssl

# offer only HTTPS
python sweat.py serve -r --ssl-only

# display a warning that the extra option will be ignored as it doesn't make much sense to use it on localhost
python sweat.py serve --ssl
python sweat.py serve --ssl-only

With that said, the config file should likely contain links to the appropriate certificate and private key but these options might as well be entered via command line (in which case they override the config file).

As for the implementation, it seems that a simple modification of the serve command might suffice.

If you'd like me to, I can try to have a go at this :).

[passiomatic]

I can see some value in this proposal. However, can we have just one option for this? Why did you include the --ssl-only option?

With that said, the config file should likely contain links to the appropriate certificate and private key but these options might as well be entered via command line (in which case they override the config file).

I would prefer not to pollute the command line with additional options for the certificate and key. It should be stored in the config file since they aren't likely to change often.

I would like to see a PR for this. If you want to work on this issue please fork the 0.9.8-wip branch.

[SkyCrawl]

However, can we have just one option for this? Why did you include the --ssl-only option?

The --ssl option would be used if the administrator would like to support both HTTP and HTTPS. This might make sense for many users with different requirements, or perhaps in terms of efficiency (as I understand it, every HTTPS Fever API call may impose certain overhead in some contexts). In other words, Coldsweat may lie behind a reverse proxy which may accept only HTTPS requests from external sources and HTTP requests from local sources. Still, such a reverse proxy may be susceptible to a misconfiguration or a hack.

The --ssl-only option would be used if the administrator is like me and would like to ensure certain level of security for everyone. From security's perspective, having both HTTP and HTTPS enabled for external access and all served sites is almost equivalent to having only HTTP enabled. For example, a MitM attacker may always downgrade an HTTPS request to an HTTP request and the user may not notice. Especially so if a client application (not the browser) makes requests to Fever API.

I think both options are important but in short-term, the --ssl is a bit redundant since this proposal is about --ssl-only :).

I would prefer not to pollute the command line with additional options for the certificate and key. It should be stored in the config file since they aren't likely to change often.

Let's leave them behind then :). They were only intended as a possible convenience, albeit almost noone would likely ever need them.

[SkyCrawl]

@passiomatic Currently, I'm in the process of implementing this but I've encountered a slight problem...

A https://127.0.0.1:3333/login call is very successful but the login form's action attribute still points to http://127.0.0.1:3333/login. I've traced the problem to the corresponding template's {{application_url}} placeholder, which seems to be derived from WSGI's environ dictionary. More specifically, the dictionary has a value for wsgi.url_scheme, which seems to be guessed by this function. I've checked and there is no value for HTTPS in the dictionary - that's most likely the root cause for the issue.

However, even if we discover a way to fix the problem, it seems like WSGI server is not capable of hosting both HTTP and HTTPS with a single server instance. Creating two instances (using the make_server function) may be a solution but I don't think the server has been designed for this. Please correct me, if I'm wrong :).

Instead, I've toyed with the idea of using a different WSGI-compatible web server - one more suited for the task at hand. After reading this comparison (more information over here) and investigating a little further, I do believe that CherryPy is the best choice at the moment. Also, if we look over here, we can see an example of what I spoke moments ago.

So, I'd like to ask for your direction/approval :). While I'm at it, I might as well incorporate Bottle (or perhaps flask) for some more abstraction/control/robustness? But I think our opinion is the same: no extra dependencies unless necessary.

EDIT:
If we look over here, it says Python 3.1+ compatibility so CherryPy shouldn't form any obstacles to #109 .

[passiomatic]

Hi again. All this sounds very painful. We are close to a 1.0 release and the last thing I want is a breaking change of this magnitude.

The local web server was meant to make the bootstrap process as easy as possible. HTTPS is a very desirable feature but I'm wondering if it is better achieved by putting Coldsweat behind Apache or another "real" web server.

Does the shortcoming of WSGI server you discovered kicks in also when Coldsweat is behind of Apache using FastCGI or Passenger? This would be quite sad.

[SkyCrawl]

We are close to a 1.0 release and the last thing I want is a breaking change of this magnitude.

I understand but personally, I don't think there will be any serious impacts :). One of the goals for WSGI was to allow applications to be easily ported to a different web server. As for Coldsweat, I only see us changing the command_serve method (I've done that already anyway) and a handful of lines elsewhere. In my opinion, it is worth a try.

HTTPS is a very desirable feature but I'm wondering if it is better achieved by putting Coldsweat behind Apache or another "real" web server.

That's definitely a valid point to raise - yesterday, I've actually wondered myself. For maximum security, Apache is the choice to make. But my experience is that Apache is terribly inconvenient for people, who don't know it yet. As for me, I've encountered it many times and I always gave up trying to use it because I never succeeded in making it work... But then, I may be a little biased because since the second time (I think), I developed something of a grudge against Apache :p.

Generally though, I do support the idea that reverse proxies are the best solution in practice and as such, it may be sufficient for Coldsweat to only support HTTP. In the end, the only question is probably whether you would like to make this quick and convenient for the users (no external software and configuration needed), should they prefer :).

Does the shortcoming of WSGI server you discovered kicks in also when Coldsweat is behind of Apache using FastCGI or Passenger? This would be quite sad.

I haven't tested that yet. But I think we can fix the WSGI server's shortcoming anyway - in the worst case, we can try to override the guessing function. A slight complication is that only HTTPS would be allowed but then, the --ssl option (both HTTP and HTTPS) has never been a priority anyway :).

SUMMARY:

1. For now, the best compromise seems to be keeping WSGI server and trying to at least support --ssl-only.
2. The most secure option is to test with Apache & mod_wsgi but most importantly, to make a complete guide for people who are not familiar with Apache yet or find it unnecessarily complicated or user-hostile.
3. And finally, I think that the best option all around is to move to CherryPy (or another web server, more capable than the default one) because that way, we should be able to reap all the desired benefits. The default configuration of Coldsweat only allows HTTP so whatever the actual web server, they won't hinder Apache or another reverse proxy in any way.

That said, I await your final judgement :).

[SkyCrawl]

UPDATE:
I figured out how to fix the WSGI problem and solution nr. 1 (see my last post) is almost finished :).

[jeroenh]

Great stuff, just a minor tidbit: if you do plan to introduce this, could you please use TLS as the argument name? TLS has replaced SSL somewhere in the last century, and is currently on its way to become completely unsupported.

[SkyCrawl]

Despite the term SSL is commonly used to denote SSL/TLS, or even TLS alone, thank you... a good tidbit :). I'll change the parameter's name to --https-only in my PR.

[oz123]

This is actually a bad idea. See similar discussions with flask and bottle, the sweat command should not be used to serve in production, but rather help developers. Deployment should be done with modern wsgi server preferably with nginx or another webserver as a proxy with SSL in front.

[SkyCrawl]

In most cases, perhaps. I've never meant to push this solution at any cost (especially if redirection is problematic), but at the same time, I disagree with the statement:

This is actually a bad idea.

If you're like me and don't need/want to deal with one more technology you're not familiar with, this solution may suit your needs more. I agree that it is probably not as secure and mainstream as a battle-hardened proxy but personally, I wouldn't go that far in my closed environment anyway as I've got too many devices, operating systems and software products to maintain already and don't need any more :).

Nevertheless, thank you for a great warning! It certainly deserves to be here.

[passiomatic]

Closing this, given that now Coldsweat is a wheel/Flask app that can deployed in a various of ways (Apache, Nginx, GUnicorn, Waitress, etc.) and HTTPS is handled (and documented!) better by those software.