api to bind over SSL and TCP sockets

[benoitc]

Following #2120, I was thinking we should first discuss the API before starting any code. I do not like indeed that much the introduction of bind-no-ssl that much as it adds a new command. Also I do think we should keep TCP (no ssl) binding as the default.

I propose instead to introduce the prefix ssl:// just like we have the unix:// prefix for unix sockets. So for example binding to SSL would bethe following command line :

gunicorn --bind ssl://127.0.0.1:8443

Thoughts?

note we may also want later to add different connection profile but the api can be done separately and a connection named like this : ssl://profilename@127.0.0.1:8443 but we don't neeed it for that change.

Thoughts?

[dumblob]

If this will cope well with (future) support for SNI in gunicorn (i.e. more certificates in an SSL connection depending on the sni_callback), then I'm all for it.

Or maybe we could discuss also the interface (command line args, etc.) for the following use case: a web server running both SSL and non-SSL on the same IP address and on the same port while serving more SSL certificates based on the choice given by a Flask application (based on SSLContext.sni_callback).

[benoitc]

@dumblob does it requires any change today to run on SNI? Can't a middleware upgrade the connection from Gunicorn using the socket in the environ ? Gunicorn provides gunicorn.socket for such purpose.

[benoitc]

@dumblob does it requires any change today to run on SNI? Can't a middleware upgrade the connection from Gunicorn using the socket in the environ ? Gunicorn provides gunicorn.socket for such purpose.

hrm we need first to have SSLContext but once done i am not sure it needs more

[dumblob]

hrm we need first to have SSLContext

Definitely.

but once done i am not sure it needs more

Should probably suffice, but I'd leave this partially open as I'm not enough versed in the interaction between Gunicorn and an arbitrary app (Flask was just an example).

[javabrett]

I'm going to toss the question of https:// vs ssl:// bind prefix in the mix. I haven't yet done an assessment of what is more commonly used in bind-apis, CLIs etc. https:// is the actual protocol, but we (and other servers) do use ssl readily also although maybe not as commonly represented as a protocol?

[benoitc]

I'm going to toss the question of https:// vs ssl:// bind prefix in the mix. I haven't yet done an assessment of what is more commonly used in bind-apis, CLIs etc. https:// is the actual protocol, but we (and other servers) do use ssl readily also although maybe not as commonly represented as a protocol?

i would prefer to stick on the transport protocol for the prefix (ie. ssl or unix) there. That makes things more clear imo especially since http can be done over the unix or tcp socket as well :)

[darakian]

Can an argument be added to enforce ssl only operation as is the case today?

[benoitc]

Can an argument be added to enforce ssl only operation as is the case today?

what do you mean?

[darakian]

Sorry, I think I misread this when I posted that comment. What I wanted was a flag to ensure that only TLS connections would be accepted. I think I read this post to read that the work here was to make a bind on any protocol ie. http

[hynek]

Prior art for this would be Twisted's endpoint syntax:

ssl:port=443:privateKey=/etc/ssl/server.pem:extraCertChain=/etc/ssl/chain.pem:sslmethod=SSLv3_METHOD:dhParameters=dh_param_1024.pem

Check out https://docs.twistedmatrix.com/en/latest/core/howto/endpoints.html#servers and https://docs.twistedmatrix.com/en/latest/api/twisted.internet.endpoints.html

[pgjones]

An alternative api is to use --insecure-bind alongside --bind e.g.

hypercorn --bind 127.0.0.1:8443 --insecure-bind 127.0.0.1:8080

which I've used in Hypercorn.

Also allows for --quic-bind (also used in Hypercorn for QUIC/HTTP3).

[benoitc]

Prior art for this would be Twisted's endpoint syntax:

ssl:port=443:privateKey=/etc/ssl/server.pem:extraCertChain=/etc/ssl/chain.pem:sslmethod=SSLv3_METHOD:dhParameters=dh_param_1024.pem

Check out https://docs.twistedmatrix.com/en/latest/core/howto/endpoints.html#servers and https://docs.twistedmatrix.com/en/latest/api/twisted.internet.endpoints.html

I like it. Especially it allows to pass easily the options with a specific port. I will add that to the roadmap that will be published later today.

[benoitc]

An alternative api is to use --insecure-bind alongside --bind e.g.

hypercorn --bind 127.0.0.1:8443 --insecure-bind 127.0.0.1:8080

which I've used in Hypercorn.

Also allows for --quic-bind (also used in Hypercorn for QUIC/HTTP3).

But it will add too much options imo, making it hard for the user to remember. I Like beeing able to reuse the --bind option and enrich it withs ome parameters to the hostname. This would also allows some easy configuration inside the config file

Right now I bam thinking to something like --bind ssl://HOST:PORT#.... .

[glyph]

FWIW: our syntax is kind of gross (the use of colon as a separator is particularly hard on typing paths to certificates on Windows, for just one example) and I've wanted to revamp it for some time. If you come up with a better generalized way of spelling this, we'd be interested to hear about it and maybe to support some common subset!

[hynek]

I personally would urge you to consider using URIs for this instead of inventing your own thing. There's a bunch of great libraries for creating/parsing/validating them, so you'll save yourself a lot of grief – especially around encoding etc.

For example:

>>> from yarl import URL
>>> u = URL.build(scheme="tls", host="127.0.0.1", port=8080, query={"ciphers":"AESGCM", "keyfile":"/etc/ssl/private/key.pem", "certfile":"/etc/ssl/certs/cert.pem", "ca-certs":"/etc/ssl/certs/ca.pem"})
>>> u.scheme
'tls'
>>> u.host
'127.0.0.1'
>>> u.query["keyfile"]
'/etc/ssl/private/key.pem'
>>> str(u)
'tls://127.0.0.1:8080/?ciphers=AESGCM&keyfile=/etc/ssl/private/key.pem&certfile=/etc/ssl/certs/cert.pem&ca-certs=/etc/ssl/certs/ca.pem'

[megazhuk]

May be something like --bind 0.0.0.0:80:443 ? First port is http, second is https. When i need only http - nothing's changes --bind 0.0.0.0:80 If i need only https --bind 0.0.0.0::443 It's just minor and simple. 😉

[megazhuk]

And one more thing - i think it's important. In some situations i want to run gunicorn without certificates in other with certs. But i do not want to change params. I think some key like --skip-cert-check will be useful. Imagine that you are running a new Docker Container with fixed CMD ["gunicorn", "main:app", ... in Dockerfile. You just pull image w/o certs from repo and run it. And it will works. Then if you need HTTPS, just curl certs and do kill SIGHUP and it will reload with a fresh certs. I mean to use key --skip-cert-check is for ignoring non existing certs. Just run, check and ignore if they are non exists.

[tilgovi]

Maybe instead of the transport we should use the socket family. That would be consistent with unix:. We can already guess ipv4 vs ipv6, usually, based on how the address parses, but having those prefixes could be the canonical form.

Whether you bind to that address with UDP or TCP and with/without TLS, these maybe are separate options. Specifying these per bind address is even trickier, which is why we are all trying to come up with syntax and there is no standard one.

It may be best to have some advanced configurations only be possible with configuration files, where we have the option to take dictionaries for these options.

I'm not sure. These are just some thoughts. I don't love complicated syntax for the bind address.

[megazhuk]

Maybe instead of the transport we should use the socket family. That would be consistent with unix:. We can already guess ipv4 vs ipv6, usually, based on how the address parses, but having those prefixes could be the canonical form.

unix socket: --bind unix:./http_sck:/https_sck --bind unix::./somedir/only_https_sck --bind unix:../../only_http_sck

tcp: --bind 0.0.0.0:80:443

I mean, that you can use ip-addr or keyword 'unix', if using ip-addr then paste ports(http:https) after colon. If keyword 'unix' then paste paths(http_sck:https_sck) after colon. And of course use multibinding if necessary.

We can use pattern like this --bind x:y:z X - ip-addr or "unix" keyword (that may be omitted) Y - non secure Port or Socket Name Z - secure Port or Socket Name

In future we may add more keywords to X