Support custom SSL Context

samypr100 commented 4 years ago

Checklist

[x] There are no similar issues or pull requests for this yet.
[ ] I discussed this idea on the community chat and feedback is positive.

Is your feature related to a problem? Please describe.

I would like to pass an existing SSL Context to uvicorn.run(). For example, I have a certificate that needs a password to load. Typically I would do that by setting up a context like so:

ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
# ... customize context even more
ssl_context.load_cert_chain(ssl_crt_path, keyfile=ssl_key_path, password=ssl_key_password)

The current options are limited to these kinds of advance scenarios and I'd like to avoid keep adding/requesting --ssl-xyz options for each of those scenarios. I know I can decrypt the key before loading it into python, but I'm limited on the environment I need to deploy on since I'm given the encrypted key and the password via a secret.

Describe the solution you would like.

Adding the ability to pass a ssl_context to uvicorn.run in python code that supersedes any of the ssl_* settings if provided.

Example changes in uvicorn/config.py:

@property
def is_ssl(self) -> bool:
    return bool(self.ssl_keyfile or self.ssl_certfile)

@property
def is_ssl_context(self) -> bool:
    return isinstance(self.ssl_context, ssl.SSLContext)

# ...

if self.is_ssl and not self.is_ssl_context:
    self.ssl = create_ssl_context(
        keyfile=self.ssl_keyfile,
        certfile=self.ssl_certfile,
        ssl_version=self.ssl_version,
        cert_reqs=self.ssl_cert_reqs,
        ca_certs=self.ssl_ca_certs,
        ciphers=self.ssl_ciphers,
    )
elif self.is_ssl_context:
    self.ssl = self.ssl_context
else:
    self.ssl = None

# ...

Describe alternatives you considered

Searched source code to see if there was a way to pass a custom context to no avail.

Additional context

Since ssl context is createe via python, it would not quite be supported via command line. Unless we want to get fancy. I can attempt to do a PR if permitted. Thanks!

[!IMPORTANT]

We're using Polar.sh so you can upvote and help fund this issue.

We receive the funding once the issue is completed & confirmed by you.

Thank you in advance for helping prioritize & fund our backlog.

euri10 commented 4 years ago

I have some questions @samypr100 :)

how does gunicorn deal with that, I'm not following the project closely enough but I don't think they provide that option
if implemented, how people using uvicorn as a gunicorn worker will do ?
I proposed 2 small PR, one implements the ability pas pass a password to decrypt your ssl key, does that solve your use case ?
I'm afraid I'm not clear on this, could you expand ?

but I'm limited on the environment I need to deploy on since I'm given the encrypted key and the password via a secret.

samypr100 commented 4 years ago

Hi @euri10, thanks for taking time and considering this.

I don't know how gunicorn deals with it since I don't think they support --ssl-password either. I'm not sure since I could not find any ssl.SSLContext usage in their codebase. There seems to be related issues https://github.com/benoitc/gunicorn/pull/2012 though to add support, so support might come. I've limited my deployments to use uwsgi via nginx in those scenarios instead of gunicorn for my non-asgi deployments.
I briefly mentioned that I'm not sure how this would work via a CLI unless we have an string equivalent interface the same way we load the app. This issue was meant more for the programtic instantiation. Maybe once gunicorn supports more options related to ssl.SSLContext it might fit better.
The ability to pass a password to decrypt your ssl key would solve my immediate issue. This issue was more of a generic catch all if someone has a more advance SSL context object they wanted to pass on. Flask for example has the ability to pass an ssl_context on it's built-in development server.
All I meant by this is that I currently have cases were I receive the encrypted key string and the password via an environment variable. That's all, hopefully it didn't confuse more.

euri10 commented 4 years ago

Flask for example has the ability to pass an ssl_context on it's built-in development server.

can you point me where, this is interesting.

ok @samypr100 now that #807 and #808 got merged you can use the fixtures and expand on them for you PR should you be fancy providing one,

something like the below should work ?

@pytest.fixture
def ssl_ctx(tls_certificate):
    ssl_ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
    tls_certificate.configure_cert(ssl_ctx)
    return ssl_ctx

once there is a PR layout then maybe we can discuss more proactively about what to do with the CLI

samypr100 commented 4 years ago

can you point me where, this is interesting.

Absolutely! Code: https://github.com/pallets/werkzeug/blob/master/src/werkzeug/serving.py#L604 Docs: https://werkzeug.palletsprojects.com/serving#werkzeug.serving.run_simple (see ssl_context param)

Thanks!

aswanidutt87 commented 2 years ago

I have submitted a PR related to this issue- added param of ssl.Options list to the config of ssl_context, we are trying to resolve a vulnerability of DoS by setting the ssl.OP_NO_RENEGOTIATION to the ssl_options of ssl_context. https://github.com/encode/uvicorn/pull/1692 has the changes

Kludex commented 2 years ago

Adding the ability to pass a sslcontext to uvicorn.run in python code that supersedes any of the ssl* settings if provided.

PR welcome for what @samypr100 proposes.

aswanidutt87 commented 2 years ago

hello @Kludex I have started working on passing ssl_context directly to uvicorn.run via config.py as below. usage:

    ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
    ctx.load_cert_chain(tls_cert, tls_key, None)
    ctx.verify_mode = ssl.VerifyMode(ssl.CERT_NONE)
    ctx.set_ciphers(allowed_ciphers)
    if list_options:
        for each_option in list_options:
            ctx.options |= each_option
    uvicorn.run(
        "web:app",
        host="0.0.0.0",
        port=int(port),
        reload=True,
        ssl_context=ctx,
    )

in main.py ssl_context: typing.Optional[ssl.SSLContext],

and in the config.py

  ssl_context: Optional[ssl.SSLContext] = None,

    @property
    def is_ssl_context(self) -> bool:
        return isinstance(self.ssl_context, ssl.SSLContext)

        if self.is_ssl and not self.is_ssl_context:
            assert self.ssl_certfile
            self.ssl: Optional[ssl.SSLContext] = create_ssl_context(
                keyfile=self.ssl_keyfile,
                certfile=self.ssl_certfile,
                password=self.ssl_keyfile_password,
                ssl_version=self.ssl_version,
                cert_reqs=self.ssl_cert_reqs,
                ca_certs=self.ssl_ca_certs,
                ciphers=self.ssl_ciphers,
            )
        elif self.is_ssl_context:
            self.ssl = self.ssl_context

and it didnt like the passing of ssl.SSLContext object into Config, on the server startup its failing to pickle the SSLContext object.

here is stack trace of the server startup error:

 File "/Users/aswani/Documents/projects/project_uvicorn/uvicorn_submodule/uvicorn/main.py", line 575, in run
    ChangeReload(config, target=server.run, sockets=[sock]).run()
  File "/Users/aswani/Documents/projects/project_uvicorn/uvicorn_submodule/uvicorn/supervisors/basereload.py", line 44, in run
    self.startup()
  File "/Users/aswani/Documents/projects/project_uvicorn/uvicorn_submodule/uvicorn/supervisors/basereload.py", line 80, in startup
    self.process.start()
  File "/Users/aswani/.pyenv/versions/3.10.7/lib/python3.10/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Users/aswani/.pyenv/versions/3.10.7/lib/python3.10/multiprocessing/context.py", line 288, in _Popen
    return Popen(process_obj)
  File "/Users/aswani/.pyenv/versions/3.10.7/lib/python3.10/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/Users/aswani/.pyenv/versions/3.10.7/lib/python3.10/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/Users/aswani/.pyenv/versions/3.10.7/lib/python3.10/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/Users/aswani/.pyenv/versions/3.10.7/lib/python3.10/multiprocessing/reduction.py", line 65, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle 'SSLContext' object

I need to know how we can pass the ssl_context into uvicorn.run if this is not the correct way.. please advise.

Kludex commented 2 years ago

Can we instead pass a function that returns a SSLContext in a factory away?

def ssl_context_factory() -> SSLContext:
    ...

if __name__ == "__main__":
    uvicorn.run("main.app", ssl_context_factory=ssl_context_factory, reload=True)

aswanidutt87 commented 2 years ago

@Kludex something like this worked not sure if you meant to do this way.. usage:

   uvicorn.run(
        "web:app",
        host="0.0.0.0",
        port=int(port),
        reload=True,
        ssl_context=SSLContext(
            tls_cert,
            tls_key,
            None,
            ssl.PROTOCOL_TLS_SERVER,
            ssl.CERT_NONE,
            None,
            allowed_ciphers,
            list_options,
        ),
        workers=10,
    )

in config.py

class SSLContext:
    def __init__(
        self,
        certfile: Union[str, os.PathLike],
        keyfile: Optional[Union[str, os.PathLike]],
        password: Optional[str],
        ssl_version: int,
        cert_reqs: int,
        ca_certs: Optional[Union[str, os.PathLike]],
        ciphers: Optional[str],
        options: Optional[List[ssl.Options]],
    ):
        self.certfile = certfile
        self.keyfile = keyfile
        self.password = password
        self.ssl_version = ssl_version
        self.cert_reqs = cert_reqs
        self.ca_certs = ca_certs
        self.ciphers = ciphers
        self.options = options

     def ssl_context_factory(self) -> ssl.SSLContext:
        ctx = ssl.SSLContext(self.ssl_version)
        get_password = (lambda: self.password) if self.password else None
        ctx.load_cert_chain(self.certfile, self.keyfile, get_password)
        ctx.verify_mode = ssl.VerifyMode(self.cert_reqs)
        if self.ca_certs:
            ctx.load_verify_locations(self.ca_certs)
        if self.ciphers:
            ctx.set_ciphers(self.ciphers)
        if self.options:
            for each_option in self.options:
                ctx.options |= each_option
        return ctx

     if self.is_ssl and not self.is_ssl_context:
            assert self.ssl_certfile
            self.ssl: Optional[ssl.SSLContext] = create_ssl_context(
                keyfile=self.ssl_keyfile,
                certfile=self.ssl_certfile,
                password=self.ssl_keyfile_password,
                ssl_version=self.ssl_version,
                cert_reqs=self.ssl_cert_reqs,
                ca_certs=self.ssl_ca_certs,
                ciphers=self.ssl_ciphers,
            )
        elif self.is_ssl_context and self.ssl_context is not None:
            self.ssl = self.ssl_context.ssl_context_factory()
        else:
            self.ssl = None

but its not much of different than passing all parameters that are needed for creating a ssl_context inside config.py, I dont think we can create a custom ssl_context object and pass it into config.py

Kludex commented 1 year ago

I think the idea is more to call the factory when needed on each worker. Similar to: https://github.com/benoitc/gunicorn/pull/2649/

aswanidutt87 commented 1 year ago

thanks for the hint, here is the PR - https://github.com/encode/uvicorn/pull/1815, please review

desean1625 commented 10 months ago

In Gunicorn version 21.0+ the code that mapped the CLI flag --ssl-version TLSv1_2 into an int was removed.

This causes the uvicorn workers to always throw errors when trying to set a specific tls version and cyphers.

aswanidutt87 commented 10 months ago

hi @desean1625 currently I guess we are on 0.24.0 Uvicorn and passing ssl_version=int(ssl.PROTOCOL_TLS_SERVER) and ssl_ciphers=allowed_ciphers to the uvicorn.run() for our use case, if we are going to get impacted with new change, what will be the alternative for this change.

desean1625 commented 10 months ago

@aswanidutt87 yes with gunicorn 21.0 and uvicorn 0.24 I am seeing this because a change in gunicorn.

gunicorn --ciphers ... -k uvicorn.workers.UvicornWorker --ssl-version TLSv1_2 causes ssl_version to be TLSv1_2 instead of int(5)

 File "/usr/local/lib/python3.11/site-packages/uvicorn/config.py", line 119, in create_ssl_context
    ctx = ssl.SSLContext(ssl_version)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/ssl.py", line 500, in __new__
    self = _SSLContext.__new__(cls, protocol)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'str' object cannot be interpreted as an integer

I tried gunicorn --ciphers ... -k uvicorn.workers.UvicornWorker --ssl-version 5 but still get a TypeError: 'str' object cannot be interpreted as an integer when the worker tries to start.

I suppose my question is would uvicorn add a check to parse --ssl-version 5 as an int until gunicorn completely stops sending that flag, or would allowing the support for custom ssl contexts make this obsolete.

aswanidutt87 commented 10 months ago

This PR has been un attended since a year now, I am trying to add a custom ssl to the uvicorn as we need ssl_options and somebody in future might need something else in ssl_context so thought adding a custom ssl_context pass to uvicorn.run() a good idea, but couldn't go much far. @Kludex helped me until some point reviewing the changes and its been an year. and Yes, adding custom ssl_context would resolve all this ssl_version and cipher as you bring your own ssl_context and pass to uvicorn.run. please correct me @Kludex

cschmutzer commented 9 months ago

I am also interested in the ability to pass an SSL context. my use case is to have HTTPS using PSK instead of certificates

aswanidutt87 commented 8 months ago

@Kludex , the need for passing custom ssl_context to the uvicorn is gaining momentum, please help me close this PR, its been two years that we are struggling to close this.

https://github.com/encode/uvicorn/pull/1815 Thanks in advance.

UnshiftedEarth commented 6 months ago

@Kludex Has this feature been added yet? Seems like a pretty reasonable option as users may want to setup their own SSL context instead of being limited by basic parameters.

Torxed commented 6 months ago

The shortest workaround I've found thus far, is the following. Bear in mind that it will touch disk unless you find a workaround for it:

config = uvicorn.Config(
    app,
    host="127.0.0.1",
    port=1789,
    # We could define it here, and override the default context
    # But omitting these two make sure no context is created:
    # ssl_keyfile=privkey,
    # ssl_certfile=certificate,
    log_level="info",
    reload=False
)

ssl_ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
get_password = (lambda: password) if password else None

# `cert_file` and `key_file_name` are paths that `ssl*.load_cert_chain()` can access:
ssl_ctx.load_cert_chain(certfile=cert_file_name, keyfile=key_file_name, password=get_password)
ssl_ctx.verify_mode = ssl.VerifyMode(ssl.CERT_NONE)

# Options omitted here, that the default context creation handles:
# ssl_ctx.load_verify_locations(ca_certs)
# ssl_ctx.set_ciphers(ciphers)

config.load() # Manually calling the .load() to trigger needed actions outeside of HTTPS
# calling .loaded() multiple times will do nothing the
# following times, because unicorn
# caches of it's loaded or not.

# And once that's done, we force in the ssl context
# since it won't be replaced after .loaded ()
config.ssl = ssl_ctx

uvicorn = uvicorn.Server(
    config
)

uvicorn.run()

This is part of a larger snippet which is slightly unrelated to this conversation:

import fastapi
import datetime
import tempfile
import ssl
import uvicorn

from cryptography import x509
from cryptography.x509.oid import NameOID
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric import rsa

app = fastapi.FastAPI()
password = "passphrase"

key = rsa.generate_private_key(
    public_exponent=65537,
    key_size=2048,
)
# Write our key to disk for safe keeping
privkey = key.private_bytes(
    encoding=serialization.Encoding.PEM,
    format=serialization.PrivateFormat.TraditionalOpenSSL,
    encryption_algorithm=serialization.BestAvailableEncryption(password.encode()),
)

subject = issuer = x509.Name([
    x509.NameAttribute(NameOID.COUNTRY_NAME, "US"),
    x509.NameAttribute(NameOID.STATE_OR_PROVINCE_NAME, "California"),
    x509.NameAttribute(NameOID.LOCALITY_NAME, "San Francisco"),
    x509.NameAttribute(NameOID.ORGANIZATION_NAME, "My Company"),
    x509.NameAttribute(NameOID.COMMON_NAME, "mysite.com"),
])
cert = x509.CertificateBuilder().subject_name(
    subject
).issuer_name(
    issuer
).public_key(
    key.public_key()
).serial_number(
    x509.random_serial_number()
).not_valid_before(
    datetime.datetime.now(datetime.timezone.utc)
).not_valid_after(
    # Our certificate will be valid for 10 days
    datetime.datetime.now(datetime.timezone.utc) + datetime.timedelta(days=10)
).add_extension(
    x509.SubjectAlternativeName([x509.DNSName("localhost")]),
    critical=False,
# Sign our certificate with our private key
).sign(key, hashes.SHA256())
# Write our certificate out to disk.

certificate = cert.public_bytes(serialization.Encoding.PEM)

config = uvicorn.Config(
    app,
    host="127.0.0.1",
    port=1789,
    # We could define it here, and override the default context
    # But omitting these two make sure no context is created:
    # ssl_keyfile=privkey,
    # ssl_certfile=certificate,
    log_level="info",
    reload=False
)

ssl_ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
get_password = (lambda: password) if password else None

with tempfile.NamedTemporaryFile("wb") as cert_file, tempfile.NamedTemporaryFile("wb") as key_file:
    cert_file.write(certificate)
    cert_file.flush()
    cert_file_name = cert_file.name

    key_file.write(privkey)
    key_file.flush()
    key_file_name = key_file.name

    ssl_ctx.load_cert_chain(certfile=cert_file_name, keyfile=key_file_name, password=get_password)
    ssl_ctx.verify_mode = ssl.VerifyMode(ssl.CERT_NONE)

    # Options omitted here, that the default context creation handles:
    # ssl_ctx.load_verify_locations(ca_certs)
    # ssl_ctx.set_ciphers(ciphers)

    config.load() # Manually calling the .load() to trigger needed actions outeside of HTTPS

    # And once that's done, we force in the ssl context
    config.ssl = ssl_ctx

uvicorn = uvicorn.Server(
    config
)

uvicorn.run()

The code could be reduced to use existing certificates. But due to lack of time, I simply used a code base that auto-generates certificates from: https://github.com/encode/uvicorn/discussions/2339

encode / uvicorn