search

django / channels

Developer-friendly asynchrony for Django
https://channels.readthedocs.io
BSD 3-Clause "New" or "Revised" License
6.09k stars 801 forks source link

Using uvicorn + gunicorn as ASGI server fails to boot #1639

Closed bertonha closed 3 years ago

bertonha commented 3 years ago

With latest update of twisted 21.2.0 the app stops to inicialize when using gunicorn + uvicorn as ASGI server.

bertonha commented 3 years ago

One way to avoid this issue is not load daphne on apps.py

https://github.com/django/channels/blob/master/channels/apps.py#L4

Since it loads daphne + twisted + other libs which on this scenario is not needed

KStenK commented 3 years ago

Related to django/daphne#359

carltongibson commented 3 years ago

See https://github.com/django/channels/discussions/1643

Uninen commented 3 years ago

Is there any workaround for this? I'm trying to get a new Django asgi project deployed with uvicorn but no combination of packages seems to work. Should I use some other server (daphne?) instead for now? (Sorry if this is the wrong place to ask.)

yedpodtrzitko commented 3 years ago

no combination of packages seems to work

Show your pip freeze pls. Generally capping twisted<21 should work (see https://github.com/django/daphne/pull/359 mentioned a few comments above )

Uninen commented 3 years ago

I read all of the linked issues and tried various versions of twisted, gunicorn and daphne but everything failed with some import from daphne or channels failing. (Using Python 3.9 and Django 3.2.)

I got the http routes working with gunicorn after removing AuthMiddlewareStack (commented out from channels.auth import AuthMiddlewareStack and the corresponding websocket entry (inside ProtocolTypeRouter: "websocket": AuthMiddlewareStack(URLRouter(core.routing.websocket_urlpatterns)),) from asgi.py. This let me move on with configuring the deployment but obviously the websocket part of the app is now missing.

I'm using poetry and pip freeze didn't look very helpful but poetry show seems to work:

> poetry show

aioredis                       1.3.1     asyncio (PEP 3156) Redis support
amqp                           5.0.6     Low-level AMQP client for Python (fork of amqplib).
appdirs                        1.4.4     A small Python module for determining appropriate platform-specific dirs, e.g. a "user data dir".
appnope                        0.1.2     Disable App Nap on macOS >= 10.9
argon2-cffi                    20.1.0    The secure Argon2 password hashing algorithm.
asgiref                        3.3.4     ASGI specs, helper code, and adapters
astroid                        2.5.2     An abstract syntax tree for Python with inference support.
asttokens                      2.0.4     Annotate AST trees with source code positions
async-generator                1.10      Async generators and context managers for Python 3.5+
async-timeout                  3.0.1     Timeout context manager for asyncio programs
attrs                          20.3.0    Classes Without Boilerplate
autobahn                       21.3.1    WebSocket client & server library, WAMP real-time framework
automat                        20.2.0    Self-service finite-state machines for the programmer on the go.
backcall                       0.2.0     Specifications for callback functions passed in to an API
bcrypt                         3.2.0     Modern password hashing for your software and your servers
billiard                       3.6.4.0   Python multiprocessing fork with improvements and bugfixes
black                          20.8b1    The uncompromising code formatter.
bleach                         3.3.0     An easy safelist-based HTML-sanitizing tool.
brotli                         1.0.9     Python bindings for the Brotli compression library
celery                         5.0.5     Distributed Task Queue.
certifi                        2020.12.5 Python package for providing Mozilla's CA Bundle.
cffi                           1.14.5    Foreign Function Interface for Python calling C code.
channels                       3.0.3     Brings async, event-driven capabilities to Django. Django 2.2 and up only.
channels-redis                 3.2.0     Redis-backed ASGI channel layer implementation
chardet                        4.0.0     Universal encoding detector for Python 2 and 3
click                          7.1.2     Composable command line interface toolkit
click-didyoumean               0.0.3     Enable git-like did-you-mean feature in click.
click-plugins                  1.1.1     An extension module for click to enable registering CLI commands via setuptools entry-points.
click-repl                     0.1.6     REPL plugin for Click
colorama                       0.4.4     Cross-platform colored terminal text.
constantly                     15.1.0    Symbolic constants in Python
coreapi                        2.3.3     Python client library for Core API.
coreschema                     0.0.4     Core Schema.
coverage                       5.5       Code coverage measurement for Python
cryptography                   3.4.7     cryptography is a package which provides cryptographic recipes and primitives to Python developers.
daphne                         3.0.2     Django ASGI (HTTP/WebSocket) server
decorator                      5.0.6     Decorators for Humans
defusedxml                     0.7.1     XML bomb protection for Python stdlib modules
django                         3.2       A high-level Python Web framework that encourages rapid development and clean, pragmatic design.
django-allauth                 0.44.0    Integrated set of Django applications addressing authentication, registration, account management as well as ...
django-cacheops                5.1       A slick ORM cache with automatic granular event-driven invalidation for Django.
django-celery-beat             2.2.0     Database-backed Periodic Tasks.
django-celery-results          2.0.1     Celery result backends for Django.
django-cors-headers            3.7.0     django-cors-headers is a Django application for handling the server headers required for Cross-Origin Resourc...
django-debug-toolbar           3.2       A configurable set of panels that display various debug information about the current request/response.
django-environ                 0.4.5     Django-environ allows you to utilize 12factor inspired environment variables to configure your Django applica...
django-extensions              3.1.2     Extensions for Django
django-js-asset                1.2.2     script tag with additional attributes for django.forms.Media
django-maintenance-mode        0.16.0    django-maintenance-mode shows a 503 error page when maintenance-mode is on.
django-mptt                    0.12.0    Utilities for implementing Modified Preorder Tree Traversal with your Django Models and working with trees of...
django-mysql                   3.11.1    Django-MySQL extends Django's built-in MySQL and MariaDB support their specific features not available on oth...
django-querycount              0.7.0     Middleware that Prints the number of DB queries to the runserver console.
django-redis                   4.12.1    Full featured redis cache backend for Django.
django-reversion               3.0.9     An extension to the Django web framework that provides version control for model instances.
django-stubs                   1.7.0     Mypy stubs for Django
django-stubs-ext               0.1.0     Monkey-patching and extensions for django-stubs
django-taggit                  1.3.0     django-taggit is a reusable Django application for simple tagging.
django-templated-mail          1.1.1     Send emails using Django template system.
django-timezone-field          4.1.2     A Django app providing database and form fields for pytz timezone objects.
django-widget-tweaks           1.4.8     Tweak the form field rendering in templates, not in python-level form definitions.
djangorestframework            3.12.4    Web APIs for Django, made easy.
djangorestframework-camel-case 1.2.0     Camel case JSON support for Django REST framework.
djangorestframework-simplejwt  4.6.0     A minimal JSON Web Token authentication plugin for Django REST Framework
djangorestframework-stubs      1.4.0     PEP-484 stubs for django-rest-framework
djoser                         2.1.0     REST implementation of Django authentication system.
elasticsearch                  7.12.0    Python client for Elasticsearch
elasticsearch-dsl              7.3.0     Python client for Elasticsearch
entrypoints                    0.3       Discover and load entry points from installed packages.
executing                      0.5.4     Get the currently executing AST node of a frame, and other information
fabric2                        2.6.0     High level SSH command execution
faker                          8.0.0     Faker is a Python package that generates fake data for you.
fancycompleter                 0.9.1     colorful TAB completion for Python prompt
filelock                       3.0.12    A platform independent file lock.
flake8                         3.9.0     the modular source code checker: pep8 pyflakes and co
flake8-quotes                  3.2.0     Flake8 lint for quotes.
funcy                          1.15      A fancy and practical functional tools
future                         0.18.2    Clean single-source support for Python 3 and 2
gunicorn                       20.1.0    WSGI HTTP Server for UNIX
h11                            0.12.0    A pure-Python, bring-your-own-I/O implementation of HTTP/1.1
hiredis                        2.0.0     Python wrapper for hiredis
httptools                      0.1.1     A collection of framework independent HTTP protocol utils.
humanfriendly                  9.1       Human friendly output for text interfaces using Python
hyperlink                      21.0.0    A featureful, immutable, and correct URL for Python.
icecream                       2.1.0     Never use print() to debug again; inspect variables, expressions, and program execution with a single, simple...
idna                           2.10      Internationalized Domain Names in Applications (IDNA)
incremental                    21.3.0    A small library that versions your Python projects.
influxdb                       5.3.1     InfluxDB client
iniconfig                      1.1.1     iniconfig: brain-dead simple config-ini parsing
invoke                         1.5.0     Pythonic task execution
ipykernel                      5.5.3     IPython Kernel for Jupyter
ipython                        7.22.0    IPython: Productive Interactive Computing
ipython-genutils               0.2.0     Vestigial utilities from IPython
ipywidgets                     7.6.3     IPython HTML widgets for Jupyter
isort                          5.8.0     A Python utility / library to sort Python imports.
itypes                         1.2.0     Simple immutable types for python.
jedi                           0.18.0    An autocompletion tool for Python that can be used for text editors.
jinja2                         2.11.3    A very fast and expressive template engine.
jsonschema                     3.2.0     An implementation of JSON Schema validation for Python
jupyter                        1.0.0     Jupyter metapackage. Install all the Jupyter components in one go.
jupyter-client                 6.1.13    Jupyter protocol implementation and client libraries
jupyter-console                6.4.0     Jupyter terminal console
jupyter-core                   4.7.1     Jupyter core package. A base package on which Jupyter projects rely.
jupyterlab-pygments            0.1.2     Pygments theme using JupyterLab CSS variables
jupyterlab-widgets             1.0.0     A JupyterLab extension.
kombu                          5.0.2     Messaging library for Python.
lazy-object-proxy              1.6.0     A fast and thorough lazy object proxy.
markupsafe                     1.1.1     Safely add untrusted strings to HTML/XML markup.
mccabe                         0.6.1     McCabe checker, plugin for flake8
mistune                        0.8.4     The fastest markdown parser in pure Python
model-bakery                   1.2.1     Smart object creation facility for Django.
msgpack                        1.0.2     MessagePack (de)serializer.
mypy                           0.812     Optional static typing for Python
mypy-extensions                0.4.3     Experimental type system extensions for programs checked with the mypy typechecker.
mysqlclient                    2.0.3     Python interface to MySQL
nbclient                       0.5.3     A client library for executing notebooks. Formerly nbconvert's ExecutePreprocessor.
nbconvert                      6.0.7     Converting Jupyter Notebooks
nbformat                       5.1.3     The Jupyter Notebook format
nest-asyncio                   1.5.1     Patch asyncio to allow nested event loops
notebook                       6.3.0     A web-based notebook environment for interactive computing
oauthlib                       3.1.0     A generic, spec-compliant, thorough implementation of the OAuth request-signing logic
orjson                         3.5.1     Fast, correct Python JSON library supporting dataclasses, datetimes, and numpy
packaging                      20.9      Core utilities for Python packages
pandocfilters                  1.4.3     Utilities for writing pandoc filters in python
paramiko                       2.7.2     SSH2 protocol library
parso                          0.8.2     A Python Parser
pathlib2                       2.3.5     Object-oriented filesystem paths
pathspec                       0.8.1     Utility library for gitignore style pattern matching of file paths.
pdbpp                          0.10.2    pdb++, a drop-in replacement for pdb
pexpect                        4.8.0     Pexpect allows easy control of interactive console applications.
pickleshare                    0.7.5     Tiny 'shelve'-like database with concurrency support
pillow                         8.2.0     Python Imaging Library (Fork)
pinboard                       2.1.8     A Python wrapper for Pinboard.in
pluggy                         0.13.1    plugin and hook calling mechanisms for python
prometheus-client              0.10.1    Python client for the Prometheus monitoring system.
prompt-toolkit                 3.0.18    Library for building powerful interactive command lines in Python
psycopg2-binary                2.8.6     psycopg2 - Python-PostgreSQL Database Adapter
ptpython                       3.0.17    Python REPL build on top of prompt_toolkit
ptyprocess                     0.7.0     Run a subprocess in a pseudo terminal
py                             1.10.0    library with cross-python path, ini-parsing, io, code, log facilities
pyasn1                         0.4.8     ASN.1 types and codecs
pyasn1-modules                 0.2.8     A collection of ASN.1-based protocols modules.
pycodestyle                    2.7.0     Python style guide checker
pycparser                      2.20      C parser in Python
pyflakes                       2.3.1     passive checker of Python programs
pygments                       2.8.1     Pygments is a syntax highlighting package written in Python.
pyhamcrest                     2.0.2     Hamcrest framework for matcher objects
pyjwt                          2.0.1     JSON Web Token implementation in Python
pylint                         2.7.4     python code static checker
pynacl                         1.4.0     Python binding to the Networking and Cryptography (NaCl) library
pyopenssl                      20.0.1    Python wrapper module around the OpenSSL library
pyparsing                      2.4.7     Python parsing module
pyrepl                         0.9.0     A library for building flexible command line interfaces
pyrsistent                     0.17.3    Persistent/Functional/Immutable data structures
pytest                         6.2.3     pytest: simple powerful testing with Python
pytest-asyncio                 0.14.0    Pytest support for asyncio.
pytest-celery                  0.0.0a1   pytest-celery a shim pytest plugin to enable celery.contrib.pytest
pytest-cov                     2.11.1    Pytest plugin for measuring coverage.
pytest-django                  4.1.0     A Django plugin for pytest.
pytest-mock                    3.5.1     Thin-wrapper around the mock package for easier use with pytest
pytest-mypy                    0.8.1     Mypy static type checker plugin for Pytest
python-crontab                 2.5.1     Python Crontab API
python-dateutil                2.8.1     Extensions to the standard Python datetime module
python-dotenv                  0.17.0    Read key-value pairs from a .env file and set them as environment variables
python-gitlab                  2.6.0     Interact with GitLab API
python-twitter                 3.5       A Python wrapper around the Twitter API
python3-openid                 3.2.0     OpenID support for modern servers and consumers.
pytz                           2021.1    World timezone definitions, modern and historical
pywatchman                     1.4.1     Watchman client for python
pyzmq                          22.0.3    Python bindings for 0MQ
qtconsole                      5.0.3     Jupyter Qt console
qtpy                           1.9.0     Provides an abstraction layer on top of the various Qt bindings (PyQt5, PyQt4 and PySide) and additional cust...
raven                          6.10.0    Raven is a client for Sentry (https://getsentry.com)
redis                          3.5.3     Python client for Redis key-value store
regex                          2021.4.4  Alternative regular expression module, to replace re.
requests                       2.25.1    Python HTTP for Humans.
requests-oauthlib              1.3.0     OAuthlib authentication support for Requests.
requests-toolbelt              0.9.1     A utility belt for advanced users of python-requests
send2trash                     1.5.0     Send file to trash natively under Mac OS X, Windows and Linux.
sentry-sdk                     1.0.0     Python client for Sentry (https://sentry.io)
service-identity               18.1.0    Service identity verification for pyOpenSSL & cryptography.
six                            1.15.0    Python 2 and 3 compatibility utilities
social-auth-app-django         4.0.0     Python Social Authentication, Django integration.
social-auth-core               4.1.0     Python social authentication made simple.
sqlparse                       0.4.1     A non-validating SQL parser.
terminado                      0.9.4     Tornado websocket backend for the Xterm.js Javascript terminal emulator library.
testpath                       0.4.4     Test utilities for code working with files and commands
text-unidecode                 1.3       The most basic Text::Unidecode port
tinify                         1.5.1     Tinify API client.
toml                           0.10.2    Python Library for Tom's Obvious, Minimal Language
tornado                        6.1       Tornado is a Python web framework and asynchronous networking library, originally developed at FriendFeed.
traitlets                      5.0.5     Traitlets Python configuration system
twisted                        20.3.0    An asynchronous networking framework written in Python
txaio                          21.2.1    Compatibility API between asyncio/Twisted/Trollius
typed-ast                      1.4.2     a fork of Python 2 and 3 ast modules with type comment support
typing-extensions              3.7.4.3   Backported and Experimental Type Hints for Python 3.5+
ua-parser                      0.10.0    Python port of Browserscope's user agent parser
ujson                          4.0.2     Ultra fast JSON encoder and decoder for Python
uritemplate                    3.0.1     URI templates
urllib3                        1.26.4    HTTP library with thread-safe connection pooling, file post, and more.
user-agents                    2.2.0     A library to identify devices (phones, tablets) and their capabilities by parsing browser user agent strings.
uvicorn                        0.12.3    The lightning-fast ASGI server.
uvloop                         0.15.2    Fast implementation of asyncio event loop on top of libuv
uwsgi                          2.0.19.1  The uWSGI server
vine                           5.0.0     Promises, promises, promises.
wcwidth                        0.2.5     Measures the displayed width of unicode strings in a terminal
webencodings                   0.5.1     Character encoding aliases for legacy web content
websockets                     8.1       An implementation of the WebSocket Protocol (RFC 6455 & 7692)
whitenoise                     5.2.0     Radically simplified static file serving for WSGI applications
widgetsnbextension             3.5.1     IPython HTML widgets for Jupyter
wmctrl                         0.3       A tool to programmatically control windows inside X
wrapt                          1.12.1    Module for decorators, wrappers and monkey patching.
zope.interface                 5.3.0     Interfaces for Python

and asgi.py:

# from channels.auth import AuthMiddlewareStack
from channels.routing import ProtocolTypeRouter, URLRouter
from django.core.asgi import get_asgi_application

application = ProtocolTypeRouter(
    {
        "http": get_asgi_application(),
        # "websocket": AuthMiddlewareStack(URLRouter(core.routing.websocket_urlpatterns)),
    }
)

I haven't tried using Daphne yet, I can try that as a workaround. And in any case this is not a high priority thing for me, but hopefully above is helpful for debugging.

Thanks for your work 👍

bertonha commented 3 years ago

@Uninen just set twisted on <21

But your asgi.py has a small issue you need to instanciate django app before use the AuthMiddlewareStack(...)

As on the docs https://channels.readthedocs.io/en/stable/deploying.html#configuring-the-asgi-application

yedpodtrzitko commented 3 years ago

@Uninen then I assume you are getting different traceback than shown in the original message of this issue. In any case it would be useful to provide it too. And if it's really different, then it would make sense to create another issue rather than piggybacking here.

qeternity commented 3 years ago

Howdy @carltongibson - what's the plan for this? Shall I open a PR that pins twisted? Or do you want to leave it for the daphne decouple?

We're pinned in prod without any issues, and would be nice to have a viable current channels release until daphne can be dealt with.

carltongibson commented 3 years ago

File "/usr/local/lib/python3.9/site-packages/channels/apps.py", line 4, in <module> import daphne.server

Does removing channels from INSTALLED_APPS allow the error to be bypassed here?

In general, I'm not really happy to force the old twisted version onto all users just for those using uvicorn, but it would be good to push forward with a decent attempt on the decoupling (that includes the beginner friendly story for runserver, and adjusts the tutorial and docs accordingly)

qeternity commented 3 years ago

It would seem so, just testing locally. What are the implications of removing channels from INSTALLED_APPS? From my cursory read, it would seem that we just forgo the monkeypatching of runserver which is irrelevant to non-daphne users.

carltongibson commented 3 years ago

Management commands, templates and models generally, so just the runserver yes I'd think.

jalaziz commented 3 years ago

Management commands, templates and models generally, so just the runserver yes I'd think.

runwoker gets removed too, which still seems useful while not using daphne. Is there a good way to run workers without runworker?

littlebtc commented 3 years ago

FYI, Twisted 21.7.0 released with fixes for this issue:

carltongibson commented 3 years ago

Thanks for the confirmation @littlebtc