Upgrading Django to 4.2 crashes the zappa based deployment with Error 500

spirovskib commented 1 year ago

Context

I am running a Django web application as Lambda deployed on AWS via zappa I upgraded Django to 4.2 on my application to remedy security issues reported by Github Dependabot. Local testing didn't reveal any issues. After upgrading i ran zappa update production and the deployment failed, with zappa deployment reporting error 500 upon deployment. In the zappa log i found the following error: 'NoneType' object has no attribute 'read' I found another comment on Django forums that's closed as Django making a breaking change that requires changes in the way WSGIRequest is invoked https://code.djangoproject.com/ticket/34350 At this moment I reverted back to Django 4.1.8 for the application to run.

Is there any proper upgrade path of libraries to make Django 4.2 work?

Expected Behavior

Upgrading Django and redeployment via zappa should be trivial. Previously I have done this many times. On local testing everything is working properly.

Actual Behavior

Error 500 on redeployment through Zappa, and in the zappa log the following error: 'NoneType' object has no attribute 'read'

Possible Fix

Revert back to Django 4.1.8 for the application to run. This will make sense in the short term, but I'm looking for a long term fix.

Steps to Reproduce

Have a Django 4.1.x application running on AWS Lambda, deployed via zappa.
Upgrade Django to 4.2.x
Try zappa update application
The deployment fails, with Error 500
In the zappa tail application logs one can see "NoneType' object has no attribute 'read'"

Your Environment

Zappa version used: 0.56.1
Operating System and Python version: Python 3.8.9, running on AWS Lambda

The output of pip freeze:

alt-fake-useragent==0.2.1
argcomplete==2.0.0
asgiref==3.6.0
backports.zoneinfo==0.2.1
beautifulsoup4==4.12.2
bleach==5.0.1
boto3==1.26.114
botocore==1.29.114
build==0.8.0
certifi==2022.12.7
cfn-flip==1.3.0
charset-normalizer==2.1.0
click==8.1.3
configparser==5.3.0
courlan==0.7.1
dateparser==1.1.8
Django==4.2
django-bleach==3.0.1
django-bootstrap-v5==1.0.11
django-ckeditor==6.5.1
django-cleanup==7.0.0
django-js-asset==2.0.0
django-storages==1.13.2
django-taggit==3.1.0
djangorestframework==3.14.0
docutils==0.19
durationpy==0.5
feedparser==6.0.10
flake8==6.0.0
fpdf==1.7.2
future==0.18.3
fuzzywuzzy==0.18.0
hjson==3.1.0
htmldate==1.3.0
idna==3.4
jarowinkler==1.2.3
jmespath==1.0.1
joblib==1.2.0
jusText==3.0.0
kappa==0.6.0
langcodes==3.3.0
Levenshtein==0.20.1
lxml==4.9.1
markdownify==0.11.6
MarkupSafe==2.1.2
mccabe==0.7.0
mixpanel==4.10.0
nltk==3.7
packaging==21.3
pep517==0.13.0
Pillow==9.5.0
pip-tools==6.13.0
placebo==0.9.0
psycopg2-binary==2.9.4
pycodestyle==2.10.0
pyflakes==3.0.1
pyparsing==3.0.9
python-dateutil==2.8.2
python-dotenv==1.0.0
python-slugify==8.0.1
pytz==2023.3
pytz-deprecation-shim==0.1.0.post0
PyYAML==6.0
rapidfuzz==2.15.1
regex==2023.3.23
requests==2.28.2
s3transfer==0.6.0
sgmllib3k==1.0.0
six==1.16.0
slack-sdk==3.21.1
smart-open==6.3.0
soupsieve==2.3.2.post1
sqlparse==0.4.3
text-unidecode==1.3
tinycss2==1.2.1
tld==0.13
toml==0.10.2
tomli==2.0.1
tqdm==4.65.0
troposphere==4.0.2
tzdata==2023.3
tzlocal==4.3
urllib3==1.26.15
webencodings==0.5.1
Werkzeug==2.2.3
wsgi-request-logger==0.4.6
zappa==0.56.1

Link to your project (optional):

Your zappa_settings.json:

{
"production": {
    "django_settings": "redacted_core.settings",
    "aws_region": "redacted-1",
    "profile_name": "redacted-admin",
    "project_name": "redacted-core",
    "runtime": "python3.8",
    "s3_bucket": "goya-zappa-files",
    "exclude": ["local_data_store",".git","*.md","CODEOWNERS","github_key.txt", "secret_key.txt", "*.sqlite3", "posgtres_env.json","static_assets","media_assets","static_data","*.code-workspace","Dockerfile","docker-compose.yml","local_data_store","zappashell.sh"],
    "timeout_seconds": 600,
    "manage_roles": false,
    "role_name": "ZappaRuntimeRole",
    "vpc_config": {
        "SubnetIds": [ "subnet-0redacted_1", "subnet-0redacted_2" ], 
        "SecurityGroupIds": [ "sg-0redacted_1" ]
    },
    "xray_tracing": false,
    "certificate_arn": "arn:aws:acm:us-east-1:_redacted_:certificate/00XXXXXX-XXXX-XXXX-XXXX-XXXXXXXXac37",
    "domain": "*.beyondmachines.net",
    "debug": false,
    "log_level": "DEBUG",
    "events": [{
        "function": "content.views.internal_get_redacted_one",
        "expression": "cron(0 7 ? * MON-SUN *)"
    },
    {
        "function": "content.views.internal_get_redacted_two",
        "expression": "cron(30 7 ? * MON-SUN *)"
    }
    ]
}
}

dennybiasiolli commented 1 year ago

The solution in my PR could be to set the value of "wsgi.input"to a file-like object, like BytesIO(body).

spirovskib commented 1 year ago

Please share if the PR is merged in a new release and whether I need to configure something in order for the process to run properly for Django 4.2+

dennybiasiolli commented 1 year ago

Yes of course, let's wait for a review from the maintainer.

The following lines are not a solution, just a temporary workaround!

If you need it ASAP, then just switch your requirements line from

zappa==0.56.1

to

zappa @ https://github.com/dennybiasiolli/Zappa/archive/refs/heads/django42.zip

So you can test it with Django 4.2 (and eventually provide another feedback other than mine)

spirovskib commented 1 year ago

Great, task for the weekend.

symbiot-india commented 1 year ago

@dennybiasiolli I'm building a new project with Django 4.2 and I'm facing the same error mentioned in the thread. I wanted to try out the updated package using zappa @ https://github.com/dennybiasiolli/Zappa/archive/refs/heads/django42.zip, but it seems like the package doesn't exist there anymore. If possible, can you please share an updated location for django42.zip, so that I can test it and provide feedback as well?

Here's the error I'm getting:

  ERROR: HTTP error 404 while getting https://github.com/dennybiasiolli/Zappa/archive/refs/heads/django42.zip
ERROR: Could not install requirement zappa@ https://github.com/dennybiasiolli/Zappa/archive/refs/heads/django42.zip from https://github.com/dennybiasiolli/Zappa/archive/refs/heads/django42.zip (from -r requirements.txt (line 33)) because of HTTP error 404 Client Error: Not Found for url: https://codeload.github.com/dennybiasiolli/Zappa/zip/refs/heads/django42 for URL https://github.com/dennybiasiolli/Zappa/archive/refs/heads/django42.zip

dennybiasiolli commented 1 year ago

@symbiot-india my pull request has been approved and merged into Zappa:master, so you can use the official zappa repo with zappa @ https://github.com/zappa/Zappa/archive/refs/heads/master.zip or other official methods like git+https or similar.

symbiot-india commented 1 year ago

Thanks! I will try this out.

symbiot-india commented 1 year ago

Great, it's working now with Django 4.2! Thanks Denny

philclaridge commented 1 year ago

Fixed for me as well. Thank you.

monkut commented 1 year ago

Fixed in 0.57.0 release, published to pypi 5/19, closing.

zappa / Zappa