Closed spirovskib closed 1 month ago
Update, how I made it work after 4 hours:
bin
and lib
folders and pyenv.cfg
)virtualenv -p python3.9 .
pip install -r requirements.txt --no-cache-dir
python manage.py runserver
zappa update production
On this iteration the loading of libraries includes psycopg2-binary==2.9.4: Using locally cached manylinux wheel
(quoting the second run of the update so there is cache):
Downloading and installing dependencies..
- levenshtein==0.20.1: Downloading
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.43M/1.43M [00:00<00:00, 3.69MB/s]
- markupsafe==2.1.2: Downloading
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25.5k/25.5k [00:00<00:00, 3.82MB/s]
- pillow==9.5.0: Downloading
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3.31M/3.31M [00:01<00:00, 3.29MB/s]
- pyyaml==6.0.1: Downloading
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 739k/739k [00:00<00:00, 3.41MB/s]
- jarowinkler==1.2.3: Using locally cached manylinux wheel
- lxml==4.9.1: Using locally cached manylinux wheel
- psycopg2-binary==2.9.4: Using locally cached manylinux wheel
- rapidfuzz==2.15.1: Using locally cached manylinux wheel
- regex==2023.3.23: Using locally cached manylinux wheel
I was reviewing my actions last evening, and although I've managed to recover, it wasn't because I discovered the root cause. I'm reopening the issue for wider consideration, maybe some extra debugging is possible for everyone to benefit if this event happens again.
I assume that this problem occurred due to the latest version of psycopg2-binary
not having a manylinux wheel available for the desired python runtime.
If that's the case, I recommend pinning the version of psycopg2-binary
to the latest version where a manylinux wheel is available.
This is a very good insight. Is it possible to add a check and fail the update command if the manylinux wheel is not available and architecture of the machine is not x64?
That's a good point.
It would be better to abort the update when the associated library is not able to be installed.
At the moment the lambda package is built on the system where the command is run.
I suspect there could be a check during the package build process to determine if a package has a wheel or not.
Although it might be hard to determine which libraries would need a 'wheel' (typically those that require to be compiled/built) and those that don't.
In addition, we do allow deployment to docker, so that would need to be taken into consideration. (Since a docker deployment could potentially support the case where the expected 'wheel' is not available.
When I pin this packages, it works on my end. psycopg==3.1.10 psycopg-binary==3.1.10 psycopg2-binary==2.9.7 zappa==0.57.0
Hi there! Unfortunately, this Issue has not seen any activity for at least 90 days. If the Issue is still relevant to the latest version of Zappa, please comment within the next 10 days if you wish to keep it open. Otherwise, it will be automatically closed.
Hi there! Unfortunately, this Issue was automatically closed as it had not seen any activity in at least 100 days. If the Issue is still relevant to the latest version of Zappa, please open a new Issue.
Context
I was pushing a minor code update to a zappa managed Django site. The site was operating normally until the update. On the update I got the following error
When running zappa tail production i found the following error:
During the preparation of the deployment i don't see that psycopg2-binary is loaded from manylinux wheel. The only change i've made was updating to MacOS Ventura 13.5 yesterday. But I'm using homebrew python.
Expected Behavior
zappa update production should properly deploy the application and run it, since no library changes were made
Actual Behavior
Deployment results in an error
ImproperlyConfigured: Error loading psycopg2 or psycopg module
Possible Fix
No idea
Steps to Reproduce
Can't be sure that this reproduces, but this is my setup: Have a lambda django application configured to connect to postgress database backend On MacOS Ventura 13.5 have a Homebrew Python 3.8 install, running in virtual environment for the code try a
zappa update
of the running code.Your Environment
zappa -v 0.57.0
Python 3.8.17 (default, Jun 21 2023, 05:28:43) [Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin
Homebrew Python install, running in virtual environment for this code
alt-fake-useragent==0.2.1 argcomplete==2.0.0 asgiref==3.6.0 backports.zoneinfo==0.2.1 beautifulsoup4==4.12.2 bleach==5.0.0 blurhash==1.1.4 boto3==1.26.114 botocore==1.29.114 build==0.8.0 certifi==2022.12.7 cfn-flip==1.3.0 charset-normalizer==2.1.0 click==8.1.3 configparser==5.3.0 courlan==0.7.1 crispy-bootstrap5==0.7 dateparser==1.1.8 decorator==5.1.1 Django==4.2.3 django-bleach==3.0.1 django-bootstrap-v5==1.0.11 django-ckeditor==6.5.1 django-cleanup==7.0.0 django-countries==7.5.1 django-crispy-forms==2.0 django-js-asset==2.0.0 django-meta==2.2.0 django-ranged-response==0.2.0 django-simple-captcha==0.5.17 django-storages==1.13.2 django-taggit==3.1.0 djangorestframework==3.14.0 dnspython==2.3.0 docutils==0.19 durationpy==0.5 email-validator==2.0.0.post2 feedparser==6.0.10 flake8==6.0.0 fpdf==1.7.2 future==0.18.3 fuzzywuzzy==0.18.0 git-filter-repo==2.38.0 hjson==3.1.0 htmldate==1.3.0 idna==3.4 jarowinkler==1.2.3 jmespath==1.0.1 joblib==1.2.0 jusText==3.0.0 kappa==0.6.0 langcodes==3.3.0 Levenshtein==0.20.1 lxml==4.9.1 markdownify==0.11.6 MarkupSafe==2.1.2 Mastodon.py==1.8.1 mccabe==0.7.0 mixpanel==4.10.0 nltk==3.7 oauthlib==3.2.2 packaging==21.3 pep517==0.13.0 Pillow==9.5.0 pip-tools==6.13.0 placebo==0.9.0 psycopg2-binary==2.9.4 pycodestyle==2.10.0 pyflakes==3.0.1 pyparsing==3.0.9 python-dateutil==2.8.2 python-dotenv==1.0.0 python-magic==0.4.27 python-slugify==8.0.1 pytz==2023.3 pytz-deprecation-shim==0.1.0.post0 PyYAML==6.0.1 rapidfuzz==2.15.1 regex==2023.3.23 requests==2.28.2 requests-oauthlib==1.3.1 s3transfer==0.6.0 sgmllib3k==1.0.0 six==1.16.0 slack-sdk==3.21.1 smart-open==6.3.0 soupsieve==2.3.2.post1 sqlparse==0.4.4 text-unidecode==1.3 tinycss2==1.2.1 tld==0.13 toml==0.10.2 tomli==2.0.1 tqdm==4.64.0 troposphere==4.0.2 tweepy==4.14.0 typing_extensions==4.5.0 tzdata==2023.3 tzlocal==4.3 urllib3==1.26.15 webencodings==0.5.1 Werkzeug==2.2.3 wsgi-request-logger==0.4.6 zappa==0.57.0
{ "production": { "django_settings": "goya_core.settings", "aws_region": "eu-central-1", "profile_name": "_profile_nameredacted", "project_name": "goya-core", "runtime": "python3.8", "s3_bucket": "goya-zappa-files", "exclude": ["local_data_store",".git",".md","CODEOWNERS","github_key.txt", "secret_key.txt", ".sqlite3", "posgtres_env.json","static_assets","media_assets","static_data",".code-workspace","Dockerfile","docker-compose.yml","local_data_store","zappashell.sh"], "timeout_seconds": 600, "manage_roles": false, "role_name": "ZappaRuntimeRole", "vpc_config": { "SubnetIds": [ "subnet-0d390740cf414cb13", "subnet-0f6e2a7f4a66cfeeb" ], "SecurityGroupIds": [ "sg-03656bdc887694ed8" ] }, "xray_tracing": false, "certificate_arn": "arn:aws:acm:us-east-1:_accountredacted:certificate/_cert_redacted", "domain": ".beyondmachines.net", "debug": false, "log_level": "DEBUG", "events": [{ "function": "communicator.views.social_media_poster", "expression": "cron(1 8-20 ? MON-SUN )" }, { "function": "content.views.generate_weekly_report_content", "expression": "cron(0 16 ? MON )" }, { "function": "content.views.get_daily_events", "expression": "cron(0 10 ? MON-FRI )" } ] } }