Open anateus opened 7 years ago
I'll bet you a dollar this only occurs in us-east-1
, if you want to try in another region..
Suggest we add an optional excludes_default
to replace/override the internal excludes.
Related is to bump the version of boto the Zappa ships with.
"I'm not certain under what conditions S3 requires v4 signatures"
I've had this error when trying to pass headers dict to boto.S3.Key.set_contents_from_filename() The headers were s3 KMS encryption related e.g. {x-amz-server-side-encryption: "kms", ...} and the error returned is "Requests specifying Server Side Encryption with AWS KMS managed keys require AWS Signature Version 4" I'm using ap regions not us.
The work around did not work for me. This is really blocking me from properly deploying my app. Any other suggestions?
What i am seeing when i don't rely on the values for important things, not sure if it is related:
[1515611587391] [DEBUG] 2018-01-10T19:13:07.391Z 4a48f26f-f63a-11e7-a554-e5677c554d11 The s3 config key is not a dictionary type, ign
oring its value of: None
Context
I'm not certain under what conditions S3 requires v4 signatures, but that's the direction things are moving. Zappa currently uses the AWS-provided boto and botocore packages which default to using HMACv1 signatures. There appears to be no way to instruct those packages to use s3v4 as the signature method.
Expected Behavior
Uploading a JSON file to s3 and setting its path to the
remote_env
field inzappa_settings.json
should result in the env vars being loaded as if they were provided in theenvironment_variables
field.Actual Behavior
The app deploys fine, but then fails to load. In the logs we can see that accessing the remote json file fails due to signature error. The presence of
Calculating signature using hmacv1 auth.
in the logs indicates it's not using the v4 signature method.Possible Fix
The
botocore
changlog indicates that starting from version 1.5.71, the v4 signature method becomes the default. AWS provides older versions for both Python 2.7 and 3.6.Was able to fix this by manually modifying
zappa/cli.py
to removeboto3
andbotocore
from the list of excluded packages and having the latest versions of both in the virtual environment when the app gets deployed.Thus, a proposed fix is to add configuration parameter that allows overriding the default excludes.
Steps to Reproduce
As mentioned above, v4 seems to be the default signature method now, but some regions may allow older ones as well. If the following steps aren't reproducible this may be why.
remote_env.json
file on S3zappa_settings.json
Your Environment
pip freeze
:zappa_settings.py
: