Unable to get my first basic Zappa Deployment, Assistance would be much appreciated

andrecowie commented 7 years ago

Context

So I am trying to upskill in AWS. I have a decent understanding of python. I found zappa and have done a few lambda functions prior to finding it. Aswell as flask deployments using apache.

## Expected Behavior Watching the tutorials I assumed zappa would be easy enough to get working basically. However I was wrong, I think the main issue may be arising from my python environments but I am unsure.

Actual Behavior

Cut to the chase i want to deploy my first flask zappa application to my private aws account and have been recieving two issues and hoped that someone could help me solve the issue as I am sure it is most likely due to my misunderstanding of AWS services. Firstly following the video provided by John Wheeler i set up my user on aws with Administrator access. And used aws configure to save my access and secret keys to my ~/.aws/credientials file. I then created a ~/demo directory and wrote a flask app with one route to return "hello world". I then created my virtual environment and pip installed flask and zappa. I then did the two easy steps zappa init and zappa deploy only to be greeted with a issue i was unable to find any prior documentation on.

On my deploy i recieved:

Error 1

Calling deploy for environment dev..
Creating ZappaLambdaExecution IAM Role..
Error: Failed to manage IAM roles!

This i found strange as I assumed zappa would find my aws credientials file and get access to a user with Administrator policies. But that seems to not be the case. I then tried to bypass the issue by creating a role in my AWS account with admin access aswell for Zappa to assume. And added said role to my zappa_settings.json file in ~/demo with:

"manage_roles":false, "role_name":"MyLambdaRole"

however then on my zappa deploy zappa got a bit further and was able to download and install the dependencies but then when trying to package the project as a zip i received a new error:

Error 2

Packaging project as zip.. Oh no! An error occurred! :(

==============

Traceback (most recent call last):
  File "/home/dre/demo/venv/lib/python2.7/site-packages/zappa/cli.py", line 2165, in handle
    sys.exit(cli.handle())
  File "/home/dre/demo/venv/lib/python2.7/site-packages/zappa/cli.py", line 414, in handle
    self.dispatch_command(self.command, environment)
  File "/home/dre/demo/venv/lib/python2.7/site-packages/zappa/cli.py", line 446, in dispatch_command
    self.deploy()
  File "/home/dre/demo/venv/lib/python2.7/site-packages/zappa/cli.py", line 568, in deploy
    self.zip_path, self.s3_bucket_name)
  File "/home/dre/demo/venv/lib/python2.7/site-packages/zappa/zappa.py", line 587, in upload_to_s3
    CreateBucketConfiguration={'LocationConstraint': self.aws_region},
  File "/home/dre/demo/venv/lib/python2.7/site-packages/botocore/client.py", line 253, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/home/dre/demo/venv/lib/python2.7/site-packages/botocore/client.py", line 543, in _make_api_call
    raise error_class(parsed_response, operation_name)
 ClientError: An error occurred (RequestTimeTooSkewed) when calling the CreateBucket operation: The difference between the request time and the current time is too large.

==============

Possible Fix

See below.

Steps to Reproduce

I am sure the most likely reason for my unsuccessful deployment will be something between the relationship of where i am installing awscli and my configuration settings for the credientials file but I am unsure. I would hope someone could clarify whether awscli should be a system wide install with the .aws folder existing potentially in root / or in my user account /home/dre/ . And then a way to see if zappa deploy is using the configured keys in the credientials file for my basic free aws account which has one user with a policy for adminaccess. Aswell as the single role for MyLambdaRole with an admin policy attached aswell.

Sorry for any miscommunication I am still trying to learn my way around AWS land and any help will be very appreciated.

The main reason for creating the issue was to help with the documentation for new users like myself that have difficulties with their local environment configuration of the awscli aswell as zappas link to the file

Your Environment

I am on a Arch Linux system with both pip2.7 and pip3 i have been clarifying for all installs in this process to use 2.7. I have tried installing the awscli into all possible locations to no avail, --user, system and into the same virtual env as the project dependencies.

Zappa version used: zappa --version 0.39.1
Operating System and Python version: Arch Linux have both python 3 and 2.7 have been clarifing to use pip2.7 on all installs.
Link to your project (optional): right now the project is a single py file with flask and a single route('/') to return "Hello World"

Your zappa_settings.py: I am unsure where this resides my json in the project is as you would expect: cat zappa_settings.json

{
     "dev": {
          "app_function": "main.app", 
          "s3_bucket": "zappa-gji73jdal",
      "manage_roles": false,
      "role_name" : "MyLambdaRole"
      }
 }

with the first error occuring without the bottom two lines.

andrecowie commented 7 years ago

I am special!

Turns out awscli doesn't work if your timedatelocal isn't configured properly. I like to shoot myself in the foot for being stupid.

[dre@AndrePC ~]$ aws iam list-access-keys

An error occurred (SignatureDoesNotMatch) when calling the ListAccessKeys operation: Signature not yet current: 20170329T135034Z is still later than 20170329T010535Z (20170329T005035Z + 15 min.)

Miserlou commented 7 years ago

Hey @brnabus - thanks for your well-written bug report, and thanks for using Zappa! Sorry you didn't have a smooth first time!

Clock drift with AWS is an annoying problem that comes up from time to time. If possible, we should try to do a better job of checking for this and telling people how to fix it.

Thanks again, sorry I couldn't help sooner.

andrecowie commented 7 years ago

Allgood @Miserlou great work on an awesome library and this bug should hopefully teach me a valuable lesson!

owenfi commented 6 years ago

Sorry to comment on a long since closed issue, but I ran into this as well, and think it was due to docker causing/having clock drift.

Here is a series of events:

(ve) zappashell> zappa deploy beta
(botocore 1.10.10 (/var/task/ve/lib/python3.6/site-packages), Requirement.parse('botocore<1.9.0,>=1.8.22'), {'boto3'})
Calling deploy for stage beta..
Creating beta-beta-ZappaLambdaExecutionRole IAM Role..
Error: Failed to manage IAM roles!
You may lack the necessary AWS permissions to automatically manage a Zappa execution role.
To fix this, see here: https://github.com/Miserlou/Zappa#using-custom-aws-iam-roles-and-policies

(ve) zappashell> aws iam list-access-keys

An error occurred (SignatureDoesNotMatch) when calling the ListAccessKeys operation: Signature not yet current: 20180601T070940Z is still later than 20180601T065856Z (20180601T064356Z + 15 min.)
(ve) zappashell> date
Fri Jun  1 07:09:51 UTC 2018
(ve) zappashell> docker run --rm --privileged alpine hwclock -s
docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.
See 'docker run --help'.

Whoops, that last one should be run from the computer not docker... On a host machine shell:

[22.4 23:44:11][owen.Mimas: sotd]$ docker run --rm --privileged alpine hwclock -s
Unable to find image 'alpine:latest' locally
latest: Pulling from library/alpine
ff3a5c916c92: Pull complete 
Digest: sha256:7df6db5aa61ae9480f52f0b3a06a140ab98d427f86d8d5de0bedab9b8df6b1c0
Status: Downloaded newer image for alpine:latest

Then back to the zappa-docker shell:

(ve) zappashell> date
Fri Jun  1 06:45:44 UTC 2018

After that, the list and deploy works as expected, woohoo!

(ve) zappashell> aws iam list-access-keys
{
"AccessKeyMetadata": [
    {
        "UserName": "owen",
        "AccessKeyId": "AK...",
        "Status": "Active",
        "CreateDate": "2018-04-01T08:11:06Z"
    }
]
}
(ve) zappashell> zappa deploy dev
(botocore 1.10.10 (/var/task/ve/lib/python3.6/site-packages), Requirement.parse('botocore<1.9.0,>=1.8.22'), {'boto3'})
Calling deploy for stage dev..
Creating task-dev-ZappaLambdaExecutionRole IAM Role..
Creating zappa-permissions policy on task-dev-ZappaLambdaExecutionRole IAM Role.
Downloading and installing dependencies..
- psycopg2==2.7.4: Using locally cached manylinux wheel
- pillow==5.1.0: Using locally cached manylinux wheel
- sqlite==python36: Using precompiled lambda package
Packaging project as zip.
Uploading task-dev-1527835578.zip (27.8MiB)..

SStauden commented 4 years ago

Thanks for your bug report. Today I spend much time in setting up a Zappa configuration to AWS getting always the same error "Error: Failed to manage IAM roles! You may lack the necessary AWS permissions to automatically manage a Zappa execution role."

Thanks to your post, I encountered, that my system time was not correct and this prevented to exectute "zappa deploy" and "zappa update". So make sure, youre local system time is synchronized! ⏲

Miserlou / Zappa