`zappa init` doesn't generate compatible s3_bucket names

SongGithub commented 7 years ago

Context

I was using zappa init to generate zappa_settings.json file, with following selections:

service name = dev
app name = my_app.app
Would you like to deploy this application globally? (default 'n') [y/n/(p)rimary]: y

Problem:

However inside the json file, automatically generated s3_bucket names were using UnderScore instead of Hyphen. Therefore aws-boto-core were complaining Invalid Bucket Name, because AWS doesn't like underscore convention from Python world.

## Expected Behavior

after I run zappa deploy --all, every bucket should be created correctly on respective buckets

Actual Behavior

Instead, I got errors complaining invalid bucket names supplied, which are originated from incorrectly generated settings file. Traceback (most recent call last): File "/Users/onion/.virtualenvs/zappa_sandbox/lib/python2.7/site-packages/zappa/cli.py", line 2352, in handle sys.exit(cli.handle()) File "/Users/onion/.virtualenvs/zappa_sandbox/lib/python2.7/site-packages/zappa/cli.py", line 456, in handle self.dispatch_command(self.command, environment) File "/Users/onion/.virtualenvs/zappa_sandbox/lib/python2.7/site-packages/zappa/cli.py", line 489, in dispatch_command self.deploy() File "/Users/onion/.virtualenvs/zappa_sandbox/lib/python2.7/site-packages/zappa/cli.py", line 654, in deploy self.zip_path, self.s3_bucket_name) File "/Users/onion/.virtualenvs/zappa_sandbox/lib/python2.7/site-packages/zappa/core.py", line 636, in upload_to_s3 CreateBucketConfiguration={'LocationConstraint': self.aws_region}, File "/Users/onion/.virtualenvs/zappa_sandbox/lib/python2.7/site-packages/botocore/client.py", line 253, in _api_call return self._make_api_call(operation_name, kwargs) File "/Users/onion/.virtualenvs/zappa_sandbox/lib/python2.7/site-packages/botocore/client.py", line 557, in _make_api_call raise error_class(parsed_response, operationname) ClientError: An error occurred (InvalidBucketName) when calling the CreateBucket operation: The specified bucket is not valid.

Possible Fix

in the module that generates zappa-settings.json file, change all Underscores to Hyphens.

Steps to Reproduce

1.zappa init with all default settings

zappa deploy --all
Boom, you will find Invalid bucket name error

Your Environment

Zappa version used:
Operating System and Python version:
Link to your project (optional):
Your zappa_settings.py:

Miserlou commented 7 years ago

Good catch, I thought this was fixed a long time ago..

Miserlou commented 7 years ago

Please send a pull request!

SongGithub commented 7 years ago

@Miserlou Thank you for assigning me for the task. I am working on it, and will send you a PR soon.

SongGithub commented 7 years ago

Quick update for details of the bug: the bug, that is using invalid underscore as storage bucket name will appear when I choose to deploy the application either "Globally" or "(p)rimary". It will not appear if I chose the default 'n'

The init step referred: You can optionally deploy to all available regions in order to provide fast global service. If you are using Zappa for the first time, you probably don't want to do this! Would you like to deploy this application globally? (default 'n') [y/n/(p)rimary]

Miserlou commented 7 years ago

Oh, interesting! Thanks for using that feature, it needs more love.

SongGithub commented 7 years ago

I've managed to have zappa_settings.json/bucket_names all in hyphens not underscores. But I realise during tests that API_stage_name must not have any hyphens and must have underscores. In simple words, API_Gateway stage name and S3 bucket name formats are not compatible, and the origin of the bug is that author assumed it can be shared.

SongGithub commented 7 years ago

And BTW, unit testing for the cli.py is missing, hence the bug didn't get detected sooner. Worth raise another ticket for the unit testing?

Miserlou / Zappa