search

awslabs / dynamodb-continuous-backup

Continuous backup automation for Amazon DynamoDB
Apache License 2.0
185 stars 27 forks source link

us-west-1 errors - kinesis firehose not available? #38

Open afsheenb opened 7 years ago

afsheenb commented 7 years ago

I'm not sure whether this is a fundamental incompatibility or something that can be worked around, but:

I have a set of DynamoDB tables in us-west-1 region;

I've followed the instructions in the README.md, and they're straightforward enough;

However, when I attempt to provision backups for my tables, I receive the following error:

root@#:/dynamodb-continuous-backup/src# python provision_tables.py provisioning_whitelist.hjson 
Building Table List for Processing from provisioning_whitelist.hjson
Using compiled configuration config.hjson
Loaded configuration from config.hjson
Resolved DynamoDB Stream ARN: arn:aws:dynamodb:us-west-1:xxxxxxx:table/TestBackups/stream/2017-09-01T16:36:28.852
Exception while provisioning table TestBackups
Could not connect to the endpoint URL: "https://firehose.us-west-1.amazonaws.com/"
Proceeding...
Enabled Update Stream for SecondTable
Resolved DynamoDB Stream ARN: arn:aws:dynamodb:us-west-1:xxxxx:table/SecondTable/stream/2017-09-01T16:52:20.160
Exception while provisioning table SecondTable
Could not connect to the endpoint URL: "https://firehose.us-west-1.amazonaws.com/"
Proceeding...
Enabled Update Stream for ThirdTable
Resolved DynamoDB Stream ARN: arn:aws:dynamodb:us-west-1:xxxxx:table/ThirdTable/stream/2017-09-01T16:52:48.014
Exception while provisioning table ThirdTable
Could not connect to the endpoint URL: "https://firehose.us-west-1.amazonaws.com/"
Proceeding...

Initially I'd configured my region as us-west-1, and although deployment appeared to be successful, testing showed nothing being backed up and invocation errors (with similar messages being logged to Cloudwatch).

Removing the Cloudwatch events and Lambda function, running the deprovisioning script, then setting the region to us-west-2 and re-deploying the Lambda function is at least throwing errors for me now.

My question: Is it possible to back up DynamoDB tables in us-west-1 region using dynamodb-continuous-backup, even if Kinesis Firehose is unavailable there?

IanMeyers commented 7 years ago

Hello. Sorry about this - that's correct that Firehose isn't available in us-west-1, and I've not built in cross region support. I'll have a dig to see how hard it would be to add for this case.