AWS::DynamoDB::Table Global Tables

[PatMyron]

2. Scope

c) new attribute for existing resource is desired

5. Links

https://aws.amazon.com/dynamodb/global-tables/ https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/GlobalTables.html https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-dynamodb-table.html https://aws.amazon.com/blogs/aws/new-convert-your-single-region-amazon-dynamodb-tables-to-global-tables/

---

https://forums.aws.amazon.com/thread.jspa?threadID=288408 https://stackoverflow.com/questions/47984111/dynamodb-global-tables-using-cloudformation https://github.com/kspurrier/cfn-dynamodb-global-table

6. Category:

3. DB (RDS, DynamoDB...)

[colinhuckstep]

Yeah, I'm trying to decide if it's worth it to rebuild my service catalog product since it's working just fine with my custom resource running the API commands. I would probably still need parts of the resource as we've got a requirement to notify our DBA team on create/delete of a new table. Currently the CR is handling that for me.

On Mon, May 17, 2021, 3:31 PM Ben Bridts @.***> wrote:

@Ephesoft-Stitus https://github.com/Ephesoft-Stitus It's not listed on https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/resource-import-supported-resources.html (yet?), but since it's a new resource, the chance of it actually being supported is pretty high

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/aws-cloudformation/aws-cloudformation-coverage-roadmap/issues/57#issuecomment-842577118, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHNXFKI2R4SR276YZSIZCLDTOFVIXANCNFSM4IIJTLXQ .

[rohits-spec]

Launched on 5/14, resolving the issue.

[michaelwittig]

😭

[benkehoe]

I am supportive of a separate ::GlobalTable type, because it requires different permissions, should be able to be differentiated by things like CloudFormation Guard, etc. I believe that if ::GlobalTable supported resource import, it would address the conversion problem. You would set DeletionPolicy: Retain on your ::Table, then remove it from the stack. Then you'd import the table as a ::GlobalTable.

[mbarneyjr]

I'm not sure a separate standalone resource is an appropriate declarative representation of the resource being created/configured. It seems like the kind of thing that, if a separate resource, would make more sense as an augment to another AWS::DynamoDB::Table resource, that enables global tables on the table you target, like this:

  MyTable:
    Type: AWS::DynamoDB::Table
    ...
  GlobalTables:
    Type: AWS::DynamoDB::GlobalTable
    Properties:
      Table: !Ref MyTable
      ...

While the API is create-global-tables, it requires an existing table. It augments an existing table to enable replication/synchronization of data across the specified regions

What if there's a feature in the future that serves a similar type of augmentation? Would we have the cartesian product of each DynamoDB table augmentation as separate resources (::Table, ::GlobalTable, ::FooTable, ::FooGlobalTable)?

Resource import seems like a work around, rather than what should be the intended happy path

[kddejong]

I got some thoughts on this one too. This is my analysis from using the resource and understanding a little about how resource development works. Since ::GlobalTable creates a table in the other regions it would need access to the definition to create the resource. In theory they could get that from reading the actual ::Table definition but complex types for GetAtt aren't supported. If ::GlobalTable related to ::Table via the ARN it could read the definition live and replicate it but this would be less than ideal since we would want it defined in code.

I do agree that it would have been ideal to have ::GlobalTable as a separate resource that relates to a ::Table. My points here are similar to @mbarneyjr. It feels like so many of the attributes are duplicative between the resources with just a few new ones. I may be wrong here but I think this may be the first resource that creates a resource in another region.

In your mind what would have been the ideal flow for setting up all the tables and the global table?

1. You have to create the table in each region using CloudFormation stack in each region. Then in one of the regions you could have created the ::GlobalTable resource that relates to the ::Table. This would have made it a 3+ step process to get the global table setup. This method would relate to whats in the docs
2. Add a Regions attribute to ::Table that creates the table in each supplied region then link it up with ::GlobalTable
3. Leave it is a separate resource and create a flow to convert to the ::GlobalTable without destruction. I'm going to put a general here be dragons comment. I don't know how this will all play out between ::Table and ::GlobalTable in relation to resource import, drift detection, AWS config, etc. I haven't had the cycles to dig through it.

Willing to update this with more options as I'm sure you all have some thoughts on this as well. My goal is to understand what your ideal flow should have been if these types didn't overlap. Things like do you expect the ::GlobalTable to create the tables in the other regions or should it assume the table is already there, etc.

[PatMyron]

I am supportive of a separate ::GlobalTable type, because it requires different permissions, should be able to be differentiated by things like CloudFormation Guard

True of new property types as well as new resource types. Still prefer this being available as a property type of ::Table

[Ephesoft-Stitus]

@kddejong - I'd prefer option 1. with the assumption that it would require less permissions for our IaC user since the replica tables would be defined ahead of time. However, that method would also need to support existing ::Table to ::GlobalTable conversion (which may require more permissions?).

Currently, there are two different ways to programmatically create a global table, Add Replica and Create Replica Table.

Add Replica (create-global-table), which has reduced permissions needs, is a way to take two existing DynamoDB tables that have already been created and join them into a Global Table (https://docs.aws.amazon.com/cli/latest/reference/dynamodb/create-global-table.html). Note that none of the tables can have any data in them which is a critical block if you already have regular tables in a production environment.

Create Replica Table (update-table), which has broad permissions needs, is a way to create an entirely new table in the target region and join it with the existing table to for a Global Table (https://aws.amazon.com/blogs/aws/new-convert-your-single-region-amazon-dynamodb-tables-to-global-tables/). With this method, the replicas are automatically created however you can run this command against an existing table with data in it vs Add Replica which only works with new tables.

Add Replica seems to align with your option 1 which makes me think it may not be possible to support a ::Table to ::GlobalTable conversion with this approach.

Create Replica Table seems to be the existing approach in CloudFormation so it seems like adding support for conversion to ::GlobalTable would be achievable.

[pgxtreme]

Am I the only one that get this error when trying to deploy global tables?

Properties validation failed for resource GlobalRegionLookupTable with message: 
#/WriteProvisionedThroughputSettings: extraneous key [MinCapacity] is not permitted 
#/WriteProvisionedThroughputSettings: extraneous key [TargetTrackingScalingPolicyConfiguration] is not permitted
#/WriteProvisionedThroughputSettings: extraneous key [MaxCapacity] is not permitted

[PatMyron]

@pgxtreme WriteCapacityAutoScalingSettings is the only property of WriteProvisionedThroughputSettings and those are sub-properties of that

[pgxtreme]

@pgxtreme WriteCapacityAutoScalingSettings is the only property of WriteProvisionedThroughputSettings and those are sub-properties of that property

Oh you're right, I missed the WriteCapacityAutoScalingSettings type. Thanks for the help!

[fpronto]

I would like to get some help

I am trying to create a global table and I will need the streamArn from all the replicas. There is a way to get that?

Right now I am trying to export the stream arn in the cloudformation template, but I can only access the region where this template is being deployed. I want to be able to access this values in another cloudformation template

I am aware that the streamArn id is different in both replicas (the end part of the streamArn), so I can't construct it either.

[mm326]

When trying to create a global table stack without the billing mode it defaults to Provisioned or even whan I set it to Provisioned Cloudformation complains that: You must specify ReadCapacityUnits or ReadCapacityAutoScalingSettings for each global table replica and each of its GSIs when BillingMode is Provisioned

When I add either one of those to the template I get this error: TestDynamoDBGlobalTable with message: #: extraneous key [ReadCapacityAutoScalingSettings/ReadCapacityUnits] is not permitted it's the same if I use the ReadProvisionedThroughputSettings key.

These docs https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-dynamodb-globaltable.html don't have them in the example but do have them on the dropdown options on the left hand side menu

I know I can set BillingMode to PAY_PER_REQUEST but wanted to know if this is an issue or if there is a way around it.

[bbmehta]

When trying to create a global table stack without the billing mode it defaults to Provisioned or even whan I set it to Provisioned Cloudformation complains that: You must specify ReadCapacityUnits or ReadCapacityAutoScalingSettings for each global table replica and each of its GSIs when BillingMode is Provisioned

When I add either one of those to the template I get this error: TestDynamoDBGlobalTable with message: #: extraneous key [ReadCapacityAutoScalingSettings/ReadCapacityUnits] is not permitted it's the same if I use the ReadProvisionedThroughputSettings key.

These docs https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-dynamodb-globaltable.html don't have them in the example but do have them on the dropdown options on the left hand side menu

I know I can set BillingMode to PAY_PER_REQUEST but wanted to know if this is an issue or if there is a way around it.

Please use following file as reference to understand where exactly you need to mention ReadProvisionedThroughputSettings: test.yaml.txt

[kddejong]

As a result of AWS::DynamoDB::GlobalTable now supporting resource import we are going to close this. If you have issues with the import operation, the resource itself, or GetAtt support please create a new issue. If you would prefer to use a modifier to the AWS::DynamoDB::Table resource you will have to do this via a custom or registry based resource.

I've transferred one of the comments into an issue already.