search

clarkie / dynogels

DynamoDB data mapper for node.js. Originally forked from https://github.com/ryanfitz/vogels
Other
490 stars 110 forks source link

Accelerate table creation by creating in parallel instead of series #97

Closed mousavian closed 7 years ago

mousavian commented 7 years ago

To address issue https://github.com/ryanfitz/vogels/issues/218

coveralls commented 7 years ago

Coverage Status

Coverage remained the same at 98.656% when pulling a7124ac8850dd80fdfdee236be1eba46c11c743f on mousavian:parallel-table-creation into 6b3f75c123ec1924d70d0ab1178519012c98b079 on clarkie:master.

clarkie commented 7 years ago

I agree that these can take some time but it's not something I do all the time.

There are limits to the number of tables able to be created/updated in parallel:

CreateTable/UpdateTable/DeleteTable In general, you can have up to 10 CreateTable, UpdateTable, and DeleteTable requests running simultaneously (in any combination). In other words, the total number of tables in the CREATING, UPDATING or DELETING state cannot exceed 10.

The only exception is when you are creating a table with one or more secondary indexes. You can have up to 5 such requests running at a time; however, if the table or index specifications are complex, DynamoDB might temporarily reduce the number of concurrent requests below 5.

http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html#limits-api

I therefore think leaving this restriction in place is better for now.