99x / serverless-dynamodb-local

Serverless Dynamodb Local Plugin - Allows to run dynamodb locally for serverless
MIT License
624 stars 233 forks source link

Incorrect ValidationException: No provisioned throughput specified for the table #189

Open jslowack opened 5 years ago

jslowack commented 5 years ago

Actual Behaviour

AWS announced support for an automatic provisioning pricing plan. More info: https://medium.com/@softprops/putting-dynamodb-scalability-knobs-on-auto-pilot-3af8520439c9.

In short, you can set BillingMode: PAY_PER_REQUEST and remove the ProvisionedThoughput section. But at the moment serverless-dynamodb-local does not support this:

  Validation Exception -----------------------------------

  No provisioned throughput specified for the table

Expected Behaviour

No validation error

Steps to reproduce it

Create a template using BillingMode: PAY_PER_REQUEST: https://gist.githubusercontent.com/softprops/58c60a1ca2a8e97d6ef63bda3d3c8a9f/raw/27d2721f1d1837e3cca00f854810f484f9239ad4/serverless.yml

Would you like to work on the issue?

Yes

jslowack commented 5 years ago

ValidationException is triggered during table creation: https://github.com/99xt/serverless-dynamodb-local/blob/v1/index.js#L307

muswain commented 5 years ago

I have been also trying to use the new DynamoDB option of Billing mode but get the same exception.Now cannot change the serverless local DynamoDB behavior to try this new feature. Seems it is due to the fact that the aws-sdk as dependency has not been updated since quite a while.Its still on 2.7 although we now have 2.368 release for aws-sdk

jslowack commented 5 years ago

I've just upgraded aws-sdk to 2.368 but the issue persists. I think the problem is caused by the local DynamoDb instance (https://github.com/99xt/dynamodb-localhost/).

In the AWS Docs:

Provisioned throughput settings are ignored in downloadable DynamoDB, even though the CreateTable operation requires them. For CreateTable, you can specify any numbers you want for provisioned read and write throughput, even though these numbers are not used. You can call UpdateTable as many times as you want per day. However, any changes to provisioned throughput values are ignored.

I just tried to run sls dynamodb migrate on a local DynamoDb using a Docker image (https://hub.docker.com/r/amazon/dynamodb-local/), but the problem isn't solved.

If I'm correct, Amazon should update the implementation of the local DynamoDb instance.

hotgazpacho commented 5 years ago

Is there any way to pass the provisioned throughput argument to dynamodb-local via command line args?

carlosneves0 commented 5 years ago

As a quick monkey patch, I changed the code below index.js@line 168 to:

migrateHandler() {
        const dynamodb = this.dynamodbOptions();
        const tables = this.tables;
        tables.forEach(table => {
          if (table.BillingMode === 'PAY_PER_REQUEST') {
            table.ProvisionedThroughput = {
              ReadCapacityUnits: 1,
              WriteCapacityUnits: 1
            }
            delete table.BillingMode
          }
        })
        return BbPromise.each(tables, (table) => this.createTable(dynamodb, table));
    }

And it created the table. This way I don't have to change serverless.yml. Could this be published as a temporary fix?

jslowack commented 5 years ago

@c-neves monkey patch does not take Global Secondary Indexes into account. I've created a PR which modifies the migrations object in the createTable function for both tables and indexes. Feel free to give some remarks.

PR: https://github.com/99xt/serverless-dynamodb-local/pull/190