rynvelt
commented
3 years ago Same problem here. It occurs from time to time. Using serverless-offline-dynamodb-streams 4.2.0. This is an excerpt from the serverless.yaml:
serverless-offline-dynamodb-streams:
apiVersion: '2013-12-02'
endpoint: http://0.0.0.0:8000
region: eu-central-1
accessKeyId: root
secretAccessKey: root
skipCacheInvalidation: false
readInterval: 500
functions:
stream-lambda:
timeout: 10
handler: src/index.streamHandler
events:
- stream:
enabled: true
type: dynamodb
arn: ${self:custom.tableArn}/stream/test-stream/1970-01-01T00:00:00.000
batchSize: 10
startingPosition: LATESTI run the DynamoDB in a container with Docker (amazon/dynamodb-local:1.15.0) with the following command: ["-jar", "DynamoDBLocal.jar", "-sharedDb", "-dbPath", "/home/dynamodblocal/storage"] on port 8000. Recreating the database helps, but I assume that this is not expected behavior?
Having a look at https://github.com/CoorpAcademy/serverless-plugins/blob/master/packages/serverless-offline-dynamodb-streams/src/dynamodb-streams.js and at first glance the plugin is not doing much related to shards, but it dispatches to https://github.com/CoorpAcademy/serverless-plugins/blob/master/packages/dynamodb-streams-readable/src/index.js.
To me this looks suspicous:
const shardId = options.shardId
? data.StreamDescription.Shards.filter(function (shard) {
return shard.ShardId === options.shardId;
}).map(function (shard) {
return shard.ShardId;
})[0]
: data.StreamDescription.Shards[0].ShardId;Like if the shard ID was not passed, the first best shard is returned. And furthermore the AWS DynamoDB Local documentation (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.UsageNotes.html) says:
If you're using DynamoDB Streams, the rate at which shards are created might differ. In the DynamoDB web service, shard-creation behavior is partially influenced by table partition activity. When you run DynamoDB locally, there is no table partitioning. In either case, shards are ephemeral, so your application should not be dependent on shard behavior.
Is this maybe an out-of-sync problem with dynamodb-streams-readable not handling this property gracefully?
Searching for the error message "Invalid ShardId in ShardIterator" doesn't lead to anything helpful. Any other ideas?
mdrijwan
not working with my python project!!
this is my yml file settings:
this is error i get: