Closed tiagomelo closed 2 years ago
Hi, I've figured out what is happening.
At item
struct (https://github.com/harlow/kinesis-consumer/blob/master/store/ddb/ddb.go#L88):
type item struct {
Namespace string `json:"namespace"`
ShardID string `json:"shard_id"`
SequenceNumber string `json:"sequence_number"`
}
although you're using json
struct tag to correctly map to the field names at the table, if you do not inform TagKey
option with json
(https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/feature/dynamodb/attributevalue#hdr-Struct_tags), the item struct will be marshalled ignoring json tags, ending up with this:
item: map[Namespace:0xc000132288 SequenceNumber:0xc0001322e8 ShardID:0xc0001322a0]
which is, capitalized names that does not reflect the fields at the table.
One quick fix would be to change item
struct to this:
type item struct {
Namespace string `dynamodbav:"namespace"`
ShardID string `dynamodbav:"shard_id"`
SequenceNumber string `dynamodbav:"sequence_number"`
}
because dynamodbav
struct tag is automatically supported.
OR (preferred)
inform TagKey
option with json
(https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/feature/dynamodb/attributevalue#hdr-Struct_tags).
and that leaves to a problem: https://github.com/harlow/kinesis-consumer/blob/master/store/ddb/ddb.go#L104-L108
Key: map[string]types.AttributeValue{
"namespace": &types.AttributeValueMemberS{
Value: namespace,
},
"shard_id": &types.AttributeValueMemberS{
Value: shardID,
},
},
json
struct tag, so... I've created Namespace
, ShardID
and SequenceNumber
fields into the tableGetCheckpoint
is using namespace
and shard_id
in lowercase, it will throw an error:
api error ValidationException: The provided key element does not match the schema
@tiagomelo thanks for submitting this issue; I was able to recreate the error and fixed as you suggested using the dynamodbav
annotation for the checkpoint item
Hi,
I'm using DynamoDB as storage. After inspecting the source code and putting some debug printing, I've found out this error on
func (c *Checkpoint) save() error
fromddb.go
:the item is correctly fulfilled (I've printed out
namespace
,ShardID
andSequenceNumber
, all of them are valid):My table has everything it needs as instructed:
Any clue? Thanks.