Doing blind Lambda-based writes to Dynamo is unsustainable for larger numbers of users and bases referenced in reports. We need to transition to performing all Dynamo writes, throttling, and throughput scaling by means of a queue.
This can probably be done without breaking existing code. Lambdas responsible for Dynamo writes will instead enqueue the needed operations. Another process will take care of dequeuing and performing the actual writes, and can also handle Dynamo scaling. When the queue begins to grow to the point that the process can no longer sustain it without causing a shard to occur, it can alert us so we can figure out what to do.
aneilbaboo
commented
5 years ago
Doing blind Lambda-based writes to Dynamo is unsustainable for larger numbers of users and bases referenced in reports. We need to transition to performing all Dynamo writes, throttling, and throughput scaling by means of a queue.
This can probably be done without breaking existing code. Lambdas responsible for Dynamo writes will instead enqueue the needed operations. Another process will take care of dequeuing and performing the actual writes, and can also handle Dynamo scaling. When the queue begins to grow to the point that the process can no longer sustain it without causing a shard to occur, it can alert us so we can figure out what to do.