search

awslabs / kinesis-aggregation

AWS libraries/modules for working with Kinesis aggregated record data
Apache License 2.0
376 stars 154 forks source link

Size or time in seconds for trigger callback #136

Open reneincer opened 3 years ago

reneincer commented 3 years ago

Hi there,

I love this library, but for small data sync, I need to wait a lot of for the 1 MB max size config. I know I can change this for doing callback faster, but It will be great if we can set the max time that aggregator can wait to do the callback. For example, 10 seconds, If the buffer only have 300KB, and the last callback was like 10 seconds before, do the callback.

Whatever it comes first,

What do you think?

IanMeyers commented 3 years ago

Yep like the idea. I’d suggest a record counter as well as time.