Provide BatchingProcessor as a built-in processor

line / decaton

High throughput asynchronous task processing on Apache Kafka

Apache License 2.0

336 stars 51 forks source link

Closed ocadaruma closed 3 months ago

ocadaruma commented 3 years ago

Task batching is a common-pattern that many Decaton users often implement by their own
- i.e. Batching several tasks of type T to List<T> and process them at once. e.g. when downstream-DB supports batching I/O (which often very efficient)
- Batch-flushing should be done in time-based and size-based.
So it's better to provide BatchingProcessor built-in to meet the common needs

ryamagishi commented 3 years ago

I'll try this issue.

ocadaruma commented 3 years ago

Thank you! I assigned you.

ocadaruma commented 3 months ago

fixed by #139, #150