esbtools / event-handler

Notification in, document out.
GNU General Public License v3.0
3 stars 6 forks source link

Explore explicitly modelling processing domain around batching #23

Open alechenninger opened 8 years ago

alechenninger commented 8 years ago

@kahowell and I agreed making the batching mechanisms currently present in the lightblue implementation more explicit would improve code maintainability. The current model of returning Future from DocumentEvent#lookupDocument, Notification#toDocumentEvents, and Message#process is extremely flexible, but perhaps to a fault. The Future implementation in BulkLightblueRequester is robust in that it batches requests with optimum efficiency behind an unimposing API, but it is a bit magical and foreign to the uninitiated.

In addition, batching is a core part of the problem domain. In almost all imaginable implementations, processing events in bulk will be more efficient than processing messages one at a time, and we've deliberately made other API decisions around batch processing (for example, all repository APIs use Collections). The Future based APIs allow for the concurrency necessary to do batching transparently, but they also allow for just about any usage. This flexibility is unnecessary, and we could likely better serve the project with an API more specific to the batching problem domain.