search

Informatievlaanderen / VSDS-Linked-Data-Interactions

https://informatievlaanderen.github.io/VSDS-Linked-Data-Interactions/
European Union Public License 1.2
4 stars 6 forks source link

Make Batching Collector functionality for LdioSender component #662

Open Yalz opened 2 months ago

Yalz commented 2 months ago

Is your feature request related to a problem? Please describe. To reduce network traffic for LDIO outputs, we should move batch collecting from the Repository Sink upwards to all LDIO Output components.

Describe the solution you'd like At the LdioSender, collect members until they either reach a batch threshold or a timeout. After that, send them to our Outputs as combined model (to discuss on this decision)

Yalz commented 1 month ago

To be detailed in https://vlaamseoverheid.atlassian.net/browse/VSDSPUB-1415