search

spring-attic / jdbc

Apache License 2.0
11 stars 18 forks source link

Support for JDBC batch inserts #30

Closed oflasch closed 5 years ago

oflasch commented 5 years ago

We needed support for JDBC batch inserts to enable efficient data ingestion from fast Kafka topics.

This pull request uses Spring Integration aggregators to collect up to jdbc.batch-size messages before inserting them into the database. The configuration option jdbc.idle-timeout can be used to define a timeout in milliseconds after which the collected messages are flushed to the database, limiting the maximum latency.

pivotal-issuemaster commented 5 years ago

@oflasch Please sign the Contributor License Agreement!

Click here to manually synchronize the status of this Pull Request.

See the FAQ for frequently asked questions.

pivotal-issuemaster commented 5 years ago

@oflasch Thank you for signing the Contributor License Agreement!

sabbyanandan commented 5 years ago

@oflasch: Nicely done. Thank you for the contribution!

Could you confirm which version of the jdbc-sink you're using in production? I'm trying to understand whether if we need to backport this enhancement to a previous GA release.

oflasch commented 5 years ago

@sabbyanandan: Thank you.

We plan to start using the sink in production in January. A backport of this enhancement to the most recent GA release would be very helpful. Please let me know if I can help.

artembilan commented 5 years ago

Merged as https://github.com/spring-cloud-stream-app-starters/jdbc/commit/6b63922705a6eed6995dd449a45c29b447e925e3.

@oflasch ,

Thank you very much for the contribution. Looking forward for more!

And yes: Happy New Year!

oflasch commented 5 years ago

@artembilan ,

Happy New Year to you, too. And thank‘s for your patience!