Current TableWriter API is single-thread one writer instance at a time. For use cases where table data originates from multiple cases a tangible piece of infrastructure is missing. Such infrastructure is present in ILP TCP server code. We would like to make this code reusable and generic. The goal is that all ingestion paths can leverage new API to send data to table from multiple, share-nothing threads
Intended outcome
New Java-level API, similar to TableWriter.Row to be used concurrently by multiple threads
Java-level Insert API will migrate to the new API
ILP server will migrate to the new API
REST text import will migrate to the new API
How will it work
We would like to remove all source of "Table busy" error message. All external APIs will remain the same. New Row API will be additive and existing embedded users can migrate gradually.
Summary
Current
TableWriterAPI is single-thread one writer instance at a time. For use cases where table data originates from multiple cases a tangible piece of infrastructure is missing. Such infrastructure is present in ILP TCP server code. We would like to make this code reusable and generic. The goal is that all ingestion paths can leverage new API to send data to table from multiple, share-nothing threadsIntended outcome
TableWriter.Rowto be used concurrently by multiple threadsInsertAPI will migrate to the new APIHow will it work
We would like to remove all source of "Table busy" error message. All external APIs will remain the same. New
RowAPI will be additive and existing embedded users can migrate gradually.