I am testing storm and cassandra using this library and I notice that when the input is increased the process latency also increased. I checked our cassandra cluster's statistic and I don't see any write latency issue so I think the time it spends is to copy from queue to array. I looked at the code and it is using drainTo method to copy all available elements from the queue to collection. If there are many elements in the queue then:
It takes longer to copy elements to collection
There will be chance of out of memory.
Because of this I suggest to change the code so that caller can set maximum number of items it want to copy.
I am testing storm and cassandra using this library and I notice that when the input is increased the process latency also increased. I checked our cassandra cluster's statistic and I don't see any write latency issue so I think the time it spends is to copy from queue to array. I looked at the code and it is using drainTo method to copy all available elements from the queue to collection. If there are many elements in the queue then: