hmsonline / storm-cassandra

Storm Cassandra Integration
Apache License 2.0
179 stars 90 forks source link

Adding limit when copying elements from queue to collection #6

Closed binhnv closed 12 years ago

binhnv commented 12 years ago

I am testing storm and cassandra using this library and I notice that when the input is increased the process latency also increased. I checked our cassandra cluster's statistic and I don't see any write latency issue so I think the time it spends is to copy from queue to array. I looked at the code and it is using drainTo method to copy all available elements from the queue to collection. If there are many elements in the queue then:

binhnv commented 12 years ago

There is an implementation for this here https://github.com/ptgoetz/storm-cassandra/pull/7