search

OpenTSDB / asynchbase

A fully asynchronous, non-blocking, thread-safe, high-performance HBase client.
https://github.com/OpenTSDB/asynchbase
BSD 3-Clause "New" or "Revised" License
608 stars 304 forks source link

Calling flush() too often results in a CallbackOverflowError #100

Open tsuna opened 9 years ago

tsuna commented 9 years ago

In an app that uses batching but also aggressively flushes stuff to HBase, a high request rate plus aggressive calls to flush() lead to this:

com.stumbleupon.async.CallbackOverflowError: Too many callbacks in Deferred@1750061096(state=PENDING, result=null, callback=multi-action response -> notify Def
erredGroup@1340811160 -> notify DeferredGroup@1467258149 -> [...]
at com.stumbleupon.async.Deferred.addCallbacks(Deferred.java:669)
at com.stumbleupon.async.Deferred.addBoth(Deferred.java:772)
at com.stumbleupon.async.DeferredGroup.<init>(DeferredGroup.java:119)
at com.stumbleupon.async.Deferred.group(Deferred.java:897)
at org.hbase.async.RegionClient.sync(RegionClient.java:348)
at org.hbase.async.HBaseClient.flush(HBaseClient.java:609)
asdf2014 commented 5 years ago

@tsuna I meet the same issues, any suggestion?

binshi-bing commented 4 years ago

@tsuna @asdf2014 @manolama, I entered the same issue -- any insight and suggestion?

binshi-bing commented 4 years ago

I’m making choice between two approaches — fix AsyncHBase to avoid adding unnecessary callbacks to pending write RPCs due to NSRE; or reduce the number of flush() calls by increasing flush() granualarity in zen/ums and by avoiding aggressive retries/probing on NSRE.