search

streamnative / bookkeeper-achieved

Apache Bookkeeper
https://bookkeeper.apache.org
Apache License 2.0
3 stars 2 forks source link

ISSUE-2574: BKAsyncLogWriter swallows rootcause of the WriteException #315

Closed sijie closed 3 years ago

sijie commented 3 years ago

Original Issue: apache/bookkeeper#2574

BUG REPORT

Describe the bug

Hard to troubleshoot error in case BKAsyncLogWriter throws "writer has been closed due to error"

To Reproduce

got the exception like 

17:06:19.441 [grpc-default-executor-390] ERROR org.apache.bookkeeper.stream.storage.impl.kv.TableStoreImpl - Failed to process put request key: ...
java.util.concurrent.CompletionException: org.apache.distributedlog.exceptions.WriteException: Write rejected because stream ... has encountered an error : writer has been closed due to error.
    at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:331) ~[?:?]
    at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:346) ~[?:?]
    at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:870) [?:?]
    at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:883) [?:?]
    at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2251) [?:?]
    at org.apache.bookkeeper.common.concurrent.FutureUtils.stats(FutureUtils.java:365) ~[org.apache.bookkeeper-bookkeeper-common-4.11.2.jar:4.11.2]
    at org.apache.distributedlog.BKAsyncLogWriter.write(BKAsyncLogWriter.java:415) ~[org.apache.distributedlog-distributedlog-core-4.11.2.jar:4.11.2]
    at org.apache.bookkeeper.statelib.impl.journal.AbstractStateStoreWithJournal.writeCommandBufReturnTxId(AbstractStateStoreWithJournal.java:510) ~[org.apache.bookkeeper-statelib-4.11.2.jar:4.11.2]
    at org.apache.bookkeeper.statelib.impl.mvcc.MVCCAsyncBytesStoreImpl.writeCommandReturnTxId(MVCCAsyncBytesStoreImpl.java:87) ~[org.apache.bookkeeper-statelib-4.11.2.jar:4.11.2]
    at org.apache.bookkeeper.statelib.impl.mvcc.MVCCAsyncBytesStoreImpl.put(MVCCAsyncBytesStoreImpl.java:111) ~[org.apache.bookkeeper-statelib-4.11.2.jar:4.11.2]
...
Caused by: org.apache.distributedlog.exceptions.WriteException: Write rejected because stream ... has encountered an error : writer has been closed due to error.
    at org.apache.distributedlog.BKAsyncLogWriter.getCachedLogSegmentWriter(BKAsyncLogWriter.java:189) ~[org.apache.distributedlog-distributedlog-core-4.11.2.jar:4.11.2]
    at org.apache.distributedlog.BKAsyncLogWriter.asyncWrite(BKAsyncLogWriter.java:280) ~[org.apache.distributedlog-distributedlog-core-4.11.2.jar:4.11.2]
    at org.apache.distributedlog.BKAsyncLogWriter.write(BKAsyncLogWriter.java:416) ~[org.apache.distributedlog-distributedlog-core-4.11.2.jar:4.11.2]
    ... 29 more

No original exception that caused the WriteException

Expected behavior

rootcause exception in the log