Closed aoli-al closed 1 month ago
Hi @jpountz and @benwtrent, I saw your previous discussion about the test failure #13127 . Do you think this patch will help you to replay the failure? Also, let me know if you have any questions
This is indeed interesting @aoli-al I will have to dig deeper to see. This part of the codebase is...opaque to say the least :/
I dug a little bit into this. I tried protecting by putting synchronized
on getNextSequenceNumber
that didn't work.
I tried putting synchronize
on the DW when it flushes all threads and calls close
on the DWDQ, that didn't work.
This is exceptionally tricky because the danged deleteQueue
is passed around to many different classes and accessed through many different paths. Its not well encapsulated at all :(
Thanks for confirming this! Yes, I found the bug extremely tricky to trigger while trying to reproduce.
Making DocumentsWriterFlushControl:obtainAndLock
synchronized will make the exception disappear. But I don't know if this will introduce any deadlock to any other tests.
Description
I saw the following assertion failure when running TestSnapshotDeletionPolicy#testMultiThreadedSnapshotting
This failure might also be related to #13446 and #13127. Please apply the following patch to reproduce the failure. It takes ~30 seconds to show the error.
Some observations to reproduce the failure:
Basically, one thread tries to close the DocumentsWriterDeleteQueue, but other threads still try to call
getNextSequenceNumber
. Here is the happens-before order:Version and environment details
Commit: 33a4c1d8ef999902dacedde9c7f04a3c7e2e78c9 Java version: openjdk 21.0.3 2024-04-16