search

tikv / client-java

TiKV Java Client
https://tikv.github.io/client-java/
Apache License 2.0
111 stars 110 forks source link

`scan` operations for `KvClient` not working correctly #677

Open FiV0 opened 2 years ago

FiV0 commented 2 years ago

Bug Report

I am running into issues with the scan operations for the KvClient. Issue that might be related https://github.com/tikv/client-java/issues/631

1. Describe the bug

The following two operations don't seem to work correctly for the KvClient scan(ByteString startKey, long version, int limit) scan(ByteString startKey, long version) The former is just calling the later so that is kind of expected.

2. Minimal reproduce step

I added 11 key value pairs to tikv.

(["foo" "bar"] ["foo0" "bar"] ["foo1" "bar"] ["foo2" "bar"] ["foo3" "bar"] ["foo4" "bar"]
 ["foo5" "bar"] ["foo6" "bar"] ["foo7" "bar"] ["foo8" "bar"] ["foo9" "bar"])

scan(ByteString startKey, ByteString endKey, long version) works correctly. What follows is in Clojure but can very likely also be produced in Java.

;; the startKey to endKey version works
(->> (.scan client (ByteString/copyFromUtf8 "foo") (ByteString/copyFromUtf8 "foo5") timestamp)
     (map kv-pair->strings))
;; => (["foo" "bar"] ["foo0" "bar"] ["foo1" "bar"] ["foo2" "bar"] ["foo3" "bar"] ["foo4" "bar"])

(.scan client (ByteString/copyFromUtf8 "foo") timestamp 5)
;; => Execution error (IndexOutOfBoundsException) at org.tikv.common.operation.iterator.ScanIterator/cacheLoadFails (ScanIterator.java:100).
;;    current cache size = 11, larger than 5

;; if I specify a limit larger than the number of available keys nothing gets returned.
(.scan client (ByteString/copyFromUtf8 "foo") timestamp 12)
;; => []
(.scan client (ByteString/copyFromUtf8 "foo") timestamp)
;; => []

Here is the stacktrace for the error from the second invocation:

1. Caused by java.lang.IndexOutOfBoundsException
   current cache size = 11, larger than 5

         ScanIterator.java:  100  org.tikv.common.operation.iterator.ScanIterator/cacheLoadFails
 ConcreteScanIterator.java:  106  org.tikv.common.operation.iterator.ConcreteScanIterator/hasNext
             Iterator.java:  132  java.util.Iterator/forEachRemaining
             KVClient.java:  134  org.tikv.txn.KVClient/scan

I suspect that the bug is in the code at this location: https://github.com/tikv/client-java/blob/870a9fb8bc282e4dee4bd25a0305615dafb857a3/src/main/java/org/tikv/common/operation/iterator/ScanIterator.java#L95-L111

3. What did you see instead

See above.

4. What did you expect to see?

Correct scanning operations as described in the API at https://tikv.github.io/client-java/site/apidocs/index.html.

5. What are your Java Client and TiKV versions?

I used the tikv setup describe in this tutorial.

godliness commented 1 week ago

Is there any progress？