We are using rocksdbjni version 9.0.0. We have a usecase where we need to scan for some keys from one column family say CF-1. Let's say prefix scanning from CF-1 resulted in 10 keys. Now in the same method, we goto another column family CF-2 for fetching the value associated with each keys we fetched earlier.
The problem is while we goto CF-2 with a bunch of keys, some other thread invalidates those keys and removes their entries from CF-2. This is causing us problem. I was hoping to take a snapshot of the db and then do perform the read operations and release the snapshot. Apparently this is not helping. Pseudo code is below
public List getOrderDataForUser(String userId) {
Transaction transaction = db.beginTransaction(new WriteOptions());
ThreadLocal threadLocal = new ThreadLocal<>();
threadLocal.set(transaction.getSnapshot());
ReadOptions ro = new ReadOptions();
ro.setSnapshot(threadLocal.get());
var iterator = transaction.getIterator(ro, columnFamilyHandle);
List<String> keys = getKeysForPrefix(userId, iterator); // some function that does prefix scanning and returns the matching keys
//next - go to CF-2 and fetch order data for the keys
List<StoredOrder> orderDataList = transaction.multiGetAsList(defaultReadOptions, cfs, keys)
.stream()
.map(this::safeDeserialize)
.filter(Objects::nonNull)
.collect(Collectors.toList());
transaction.clearSnapshot();
return orderDataList;
}
Expected behavior
snapshot secures a version of db at that point in time, so that deletion by other threads do not disturb the read operations
Actual behavior
snapshot doesn't work. When we goto CF-2 for fetching the value associated with some keys, we notice that some other threads have modified the data
Steps to reproduce the behavior
create a column family
add some key value pairs
begin a transaction and take a snapshot
read value associated with each keys in a loop
make another thread delete of the keys
we can see that data in our snapshot is also modified
@vishaltk It looks to me like you are using different read options objects when accessing the database. If you want to work with the snapshot, you need to consistently use the ReadOptions that has that snapshot.
We are using rocksdbjni version 9.0.0. We have a usecase where we need to scan for some keys from one column family say CF-1. Let's say prefix scanning from CF-1 resulted in 10 keys. Now in the same method, we goto another column family CF-2 for fetching the value associated with each keys we fetched earlier.
The problem is while we goto CF-2 with a bunch of keys, some other thread invalidates those keys and removes their entries from CF-2. This is causing us problem. I was hoping to take a snapshot of the db and then do perform the read operations and release the snapshot. Apparently this is not helping. Pseudo code is below
public List getOrderDataForUser(String userId) {
Transaction transaction = db.beginTransaction(new WriteOptions());
ThreadLocal threadLocal = new ThreadLocal<>();
threadLocal.set(transaction.getSnapshot());
ReadOptions ro = new ReadOptions();
ro.setSnapshot(threadLocal.get());
return orderDataList; }
Expected behavior
snapshot secures a version of db at that point in time, so that deletion by other threads do not disturb the read operations
Actual behavior
snapshot doesn't work. When we goto CF-2 for fetching the value associated with some keys, we notice that some other threads have modified the data
Steps to reproduce the behavior
create a column family add some key value pairs begin a transaction and take a snapshot read value associated with each keys in a loop make another thread delete of the keys
we can see that data in our snapshot is also modified