MrCroxx / RunKV

[WIP] An experimental cloud-native distributed KV engine for OLTP workload.
MIT License
68 stars 7 forks source link

bug: some requests are not applied or applied out of order without error #142

Closed MrCroxx closed 2 years ago

MrCroxx commented 2 years ago

When running test_multi_raft_group_concurrent_put_get with 100 loops for each group, sometimes assert failed.

e.g.

thread 'test_multi_raft_group_concurrent_put_get::test_multi_raft_group_concurrent_put_get' panicked at 'assertion failed: `(left == right)`
  left: `[118, 52, 52, 52, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 48]`,
 right: `[]`', tests/integrations/test_multi_raft_group_concurrent_put_get.rs:214:21

L214 asserts the result of get after put

https://github.com/MrCroxx/RunKV/blob/b0d596c93ef2439241eaf6d5181228f901b68b41/tests/integrations/test_multi_raft_group_concurrent_put_get.rs#L184-L225

which should be None but got unexpected result.

When digging into the trace log, I found that the delete was not applied:

[2022-05-07T06:16:17Z TRACE runkv_wheel::components::lsm_tree] put; self=ObjectStoreLsmTree { raft_node: 52 } key=b"k444                                                            0" value=b"v444                                                            0" sequence=36656013143 apply_index=6530
[2022-05-07T06:16:18Z TRACE runkv_wheel::components::lsm_tree] get; self=ObjectStoreLsmTree { raft_node: 52 } key=b"k444                                                            0" sequence=36656013147
MrCroxx commented 2 years ago

Seems caused by key generation rule by integration tests.......

Must be drunk...

https://github.com/MrCroxx/RunKV/blob/98d8e446f2464ea7c1479039182014c6e1e99381/tests/integrations/test_multi_raft_group_concurrent_put_get.rs#L182-L187

i should be c...