Introduce Mutex Map in IndexInfo to Ensure Thread Safety During Index Construction

[Beihao-Zhou]

Search before asking

• [X] I had searched in the issues and found no similar issues.

Motivation

In the context of index, the absence of a locking mechanism could lead to issues when multiple threads attempt to modify shared resources simultaneously.

[1] Original issue raised from HNSW #2481 [2] Marked TODO https://github.com/apache/kvrocks/blob/3408318934cb68c0c1f2ffa937f19f02346d0672/src/server/redis_connection.cc#L522-L528

We might want to introduce locks to solve the problem.

Solution

Solution 1: Mutex (Straight-forward)

We can introduce a mutex map within each IndexInfo structure. This mutex map would contain a mutex for each field that requires protection against concurrent access.

Con

• Blocking if multiple connections update the same index

Solution 2: Queue

We can store the Update task for each IndexInfo field in a queue, and have a background thread to schedule tasks in order asynchronously. I'm not quite sure about how many number of threads would be appropriate, but can discuss further if everyone leans towards this solution.

cc @PragmaTwice

Are you willing to submit a PR?

• [X] I'm willing to submit a PR!

[PragmaTwice]

We can first try to implement the first method because it is simpler and does ensure correctness of indexing.

We can then benchmark to determine its impact on performance and consider how to further optimize it.