DiceDB / dice

DiceDB is a redis-compliant, reactive, scalable, highly-available, unified cache optimized for modern hardware.
https://dicedb.io/
Other
6.83k stars 1.08k forks source link

Command migration for single shard, multi-shard and custom commands #1276

Closed AshwinKul28 closed 1 week ago

AshwinKul28 commented 1 week ago

This PR migrates multiple DiceDB commands to support both multi-shard and single-shard scenarios. The migration follows a structured flow for each command type as described below.

Multi-Shard Commands The following commands are migrated with support for multi-shard processing:

  1. MSET
  2. MGET
  3. SINTER
  4. SDIFF
  5. JSON.MGET

For the COPY and RENAME commands, the flow includes preprocessing, followed by decomposition and composition steps to manage multi-shard data efficiently. For the remaining multi-shard commands, only decomposition and composition steps are involved.

Single-Shard Commands The following commands are migrated with support for single-shard processing:

  1. JSON.SET
  2. JSON.GET
  3. JSON.TYPE
  4. JSON.INGEST
  5. JSON.HGETALL
  6. JSON.STRAPPEND
  7. Dump
  8. Restore

These single-shard commands follow a scatter-gather approach with a straightforward flow, where each command operates directly within the target shard, without the need for complex decomposition.

JyotinderSingh commented 1 week ago

Looks like a conflict came up. @AshwinKul28 please feel free to resolve conflict and merge immediately.

AshwinKul28 commented 1 week ago

@apoorvyadav1111 @JyotinderSingh any idea why LPush tests are failing? Also HTTP context cancelled error is coming out of nowhere :(

JyotinderSingh commented 1 week ago

Thanks for the massive effort @AshwinKul28, and thanks for all the debugging help @apoorvyadav1111. We are now much closer to a fully multi-threaded system!