Command Migration: ('DEL', 'EXISTS', 'PERSIST', 'TYPE')

AshwinKul28 commented 4 weeks ago

This issue tracks the migration of the mentioned commands - ('DEL', 'EXISTS', 'PERSIST', 'TYPE') to make them compatible across the three protocols supported by the Dice database: RESP, HTTP, and WebSocket. DiceDB now supports more than one protocols (Resp/http/websocket). We don't want eval operation of each command to be strictly bound with any of the protocols. Currently eval function return statements of each command is bounded to the RESP protocol.

The migration is required because:

The current implementation is specific to the RESP protocol
Evaluation functions of each command must be refactored to ensure protocol-agnostic behavior.
Evaluation functions should return raw responses instead of protocol specific responses.

The goal is to make the command logic protocol-independent, allowing all three protocols to call the same core functionality seamlessly.

Requirements

Refactor the evaluation function of [Command Name] to be generic.
Ensure the evaluation function does not include protocol-specific logic.

Migration Steps

Analyze Current Implementation
- Review the current codebase to understand how the command logic is implemented.
Refactor Return Logic
- Create a new function with the function definition as evalXXX(args []string, store *dstore.Store) *EvalResponse under the file /internal/eval/store_eval.go
- Analyse return statements of the eval and modify them to send raw types without encoding
- Use errors in the return statements from /errors/migrated_errors.go file
- Use perdefined responses from /internal/clientio/resp.go file as RespType variables
Command/Worker specific Changes
- Make IsMigrated flag to true in the commands information under /internal/eval/commands.go file.
- Use the newly written eval function against the NewEval parameter in the same command structure.
- Delete the old eval function from the /internal/eval/eval.go file.
- Add the migrated command to the /internal/worker/CommandsMeta map and make it's type as SingleShard
Update Unit Tests
- Refactor existing unit tests to accommodate the new implementation.
- Add new unit tests if necessary to cover all possible cases.
Integration Tests
- Run all integration tests to ensure successful migration.
- Ensure that each protocol (RESP, HTTP, WebSocket) works correctly after migration.

Checklist

[ ] Migrated the evalXXX function with the latest definition
[ ] Update or add unit tests for the new implementation.
[ ] All unit tests pass successfully.
[ ] Ensure all integration tests pass successfully.

Additional Notes

Describe any edge cases that need to be handled in the generic function.
Mention any protocol-specific optimizations required in wrappers.

If there are any questions or concerns about this migration, please mention them here.

Related Issues/PRs

Sample implementation for the Get, Set, GetSet and SetEx can be found in this Pull Request.

parzival272000 commented 4 weeks ago

Hi, Can you assign this to me ? Regards, Mayank Jha

AshwinKul28 commented 4 weeks ago

Thanks @parzival272000 go for it. 🚀

mohit-nagaraj commented 3 weeks ago

@parzival272000 whts the status?

AshwinKul28 commented 3 weeks ago

HI @parzival272000 , I hope you are doing well. Do you happen to have any updates on this? If you've any doubts please let us know on the discord.

parzival272000 commented 3 weeks ago

Hi @AshwinKul28 couldn't progress with this yet. You can allot it to someone else, am preoccupied with some stuff

AshwinKul28 commented 3 weeks ago

Sure thanks @parzival272000 ,

AshwinKul28 commented 3 weeks ago

@mohit-nagaraj if you are still interested, go for it! 🚀

mohit-nagaraj commented 3 weeks ago

Sure thanks on it

soumya-codes commented 2 weeks ago

@mohit-nagaraj hope you are doing well. Any updates on the progress? We are targeting these issues little aggressively as we maintaining the commands across two separate servers is getting very cumbersome.

mohit-nagaraj commented 2 weeks ago

hey i do get it raising a pr tmr

mohit-nagaraj commented 2 weeks ago

Hey i need to ask a few doubts before raising a pr How to contact one of u guys

apoorvyadav1111 commented 2 weeks ago

Hey, you can reach out to us here or on the discord community.

mohit-nagaraj commented 2 weeks ago

so while writing test cases how should i write it, could u give a brief overview, i am following this pr https://github.com/DiceDB/dice/pull/1097/files#diff-3941f5aad18919b2441c1d32cc30a869cb791fd4709ff27d8f34b61f0c4c0455

should i just follow it to make the test cases like them? i am little new to writing test cases so..... Wht modifications should i do

mohit-nagaraj commented 2 weeks ago

Btw i saw prs updating even internal/server/cmd_meta.go

should i also update it

mohit-nagaraj commented 2 weeks ago

@apoorvyadav1111 @AshwinKul28 @soumya-codes Theres this issue with naming convection for type

JyotinderSingh commented 2 weeks ago

so while writing test cases how should i write it, could u give a brief overview, i am following this pr https://github.com/DiceDB/dice/pull/1097/files#diff-3941f5aad18919b2441c1d32cc30a869cb791fd4709ff27d8f34b61f0c4c0455

should i just follow it to make the test cases like them? i am little new to writing test cases so..... Wht modifications should i do

You mostly just need to move the existing tests from async folder to resp folder. Then check if those tests already also exist inside HTTP and websockets folder. If they do, well and good, otherwise, add the same tests in these folders (in HTTP/websocket format, refer to existing tests for these).

A good idea is to see the different PRs already submitted (go to PRs tab, and replace the is:open filter with is:closed) and gather inspiration from them.

JyotinderSingh commented 2 weeks ago

Btw i saw prs updating even internal/server/cmd_meta.go

should i also update it

Yes

mohit-nagaraj commented 2 weeks ago

so while writing test cases how should i write it, could u give a brief overview, i am following this pr https://github.com/DiceDB/dice/pull/1097/files#diff-3941f5aad18919b2441c1d32cc30a869cb791fd4709ff27d8f34b61f0c4c0455 should i just follow it to make the test cases like them? i am little new to writing test cases so..... Wht modifications should i do

You mostly just need to move the existing tests from async folder to resp folder. Then check if those tests already also exist inside HTTP and websockets folder. If they do, well and good, otherwise, add the same tests in these folders (in HTTP/websocket format, refer to existing tests for these).

A good idea is to see the different PRs already submitted (go to PRs tab, and replace the is:open filter with is:closed) and gather inspiration from them.

Sure thanks for this info

mohit-nagaraj commented 2 weeks ago

hey @JyotinderSingh I have added all the test cases thing, but when i tried running the proj, i was getting some error "memory leak". I was able to revert n check one by one to narrow it down to the function. Now i have identified its coming from evalPERSIST. There seems to be some error which is occuring when migrating. Will go through it in detail later. If you have any clue about this do help me out

AshwinKul28 commented 1 week ago

HI @mohit-nagaraj anything specific did you observe on the "memory leak" please mention it here, since it has not happened before migration, I'm curious to know what's the exact issue. It would be great if you could explain it here or on the discord, and we can try to find some solution to it, so you will be unblocked. We need to move little faster now in the migration work, so please reach out to us for any blockers on discord. Thanks a lot!

mohit-nagaraj commented 1 week ago

@AshwinKul28 Hey i fixed off that thing, it was due to some error in wrongly calling the the persist function. The migration is completely done. I am working on test cases which will be done by today eod

DiceDB / dice