Add integration test for testing WHERE clause in QWATCH.

JyotinderSingh commented 3 months ago

WHERE clause support was added to DSQL in #187. However, we don't have any integration test cases validating its behavior.

Requirements

Add test cases that validate E2E flow for QWATCH queries using the WHERE clause.
See tests/qwatch_test.go for reference.
benchmark the code and measure the time taken and memory allocs using benchmem and try to keep them to the bare minimum.
Add integration test results as a screenshot in the PR description (since we don't run integration tests on the CI for now)

rsdel2007 commented 3 months ago

Hi @JyotinderSingh I would like to pick this.

JyotinderSingh commented 3 months ago

Hi @JyotinderSingh I would like to pick this.

Assigned

JyotinderSingh commented 3 months ago

Issue is unassigned now. Feel free to pick this up.

deep-adeshraa commented 3 months ago

Hi @JyotinderSingh I would like to pick this.

JyotinderSingh commented 3 months ago

Hi @JyotinderSingh I would like to pick this.

Assigned.

deep-adeshraa commented 2 months ago

@JyotinderSingh is there any way to overcome comparison issue between string and integer. I am setting up integer in SDK test case but while comparison in where condition, it is giving me error that can't compare string and integer.

JyotinderSingh commented 2 months ago

@JyotinderSingh is there any way to overcome comparison issue between string and integer. I am setting up integer in SDK test case but while comparison in where condition, it is giving me error that can't compare string and integer.

The issue was fixed recently, please pull the latest master

AshwinKul28 commented 2 months ago

@deep-adeshraa Thanks a lot for picking this up. Do you have any updates on it? If you have any blockers please discuss this over the discord.

deep-adeshraa commented 2 months ago

Hi @AshwinKul28 I have started working on it but having some difficulties in understanding code. I will try to push asap

deep-adeshraa commented 2 months ago

Hi @JyotinderSingh @AshwinKul28 - I am working on benchmarking the Qwatch, however I am not sure how to benchmark the code. I am checking BenchmarkLargeByteArray2 tests and seems like we run the function for large number of inputs and check the time taken. Could you please give me some suggestions what would be the best way to benchmark Qwatch ?

JyotinderSingh commented 2 months ago

Hi @JyotinderSingh @AshwinKul28 - I am working on benchmarking the Qwatch, however I am not sure how to benchmark the code. I am checking BenchmarkLargeByteArray2 tests and seems like we run the function for large number of inputs and check the time taken. Could you please give me some suggestions what would be the best way to benchmark Qwatch ?

A good strategy might be to check performance of watch queries with high number of incoming updates to the query keyset.

The watch query will be looking at a rapidly changing keyset, we should see how long it takes for each new query result to get delivered.

Additionally, another benchmark can be to validate query performance with several queries running in parallel. In such a case you could benchmark both query and SET performance (because each SET operation will compete against the query Executor for lock on the dataset in the current implementation)

arpitbbhayani commented 2 months ago

Hello @deep-adeshraa,

There has been no activity on this issue for the past 5 days. It would be awesome if you keep posting updates to this issue so that we know you are actively working on it.

We are really eager to close this issue at the earliest, hence if we continue to see the inactivity, we will have to reassign the issue to someone else. We are doing this to ensure that the project maintains its momentum and others are not blocked on this work.

Just drop a comment with the current status of the work or share any issues you are facing. We can always chip in to help you out.

Thanks again.

deep-adeshraa commented 2 months ago

Hello @arpitbbhayani - I have created PR for integration test cases. However for Benchmark JyotiderSingh asked me to wait until some fixes are deployed. The reason was QWATCH command was failing if I fire large amount of concurrent requests.

arpitbbhayani commented 1 month ago

Hello @deep-adeshraa,

There has been no activity on this issue for the past 5 days. It would be awesome if you keep posting updates to this issue so that we know you are actively working on it.

We are really eager to close this issue at the earliest, hence if we continue to see the inactivity, we will have to reassign the issue to someone else. We are doing this to ensure that the project maintains its momentum and others are not blocked on this work.

Just drop a comment with the current status of the work or share any issues you are facing. We can always chip in to help you out.

Thanks again.

psrvere commented 1 month ago

@JyotinderSingh - please assign to me.

arpitbbhayani commented 1 month ago

Hello @psrvere,

There has been no activity on this issue for the past 5 days. It would be awesome if you keep posting updates to this issue so that we know you are actively working on it.

We are really eager to close this issue at the earliest, hence if we continue to see the inactivity, we will have to reassign the issue to someone else. We are doing this to ensure that the project maintains its momentum and others are not blocked on this work.

Just drop a comment with the current status of the work or share any issues you are facing. We can always chip in to help you out.

Thanks again.

DiceDB / dice

Add integration test for testing WHERE clause in QWATCH. #188

Requirements