search

zkteco-home / redis-windows

Native port of Redis for Windows,it can be installed as service.
MIT License
1.84k stars 167 forks source link

Scan is malfunction. #5

Closed HoytRen closed 2 years ago

HoytRen commented 2 years ago

Hello, I have another issue. I tried both scan and sscan, and they return some duplicated items which could not be right. The options seems don't work too. I'm not sure of hscan and zscan.

zkteco-home commented 2 years ago

Hello, I have another issue. I tried both scan and sscan, and they return some duplicated items which could not be right. The options seems don't work too. I'm not sure of hscan and zscan.

I will fix it ASAP,THANKS

HoytRen commented 2 years ago

By the way, after I insert about 80,000 keys, the keys * or smembers hang up for an hour with 0% cpu, and I have to kill it.

zkteco-home commented 2 years ago

Hello, I have another issue. I tried both scan and sscan, and they return some duplicated items which could not be right. The options seems don't work too. I'm not sure of hscan and zscan.

I fixed this issue,please test it and tell me result,thanks

zkteco-home commented 2 years ago

By the way, after I insert about 80,000 keys, the keys * or smembers hang up for an hour with 0% cpu, and I have to kill it.

i didnt test this issue,maybe related to that issue,please test it again,if there is still issue,please tell me,i will fix it.

HoytRen commented 2 years ago

I tried scan, it seems work, and the options works now. I need more time to check missing and duplicates. The keys * still don't work.. However for a large db, it's not usefull, when it exceded, I suggest throw an error instead of hang up, because the cli process has to be killed to continue.

zkteco-home commented 2 years ago

I tried scan, it seems work, and the options works now. I need more time to check missing and duplicates. The keys * still don't work.. However for a large db, it's not usefull, when it exceded, I suggest throw an error instead of hang up, because the cli process has to be killed to continue.

i tested keys *,it work fiine, image please send me you screenshot,thanks

"The keys * still don't work",i guess you meant for large db.right?

HoytRen commented 2 years ago

When here are few keys, it will work, but when here are thousends of keys, it don't work. I tried on some commercial redis instance, same keys are returned in about 1 scound, then the windows version seems has problem.

Another problem, both the win and commercial instance have, I'm not sure you could fix it or I should turn to commercial supplier or redis official team. That is after I did a full scan, which means the cursor from 0 to 0. I find the total output is right, but here are duplicates, which means some records are missing. I'm not modify the db during scan. According to redis document about scan, here should be only duplicates but not missing, and when here are not modification, the result should be exact.

zkteco-home commented 2 years ago

i updated it,now i just export 20000 keys at most,if it is stable,next version i will add new parameter to control key numbers

zkteco-home commented 2 years ago

When here are few keys, it will work, but when here are thousends of keys, it don't work. I tried on some commercial redis instance, same keys are returned in about 1 scound, then the windows version seems has problem.

Another problem, both the win and commercial instance have, I'm not sure you could fix it or I should turn to commercial supplier or redis official team. That is after I did a full scan, which means the cursor from 0 to 0. I find the total output is right, but here are duplicates, which means some records are missing. I'm not modify the db during scan. According to redis document about scan, here should be only duplicates but not missing, and when here are not modification, the result should be exact.

the following is source code image it looks simply,like what you said,if commercial redis can return in 1 second,i also can implement it,i need some time to check reason.

about you mentioned duplicate issue,i checked code,it will check if existed,it means key is unique,maybe different DB? image

HoytRen commented 2 years ago

I tried, and it seems don't work. 'keys *' still frozen. Here must be misunderstand. I don't think the problem is because of here are too many keys. Here must be a bug cause this, because the CPU stay on 0%, it's idle but not work at all. It's just a suggestion that limit the returned keys, and it may not be necessary after the real bug is addressed.

About the duplicates and missing. Here are misunderstand too. The SET never have duplicates, but when I retrieve the members of the SET by scan, I get duplicates and missing. I contacted my commercial supplier already, and they are checking, but no feedback so far.

PS C:\Users\Administrator\source\redis-windows> .\redis-cli.exe 127.0.0.1:6379> info keyspace

Keyspace

db0:keys=82062,expires=0,avg_ttl=0 db3:keys=13841,expires=0,avg_ttl=0 db4:keys=133844,expires=0,avgttl=0 127.0.0.1:6379> keys * (frozen)

zkteco-home commented 2 years ago

please update new version,i fixed about 80,000 keys issue,that is bcz there is bug.not bcz of large db.now you can get 100k keys in a second. thanks for your issue,i didnt notice this.

HoytRen commented 2 years ago

I tried the new version. It works now. I'm very happy and thank you very much.