Netflix / dynomite

A generic dynamo implementation for different k-v storage engines
Apache License 2.0
4.2k stars 532 forks source link

SADD inconsistency #823

Open ChananyaWinston opened 11 months ago

ChananyaWinston commented 11 months ago

I have a cluster in two dcs with one node and one rack. regular keys seems to be consistent but I have a set with 65000 ips and I use spop and sadd to take and return an ip (data sessions management) currently my servers are only connecting to one dc and the ip pool in the other dc seems to be emptying but not refilling the dc see that is being updated directly usually has about 61000 ip left in the other dc is going lower and lower I did see the that some do go back in I have hight traffic here is my conf dyn_o_mite: datacenter: dc2 rack: rack1 dyn_listen: 0.0.0.0:8101 dyn_seeds:

I always have 64 client connection to my dynomite that is being interacted with directly

ChananyaWinston commented 11 months ago

I found the problem. I was using SPOP to get an IP and the way dynomite works is that it just does SPOP on all the other nodes so really when i wanted an IP it took out 2 but only one was retuned when the session was over because only was used and recorded. Is this behaviour intentional and I'm just not using SPOP with a cluster correctly or is this a bug?