ZoroXing / xmemcached

Automatically exported from code.google.com/p/xmemcached
Apache License 2.0
0 stars 0 forks source link

memcached on localhost and other servers causing key duplicated #189

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago
I had a problem with our current configuration, I'll try to explain and maybe 
could something similar to a bug but I am not 100% sure.

Our configuration is as follows:
 * array of servers (jboss) each one with a client
 * On some servers (1 & 2) we have also one memcached per node (mc1 & mc2)
 * We have logical names to refer to each server, something like server1,server2,server3,etc. But this names do have the best resolution, using the address 127.0.0.1 as the address of server1 when we are on server1 (this is my problem)

Each server is configured to the mc1 and mc2, but as internally the resolution 
is not the same finally they are having different hash for different servers. 
Obviously changing to the IP it does work on any server, but this is  not too 
much flexible for the future (we use EC2 instances which change their IP on 
some situations).

Probably is not a problem is just a matter of configuration, but was natural to 
me configure directly the servers with their names.

As a possible improvement would be to try to resolve the IP to the external IP 
if is really available and finally this IP is used for the hashing algorithm.

Original issue reported on code.google.com by rrami...@netquest.com on 28 Mar 2012 at 5:05

GoogleCodeExporter commented 8 years ago
It seems it was caused by DNS resolve.

Please add memcached servers to your clients machines /etc/hosts to avoid DNS 
resolve.

1.4.2 was released,it will try to keep the session map in 
KetamaMemcachedSessionLocator in consistent.

Original comment by killme2...@gmail.com on 19 Jul 2013 at 6:51

GoogleCodeExporter commented 8 years ago
In fact is the way we had and it didn't worked. Seems that the local 
resolution, in case I have in the hosts file more than one entry for the same 
machine (like localhost -> 127.0.0.1, mc1 -> X.Y.Z.K, hostname "mc1"), I don't 
know why but java resolves the hostname "mc1" to 127.0.0.1 instead of X.Y.Z.K

Original comment by rober...@gmail.com on 19 Jul 2013 at 6:57

GoogleCodeExporter commented 8 years ago

What's the command of 'hostname' output?

I think you don't set your machine's hostname properly.

Original comment by killme2...@gmail.com on 19 Jul 2013 at 6:59

GoogleCodeExporter commented 8 years ago
The result of hostname is "mc1".

We don't have this configuration anymore, we are using directly the IP's to 
configure them. As is an old configuration (we do not have changed it for a 
while) I did not remembered well the configuration, so in previous comment was 
not right. 

What we have on our server 1 (with external ip1) is something like:
127.0.0.1 server1 localhost localhost.localdomain

And on server 2 (with external ip2) we have:
127.0.0.1 server2 localhost localhost.localdomain

Both of this servers has the memcached configured. Also we have a resolution on 
to the servers to the opposite one and when crossing resolutions the ip's 
resolved are obviously the correct ones (externals) but when resolving from one 
server itself the ip is always 127.0.0.1, so finally we have:

server1 memcached configuration.
     SERVERS: server1:11211 server2:11211

server2 memcached configuration.
     SERVERS: server1:11211 server2:11211

But the thing that is calculated for the servers was (we are accessing it 
through JMX):
server1.
     SERVERS: 127.0.0.1:11211 ip2:11211

server2.
     SERVERS: ip1:11211 127.0.0.1:11211

Don't know if it's clear, but I think the configuration can be like in a normal 
situation, or maybe we are doing something wrong.

Thanks in advance.

Original comment by rober...@gmail.com on 19 Jul 2013 at 7:17

GoogleCodeExporter commented 8 years ago
Thanks for your comment.

I think the problem is you don't put the localhost's hostname to /etc/hosts.

We put our all memcached machines internal ip address and hostname to every 
client machine's /etc/hosts,even if the memcached is running on client 
machine.For example in /etc/hosts:

10.124.57.235   p1
10.124.57.236   p2
10.124.57.237   p3

The hosts list is consistent in all client machines,then we use 'p1:11211 
p2:11211 p3:11211' as our memcached server list,it works well.

Original comment by killme2...@gmail.com on 19 Jul 2013 at 7:32

GoogleCodeExporter commented 8 years ago
Thanks for your help too!

Ok I see what you proposed. 

I think just adding this is not good enough because then you will have like 
internet traffic (not loopback) for every connection made to the current 
server. I mean, If I am on server1 and made a ping to server1 I will have to go 
to the external IP (10.124.57.235) instead of the internal one (127.0.0.1) 
which I think is not a good system configuration.

But with your suggestion I think what is more ideal is to have two names, one 
for the server itself and another name for the memcached which is on the same 
server (which is just circumstantial). The second name (let's call it mc1) can 
have an external IP like you proposed and then the resolution will never be 
127.0.0.1, so will be consistent between all the servers.

I assume is what you was trying to tell me at the first comment, so sorry for 
the inconveniences and I think now I got it :P 

If I am right with the interpretation I think we can close this issue and 
thanks you again.

Original comment by rober...@gmail.com on 20 Jul 2013 at 9:55