search

magro / memcached-session-manager

A tomcat session manager that backups sessions in memcached and pulls them from there if asked for unknown sessions
Apache License 2.0
759 stars 346 forks source link

Failover Issues - Session Not Being Stored to ElastiCache? #346

Open scott-wood-vgh opened 7 years ago

scott-wood-vgh commented 7 years ago

Details: My application runs on AWS EC2 instances backed by an Elasticache per trio. The instances are fronted by an ELB with LoadBalancer stickiness enabled. On memcached-session-manager/tc7-1.9.6.jars and the AmazonElastiCacheClusterClient-no-version.jar. In my application, I'm logging the session ID on login, i.e. "User logged in with session 9B288A71532ED1AB94C5E63A8575A21C".

When I stop the Tomcat service and expect a failover, the user is prompted to log back in. I checked the ElastiCache nodes both before (i.e. right after login) and after the failover, and I can't find a session with that ID, though there are quite a few key/value pairs of similar sessions.

I'm wondering if anyone knows any reason why the session would seemingly be failing to get to memcached, and if there is any additional debug logging I can enable to see what's happening if not. I recently upgraded the instances to Ubuntu 16.04/Tomcat 7.0.68, as well as memcached from 1.9.2 to 1.9.6, and am wondering if I can correlate an infrastructure issue, or if there is an app change at fault.

scott-wood-vgh commented 7 years ago

For clarity, here is my context.xml:

WEB-INF/web.xml
aaroncline commented 6 years ago

We are seeing a similar issue. Using AWS Elasticache. Tomcat 8.0.47. The suggested Elasticache config. We recently upgraded Tomcat a couple of weeks ago from Tomcat 8.0.24 to 8.0.47.

I ran a tcpdump on the app servers and could see MSM doing node discover and attempting to delete keys (all deletes failed), but I never saw it try to submit a key to Elasticache.