leeoo / spymemcached

Automatically exported from code.google.com/p/spymemcached
0 stars 0 forks source link

spymemcached hangs under load ... #261

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago
Hi all.
Our environment:
------
OS: CentOS 5 
2.6.18-238.el5 #1 SMP x86_64 x86_64 x86_64 
JVM: Oracle JRockit(R) (build 
R28.1.3-11-141760-1.6.0_24-20110301-1432-linux-x86_64
memcached - 1.4.10
spymemcached -2.6
-------
Recently we start to get next hangs in spymemcached:
--------
"479483953@qtp-278043096-13494" id=99467 idx=0x750 tid=25001 prio=5 alive, 
parked, native_blocked
    at jrockit/vm/Locks.park0(J)V(Native Method)
    at jrockit/vm/Locks.park(Locks.java:2230)
    at jrockit/proxy/sun/misc/Unsafe.park(Unsafe.java:616)[inlined]
    at java/util/concurrent/locks/LockSupport.parkNanos(LockSupport.java:198)[inlined]
    at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)[optimized]
    at java/util/concurrent/ArrayBlockingQueue.offer(ArrayBlockingQueue.java:287)[optimized]
    at net/spy/memcached/protocol/TCPMemcachedNodeImpl.addOp(TCPMemcachedNodeImpl.java:271)[optimized]
    at net/spy/memcached/MemcachedConnection.addOperation(MemcachedConnection.java:602)[optimized]
    at net/spy/memcached/MemcachedConnection.addOperation(MemcachedConnection.java:582)[inlined]
    at net/spy/memcached/MemcachedClient.addOp(MemcachedClient.java:277)[inlined]
    at net/spy/memcached/MemcachedClient.asyncGet(MemcachedClient.java:797)[inlined]
    at net/spy/memcached/MemcachedClient.asyncGet(MemcachedClient.java:811)[optimized]
    at ru/cti/memcached/MemcachedManager.getObjectFromMemcached(MemcachedManager.java:405)[optimized]

----
memcached-top show:
memcache-top v0.6       (default port: 11211, color: on, refresh: 3 seconds)

INSTANCE                USAGE   HIT %   CONN    TIME    EVICT/s READ/s  WRITE/s
127.0.0.1:11211         88.9%   86.9%   26      0.6ms   0.7     181.3K  3.3M

AVERAGE:                88.9%   86.9%   26      0.6ms   0.7     181.3K  3.3M

TOTAL:          4.3GB/  4.9GB           26      0.6ms   0.7     181.3K  3.3M

----
But memcached works OK, because nginx get data from it successfully.
Why spy-memcached can hangs ?
Thanks in advance.

Original issue reported on code.google.com by JohnBa...@gmail.com on 2 Nov 2012 at 1:13