search

bigdata4u / spymemcached

Automatically exported from code.google.com/p/spymemcached
0 stars 0 forks source link

java.lang.AssertionError: Unknown line type: STORED #317

Open GoogleCodeExporter opened 8 years ago

GoogleCodeExporter commented 8 years ago 
Using spymemcached 2.10.0

I saw this happen in the error log of a very long running service:

<pre>
 {MemcachedConnection to localhost/127.0.0.1:11211}] UncaughtExceptionLogger - Uncaught exception in thread {MemcachedConnection to localhost/127.0.0.1:11211}
 {MemcachedConnection to localhost/127.0.0.1:11211}] UncaughtExceptionLogger -      java.lang.AssertionError: Unknown line type: STORED
 {MemcachedConnection to localhost/127.0.0.1:11211}] UncaughtExceptionLogger -      at net.spy.memcached.protocol.ascii.BaseGetOpImpl.handleLine(BaseGetOpImpl.java:117)
 {MemcachedConnection to localhost/127.0.0.1:11211}] UncaughtExceptionLogger -      at net.spy.memcached.protocol.ascii.OperationImpl.readFromBuffer(OperationImpl.java:153)
 {MemcachedConnection to localhost/127.0.0.1:11211}] UncaughtExceptionLogger -      at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:628)
 {MemcachedConnection to localhost/127.0.0.1:11211}] UncaughtExceptionLogger -      at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:543)
 {MemcachedConnection to localhost/127.0.0.1:11211}] UncaughtExceptionLogger -      at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:322)
 {MemcachedConnection to localhost/127.0.0.1:11211}] UncaughtExceptionLogger -      at net.spy.memcached.MemcachedConnection.run(MemcachedConnection.java:1047)
 - Timeout waiting for bulk values: waited 2,500 ms. Node status: Connection Status { localhost/127.0.0.1:11211 active: true, authed: true, last read: 2,501 ms ago }
 -        net.spy.memcached.OperationTimeoutException: Timeout waiting for bulk values: waited 2,500 ms. Node status: Connection Status { localhost/127.0.0.1:11211 active: true, authed
 -        at net.spy.memcached.MemcachedClient.getBulk(MemcachedClient.java:1462)
 -        at net.spy.memcached.MemcachedClient.getBulk(MemcachedClient.java:1495)
</pre>

After a while this happened:
<pre>
 - Shutting down
 -     java.lang.IllegalStateException: Shutting down
 -     at net.spy.memcached.MemcachedConnection.checkState(MemcachedConnection.java:1035)
 -     at net.spy.memcached.MemcachedConnection.enqueueOperation(MemcachedConnection.java:825)
 -     at net.spy.memcached.MemcachedClient.asyncStore(MemcachedClient.java:315)
 -     at net.spy.memcached.MemcachedClient.set(MemcachedClient.java:833)
</pre>

This resulted in all future memcache operations failing until the service was 
restarted.

Original issue reported on code.google.com by kristofer.karlsson@gmail.com on 8 Feb 2015 at 8:49

GoogleCodeExporter commented 8 years ago 
This seems like a corrupt communication state between the client and server 
(some request/responses being dropped or shuffled?)
I suppose we can't know if the problem is in memcached or spymemcached, but 
there could probably be some better way of repairing itself once the problem is 
detected.
(Fail all outstanding requests and reconnect?)

Original comment by kristofer.karlsson@gmail.com on 8 Feb 2015 at 8:58