search

KevinQi1981 / jmemcache-daemon

Automatically exported from code.google.com/p/jmemcache-daemon
Apache License 2.0
0 stars 0 forks source link

daemon does not gracefully handle early-terminated connections #1

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1. Start jmemcache-daemon on Windows 2004, JDK 1.6.0_12 (-server -Xmx1600M)
2. Use spymemcached 1.2.8 and call memcachedClient.set(key) and then
memcachedClient.get(key).
3. spymemcached's client will return null

What is the expected output? What do you see instead?
After a while, jmemcache-daemon will start throwing what-appears-to-be the
same exception over and over again:

        at
org.jboss.netty.channel.AbstractChannel.write(AbstractChannel.java:236)
        at
com.thimbleware.jmemcached.protocol.text.MemcachedResponseEncoder.exceptionCaugh
t(MemcachedResponseEncoder.java:45)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChanne
lUpstreamHandler.java:129)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.exceptionCaught(SimpleChann
elUpstreamHandler.java:155)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChanne
lUpstreamHandler.java:129)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.exceptionCaught(SimpleChann
elUpstreamHandler.java:155)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChanne
lUpstreamHandler.java:129)
        at
org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.ja
va:206)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChanne
lUpstreamHandler.java:129)
        at
org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:607)
        at
org.jboss.netty.channel.socket.nio.NioWorker.cleanUpWriteBuffer(NioWorker.java:5
77)
        at
org.jboss.netty.channel.socket.nio.NioWorker.write(NioWorker.java:326)
        at
org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.handleAcceptedSoc
ket(NioServerSocketPipelineSink.java:140)
        at
org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.eventSunk(NioServ
erSocketPipelineSink.java:79)
        at org.jboss.netty.channel.Channels.write(Channels.java:898)
        at
org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEn
coder.java:76)
        at org.jboss.netty.channel.Channels.write(Channels.java:878)
        at org.jboss.netty.channel.Channels.write(Channels.java:826)
        at
org.jboss.netty.channel.AbstractChannel.write(AbstractChannel.java:236)
        at
com.thimbleware.jmemcached.protocol.text.MemcachedResponseEncoder.exceptionCaugh
t(MemcachedResponseEncoder.java:45)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChanne
lUpstreamHandler.java:129)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.exceptionCaught(SimpleChann
elUpstreamHandler.java:155)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChanne
lUpstreamHandler.java:129)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.exceptionCaught(SimpleChann
elUpstreamHandler.java:155)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChanne
lUpstreamHandler.java:129)
        at
org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.ja
va:206)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChanne
lUpstreamHandler.java:129)
        at
org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:607)
        at
org.jboss.netty.channel.socket.nio.NioWorker.cleanUpWriteBuffer(NioWorker.java:5
77)
        at
org.jboss.netty.channel.socket.nio.NioWorker.write(NioWorker.java:326)
        at
org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.handleAcceptedSoc
ket(NioServerSocketPipelineSink.java:140)
        at
org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.eventSunk(NioServ
erSocketPipelineSink.java:79)
        at org.jboss.netty.channel.Channels.write(Channels.java:898)
        at
org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEn
coder.java:76)
        at org.jboss.netty.channel.Channels.write(Channels.java:878)
        at org.jboss.netty.channel.Channels.write(Channels.java:826)
        at
org.jboss.netty.channel.AbstractChannel.write(AbstractChannel.java:236)
        at
com.thimbleware.jmemcached.protocol.text.MemcachedResponseEncoder.exceptionCaugh
t(MemcachedResponseEncoder.java:45)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChanne
lUpstreamHandler.java:129)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.exceptionCaught(SimpleChann
elUpstreamHandler.java:155)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChanne
lUpstreamHandler.java:129)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.exceptionCaught(SimpleChann
elUpstreamHandler.java:155)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChanne
lUpstreamHandler.java:129)
        at
org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.ja
va:206)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChanne
lUpstreamHandler.java:129)
        at
org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:607)
        at
org.jboss.netty.channel.socket.nio.NioWorker.cleanUpWriteBuffer(NioWorker.java:5
77)
        at
org.jboss.netty.channel.socket.nio.NioWorker.write(NioWorker.java:326)
        at
org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.handleAcceptedSoc
ket(NioServerSocketPipelineSink.java:140)
        at
org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.eventSunk(NioServ
erSocketPipelineSink.java:79)
        at org.jboss.netty.channel.Channels.write(Channels.java:898)
        at
org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEn
coder.java:76)

What version of the product are you using? On what operating system?
- spymemcached 1.2.8 on Windows XP with JDK 1.6.0_14
- jmemcache-daemon 0.7 on Windows 2003 with JDK 1.6.0_12

Please provide any additional information below.

I hope that's enough.  I can provide more information or do more testing if
necessary.

Original issue reported on code.google.com by kap4...@gmail.com on 23 Jun 2009 at 6:34

GoogleCodeExporter commented 9 years ago 
   The codes of the exceptionCaught methods of the "last" upstream handler doesn't
properly handle the client forcibly closing connection before getting the 
response.
   when something wrong happens during processing the downstream event,ie.
IOException, Netty generally will re-throw the exception to the last handler in 
the
upstream pipeline.
   I think it's better to check if the channel is still open, before write the exception.

Original comment by popd...@gmail.com on 24 Jun 2009 at 9:10

GoogleCodeExporter commented 9 years ago 
Thanks for pointing out the fault with the exceptionCaught; I noticed this last 
night, 
and am about to commit a fix for it in both protocols.

Unfortunately, I can't seem to reproduce the above issue with spymemcached 2.3. 
Is 
there something particular about 1.2.8 that requires you to use it?

Original comment by ryan.daum on 25 Jun 2009 at 12:34

GoogleCodeExporter commented 9 years ago 
Any client improperly closing the connection will cause this problem, I think.
Actually I'm using spymemcached 2.3.1, and I also met it. The attached file has 
some
detailed exception.

Original comment by popd...@gmail.com on 25 Jun 2009 at 7:25

Attachments:

GoogleCodeExporter commented 9 years ago 
I need to talk to some netty folks about the right way of handling this. If you 
checkout from the head of hg you'll 
see I added checks for open channel in the response encoders, but I still seem 
to run into occasional problems 
with being unable to write to the connection after it has been closed. Any 
ideas?

Original comment by ryan.daum on 25 Jun 2009 at 1:03

GoogleCodeExporter commented 9 years ago

Original comment by ryan.daum on 2 Jul 2009 at 2:02

GoogleCodeExporter commented 9 years ago 
ryan.daum:

I'll gladly test any fixes if/when you get around to this issue :)

Original comment by kap4...@gmail.com on 2 Jul 2009 at 3:32

GoogleCodeExporter commented 9 years ago 
I've actually committed a couple changes. Feel free to test with any examples 
you have.

If you have reproducible failures, I'd like to get them into the integration 
test.

Original comment by ryan.daum on 2 Jul 2009 at 3:34

GoogleCodeExporter commented 9 years ago 
What I've found is that when you use spymemcached to do that simple set-then-get
test, the get may return null because the set may have not completed yet. 
Remember;
"setting" data is asynchronous in spymemcached (which internally batches 
commands),
whereas "getting" data is synchronous.

If you use the code like the following then a set-then-get test should succeed 
every
time (it essentially makes the "set" synchronous):

boolean added = false;
Future<Boolean> addedFuture = client.set(key, expiryTimeInS, value);
try {
  added = addedFuture.get();
}
catch (Exception e) { // Something happened, "set" failed.
  added = false;
}
if (added) {
  client.get(key);
}

Original comment by daryl.be...@gmail.com on 18 Aug 2009 at 7:50

GoogleCodeExporter commented 9 years ago 
Interesting, I wasn't aware that all sets in spymemcached were asynchronous, I 
thought 
only the async series of ops were.

I've committed and pushed changes to the integration test to always wait on the 
future 
before proceeding.

Original comment by ryan.daum on 18 Aug 2009 at 8:18

GoogleCodeExporter commented 9 years ago

Original comment by ryan.daum on 18 Aug 2009 at 8:20