search

seanjensengrey / jmemcache-daemon

Automatically exported from code.google.com/p/jmemcache-daemon
Apache License 2.0
0 stars 0 forks source link

Error storing binary data in jmemcached using binary protocol #3

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
Please, follow these steps to reproduce the error:

0. Unzip the attached file.
1. Run the server: java -jar ./3rdparty/jmemcached-cli-0.8-main.jar -l
<host> -p <port> -v
2. Run the first test: java -Xms100m -Xmx1000m -classpath
"./bin;./3rdparty/memcached-2.3.1.jar" org.mcpolu. issue3 .test1
<host> <port>

I get the following trace:

2009-06-24 22:28:01.737 INFO net.spy.memcached.MemcachedConnection:  Added
{QA s
a=localhost/127.0.0.1:11311, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null,
toWrite=0, interested=0} to connect queue
2009-06-24 22:28:01.752 INFO net.spy.memcached.MemcachedConnection:  Connection
state changed for sun.nio.ch.SelectionKeyImpl@95c083
2009-06-24 22:28:02.735 INFO
net.spy.memcached.transcoders.SerializingTranscoder
:  Compressed java.util.HashMap from 2879038 to 739793
Exception in thread "Memcached IO over {MemcachedConnection to
localhost/127.0.0
.1:11311}" java.lang.OutOfMemoryError: Java heap space
        at
net.spy.memcached.protocol.binary.OperationImpl.readFromBuffer(Operat
ionImpl.java:112)
        at
net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection
.java:361)
        at
net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.ja
va:305)
        at
net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.ja
va:192)
        at net.spy.memcached.MemcachedClient.run(MemcachedClient.java:1444)
Exception in thread "main" net.spy.memcached.OperationTimeoutException: Timeout
waiting for value
        at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:840)
        at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:855)
        at org.mcpolu.issue3.test1.main(test1.java:25)
Caused by: java.util.concurrent.TimeoutException: Timed out waiting for
operatio
n
        at
net.spy.memcached.MemcachedClient$OperationFuture.get(MemcachedClient
.java:1656)
        at
net.spy.memcached.MemcachedClient$GetFuture.get(MemcachedClient.java:
1708)
        at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:833)
        ... 2 more

The server throws the same exception over and over again, just as described
in Issue1. This is the trace I get:

        at
com.thimbleware.jmemcached.protocol.text.MemcachedResponseEncoder.exc
eptionCaught(MemcachedResponseEncoder.java:45)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(S
impleChannelUpstreamHandler.java:129)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.exceptionCaught(
SimpleChannelUpstreamHandler.java:155)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(S
impleChannelUpstreamHandler.java:129)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.exceptionCaught(
SimpleChannelUpstreamHandler.java:155)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(S
impleChannelUpstreamHandler.java:129)
        at
org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(Fram
eDecoder.java:206)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(S
impleChannelUpstreamHandler.java:129)
        at
org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:60
7)
        at
org.jboss.netty.channel.socket.nio.NioWorker.cleanUpWriteBuffer(NioWo
rker.java:577)
        at
org.jboss.netty.channel.socket.nio.NioWorker.write(NioWorker.java:326
)
        at
org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.handle
AcceptedSocket(NioServerSocketPipelineSink.java:140)
        at
org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.eventS
unk(NioServerSocketPipelineSink.java:79)
        at org.jboss.netty.channel.Channels.write(Channels.java:898)
        at
org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream
(OneToOneEncoder.java:76)
        at org.jboss.netty.channel.Channels.write(Channels.java:878)
        at org.jboss.netty.channel.Channels.write(Channels.java:826)
        at
org.jboss.netty.channel.AbstractChannel.write(AbstractChannel.java:23
6)
        at
com.thimbleware.jmemcached.protocol.text.MemcachedResponseEncoder.exc
eptionCaught(MemcachedResponseEncoder.java:45)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(S
impleChannelUpstreamHandler.java:129)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.exceptionCaught(
SimpleChannelUpstreamHandler.java:155)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(S
impleChannelUpstreamHandler.java:129)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.exceptionCaught(
SimpleChannelUpstreamHandler.java:155)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(S
impleChannelUpstreamHandler.java:129)
        at
org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(Fram
eDecoder.java:206)
        at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(S
impleChannelUpstreamHandler.java:129)

3. Run the second test: java -Xms100m -Xmx1000m -classpath
"./bin;./3rdparty/memcached-2.3.1.jar;./3rdparty/mail.jar" org.mcpolu.
issue3.test2 <host> <port>

I get the following trace:

2009-06-24 22:33:41.433 INFO net.spy.memcached.MemcachedConnection:  Added
{QA s
a=localhost/127.0.0.1:11311, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null,
toWrite=0, interested=0} to connect queue
2009-06-24 22:33:41.457 INFO net.spy.memcached.MemcachedConnection:  Connection
state changed for sun.nio.ch.SelectionKeyImpl@191d8c1
2009-06-24 22:33:42.492 INFO org.mcpolu.issue3.Base64Transcoder: 
Compressed jav
a.util.HashMap from 2879038 to 986392
2009-06-24 22:33:43.023 INFO net.spy.memcached.MemcachedClient:  Shut down
memca
ched client

In the first test, a big Java Object is serialized, then compressed
and sent to the server. When retrieved, the byte stream appears to be
infinite, and the client runs out of memory. It could be related to Issue2.
In the second test, the same big Java Object is serialized, compressed
and finally encoded using Base64. When retrieved, I can successfully
Base64 decode it, then uncompress it and finally recreate the Object.

This appears to be the same problem that was present in jmemcached v0.7
(see http://code.google.com/p/spymemcached/issues/detail?id=73 for
details). In jmemcached v0.8 it works over the text protocol. However, it
fails if the binary protocol is used.

Original issue reported on code.google.com by McP...@gmail.com on 24 Jun 2009 at 9:39

Attachments:

GoogleCodeExporter commented 9 years ago 
I don't seem to be able to reproduce this. Not on my local branch anyways...

The following test passes for me:

  @Test
    public void testBigBinaryObject() {
        Object bigObject = getBigObject();
        _client.set(KEY, TWO_WEEKS, bigObject);
        final Map<String, Double> map = (Map<String, Double>)_client.get(KEY);
        for (String key : map.keySet()) {
            Integer kint = Integer.valueOf(key);
            Double val = map.get(key);
            assertEquals(val, kint/42.0, 0.0);
        }
    }

Original comment by ryan.daum on 25 Jun 2009 at 1:19

GoogleCodeExporter commented 9 years ago

Original comment by ryan.daum on 2 Jul 2009 at 2:01

GoogleCodeExporter commented 9 years ago 
Hi, sorry for the delay.

I tried to reproduce the error today and I couldn't, but it doesn't work 
properly
either, now I'm getting a time out error every time I try to use the binary
protocol... however, the same code using the text protocol works perfectly. I 
guess
I'm doing something wrong, but I cannot find it :-(

I'm sorry I cannot help you reproduce it :-(

Original comment by McP...@gmail.com on 4 Jul 2009 at 9:08