search

yingmingbao / memcached-session-manager

Automatically exported from code.google.com/p/memcached-session-manager
0 stars 0 forks source link

kryo-users › com.esotericsoftware.kryo.SerializationException: Buffer limit exceeded writing object of type: #199

Open GoogleCodeExporter opened 8 years ago

GoogleCodeExporter commented 8 years ago 
What steps will reproduce the problem?
1. msm + kryo + legacy spring mvc application
2. Override the default max buffer size to 10 
MB(-Dmsm.kryo.buffersize.max=10240000)
3. <Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
            sticky="false"
            memcachedNodes="n1:host1:11211,n2:host2:11211"
            failoverNodes="n1"
            lockingMode="all"
            requestUriIgnorePattern=".*\.(png|gif|jpg|css|js|ico)$"
            sessionBackupAsync="false"
            sessionBackupTimeout="100"
            copyCollectionsForSerialization="true"
            transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory"
    />

What is the expected output? What do you see instead?
Application should work as it did work currently with HttpSession to store 
session info. 
But it's failing due to Buffer limit exceeded writing object of type:

What version of the product are you using? On what operating system?
spymemcached-2.10.2.jar
memcached-session-manager-tc6-1.8.1.jar
memcached-session-manager-1.8.1.jar
reflectasm-1.01.jar
msm-kryo-serializer-1.8.1.jar
minlog-1.2.jar
kryo-serializers-0.11.jar
kryo-1.04.jar
asm-3.3.1.jar

Please provide any additional information below.
We are integrating memcached session manager (msm) + kryo with our legacy 
applications, as part of that testing getting following error. I don't have 
permission to change any code in the legacy applications. So I have to 
transparently migrate an application from HttpSession  to Memcached.

Feb 28, 2014 5:26:36 PM de.javakaffee.web.msm.BackupSessionService backupSession
INFO: Could not store session BA46520D8EA570DEC2A6674340FF22C7-n2.tomcat1 in 
memcached.
java.util.concurrent.ExecutionException: 
com.esotericsoftware.kryo.SerializationException: Buffer limit exceeded 
serializing object of type: java.util.concurrent.ConcurrentHashMap
    at de.javakaffee.web.msm.BackupSessionService$SynchronousExecutorService.submit(BackupSessionService.java:348)
    at de.javakaffee.web.msm.BackupSessionService.backupSession(BackupSessionService.java:205)
    at de.javakaffee.web.msm.MemcachedSessionService.backupSession(MemcachedSessionService.java:1040)
    at de.javakaffee.web.msm.RequestTrackingHostValve.backupSession(RequestTrackingHostValve.java:230)
    at de.javakaffee.web.msm.RequestTrackingHostValve.invoke(RequestTrackingHostValve.java:159)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
    at com.etrade.neo.tomcat.accessvalve.NEOAccessLogValve.invoke(NEOAccessLogValve.java:572)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
    at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190)
    at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291)
    at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:769)
    at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:698)
    at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:891)
    at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
    at java.lang.Thread.run(Thread.java:662)
Caused by: com.esotericsoftware.kryo.SerializationException: Buffer limit 
exceeded serializing object of type: java.util.concurrent.ConcurrentHashMap
    at com.esotericsoftware.kryo.ObjectBuffer.writeObject(ObjectBuffer.java:256)
    at de.javakaffee.web.msm.serializer.kryo.KryoTranscoder.serializeAttributes(KryoTranscoder.java:303)
    at de.javakaffee.web.msm.TranscoderService.serializeAttributes(TranscoderService.java:155)
    at de.javakaffee.web.msm.BackupSessionTask.serializeAttributes(BackupSessionTask.java:176)
    at de.javakaffee.web.msm.BackupSessionTask.call(BackupSessionTask.java:110)
    at de.javakaffee.web.msm.BackupSessionTask.call(BackupSessionTask.java:51)
    at de.javakaffee.web.msm.BackupSessionService$SynchronousExecutorService.submit(BackupSessionService.java:346)
    ... 15 more
Caused by: com.esotericsoftware.kryo.SerializationException: Buffer limit 
exceeded writing object of type: java.util.concurrent.ConcurrentHashMap
    at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:523)
    at com.esotericsoftware.kryo.ObjectBuffer.writeObject(ObjectBuffer.java:251)
    ... 21 more
Caused by: com.esotericsoftware.kryo.SerializationException: Buffer limit 
exceeded writing object of type: 
org.springframework.web.context.request.DestructionCallbackBindingListener
    at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:492)
    at com.esotericsoftware.kryo.serialize.MapSerializer.writeObjectData(MapSerializer.java:104)
    at de.javakaffee.kryoserializers.CopyForIterateMapSerializer.writeObjectData(CopyForIterateMapSerializer.java:56)
    at com.esotericsoftware.kryo.Serializer.writeObject(Serializer.java:43)
    at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:520)
    ... 22 more
Caused by: com.esotericsoftware.kryo.SerializationException: Buffer limit 
exceeded writing object of type: 
org.apache.cxf.bus.spring.Jsr250BeanPostProcessor
Serialization trace:
beanPostProcessors 
(org.springframework.beans.factory.support.DisposableBeanAdapter)
destructionCallback 
(org.springframework.web.context.request.DestructionCallbackBindingListener)
    at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:492)
    at com.esotericsoftware.kryo.serialize.CollectionSerializer.writeObjectData(CollectionSerializer.java:86)
    at de.javakaffee.kryoserializers.CopyForIterateCollectionSerializer.writeObjectData(CopyForIterateCollectionSerializer.java:46)
    at com.esotericsoftware.kryo.serialize.FieldSerializer.writeObjectData(FieldSerializer.java:175)
    at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.writeObjectData(ReferenceFieldSerializer.java:52)
    at com.esotericsoftware.kryo.serialize.FieldSerializer.writeObjectData(FieldSerializer.java:175)
    at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.writeObjectData(ReferenceFieldSerializer.java:52)
    at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:489)
    ... 26 more
Caused by: com.esotericsoftware.kryo.SerializationException: Buffer limit 
exceeded writing object of type: java.lang.Package
Serialization trace:
packages (org.apache.catalina.loader.WebappClassLoader)
beanClassLoader 
(org.springframework.beans.factory.support.DefaultListableBeanFactory)
beanFactory 
(org.springframework.context.event.SimpleApplicationEventMulticaster)
applicationEventMulticaster 
(com.etrade.neo.spring.context.NeoPISXmlWebApplicationContext)
context (org.apache.cxf.bus.spring.Jsr250BeanPostProcessor)
    at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:492)
    at com.esotericsoftware.kryo.serialize.MapSerializer.writeObjectData(MapSerializer.java:104)
    at de.javakaffee.kryoserializers.CopyForIterateMapSerializer.writeObjectData(CopyForIterateMapSerializer.java:56)
    at com.esotericsoftware.kryo.serialize.FieldSerializer.writeObjectData(FieldSerializer.java:175)
    at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.writeObjectData(ReferenceFieldSerializer.java:52)
    at com.esotericsoftware.kryo.serialize.FieldSerializer.writeObjectData(FieldSerializer.java:175)
    at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.writeObjectData(ReferenceFieldSerializer.java:52)
    at com.esotericsoftware.kryo.serialize.FieldSerializer.writeObjectData(FieldSerializer.java:175)
    at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.writeObjectData(ReferenceFieldSerializer.java:52)
    at com.esotericsoftware.kryo.serialize.FieldSerializer.writeObjectData(FieldSerializer.java:175)
    at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.writeObjectData(ReferenceFieldSerializer.java:52)
    at com.esotericsoftware.kryo.serialize.FieldSerializer.writeObjectData(FieldSerializer.java:175)
    at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.writeObjectData(ReferenceFieldSerializer.java:52)
    at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:489)
    ... 33 more
Caused by: com.esotericsoftware.kryo.SerializationException: Serialization 
trace:
pkgName (java.lang.Package)
    at com.esotericsoftware.kryo.serialize.FieldSerializer.writeObjectData(FieldSerializer.java:191)
    at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.writeObjectData(ReferenceFieldSerializer.java:52)
    at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:489)
    ... 46 more
Caused by: java.nio.BufferOverflowException
    at java.nio.Buffer.nextPutIndex(Buffer.java:493)
    at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:145)
    at com.esotericsoftware.kryo.serialize.StringSerializer.put(StringSerializer.java:35)
    at com.esotericsoftware.kryo.serialize.StringSerializer.writeObjectData(StringSerializer.java:24)
    at com.esotericsoftware.kryo.Serializer.writeObject(Serializer.java:43)
    at com.esotericsoftware.kryo.serialize.FieldSerializer.writeObjectData(FieldSerializer.java:182)
    ... 48 more

Some useful information, prior to this error I was getting lot of 
ClassNotFoundException, resolved those by placing the jars under 
tomcat6/common/lib. This might not be the good solution, just a work around and 
proceed further I did this workaround. Please suggest if any other solutions to 
overcome this issue.

Caused by: java.lang.ClassNotFoundException: 
org.eclipse.jetty.util.component.Container$Listener
Caused by: java.lang.ClassNotFoundException: 
org.zeroturnaround.javarebel.ClassEventListener
Caused by: java.lang.ClassNotFoundException: org.eclipse.jetty.http.Generator
Caused by: java.lang.ClassNotFoundException: org.python.core.PyException
Caused by: java.lang.ClassNotFoundException: org.jvnet.staxex.XMLStreamWriterEx
Caused by: java.lang.ClassNotFoundException: org.eclipse.jetty.server.Connector

Thanks
Gowda

Original issue reported on code.google.com by yoga.und...@gmail.com on 3 Mar 2014 at 10:55

GoogleCodeExporter commented 8 years ago 
OS : Linux x86_64 x86_64 x86_64 GNU/Linux

Original comment by yoga.und...@gmail.com on 3 Mar 2014 at 11:01

GoogleCodeExporter commented 8 years ago 
I'd say you're serializing the whole spring ApplicationContext. Or might there 
be another reason for such big sessions?

Original comment by martin.grotzke on 6 Mar 2014 at 12:45

GoogleCodeExporter commented 8 years ago 
The same application + Coherence Web Session+ Kryo and same application + 
Hazelcast Wen Session + Kryo doesn't have this issue. Also we measured the max 
size of the session in those two web sessions framework, max session size is 30 
KB.

Original comment by yoga.und...@gmail.com on 6 Mar 2014 at 4:42

GoogleCodeExporter commented 8 years ago 
msm +  Java Serialization + Spring MVC, going good, but I see one or two issue 
while serializing SPRING SECURITY CONTEXT. Looking into it now. It seems like 
one of our custom authentication class not implemented Serializable, causing 
this issue, let me updated once I'm done with testing.

Cannot serialize session attribute SPRING_SECURITY_CONTEXT for session

Thanks
Yoga

Original comment by yoga.und...@gmail.com on 27 Mar 2014 at 6:32

GoogleCodeExporter commented 8 years ago 
For this one, can you provide a simple spring app that allows to reproduce the 
issue that I could use to debug this?

Original comment by martin.grotzke on 27 Mar 2014 at 8:12

GoogleCodeExporter commented 8 years ago 
Hi Martin,

Please find attached  Spring Mvc , where I'm still getting lot below exception, 
when I remove implements Serializable from User class. This is same happening 
in our legacy spring mvc based application. I'm adding Serializable interface 
to classes which are failing one by one, but still the list keep on growing.

Mar 28, 2014 5:32:56 PM de.javakaffee.web.msm.JavaSerializationTranscoder 
writeAttributes
WARNING: Cannot serialize session attribute 
org.springframework.web.context.request.ServletRequestAttributes.DESTRUCTION_CAL
LBACK.scopedTarget.userProfile for session 
F26FF1B9DFDFE7C3EB1110F11449A5F3-n2.tomcat1
java.io.NotSerializableException: com.dev.spring.context.JaxWsClientProxy
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1164)
    at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1518)
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1483)
    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1400)
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1158)
    at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1518)
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1483)
    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1400)
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1158)
    at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1518)
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1483)
    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1400)

Original comment by yoga.und...@gmail.com on 31 Mar 2014 at 4:33

Attachments:

GoogleCodeExporter commented 8 years ago 
Hi Martin,

Any suggestion/direction to avoid the above exception gracefully, because our 
legacy application has tons of classes without Serilizable interface 
implementation.

Thanks
Yoga

Original comment by yoga.und...@gmail.com on 31 Mar 2014 at 4:46