Deserialization fails on ConcurrentHashMap in Spring User object

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Try to serialize a spring User object that contains a TreeSet
2.
3.

What is the expected output? What do you see instead?

Correct serialization. Stack Trace
WARNING: Could not load session with id C85C5CD2388F9929DC99908ED6013146-n1 
from memcached.
com.esotericsoftware.kryo.SerializationException: Unable to deserialize object 
of type: java.util.concurrent.ConcurrentHashMap
    at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:593)
    at com.esotericsoftware.kryo.ObjectBuffer.readObject(ObjectBuffer.java:213)
    at de.javakaffee.web.msm.serializer.kryo.KryoTranscoder.deserializeAttributes(KryoTranscoder.java:256)
    at de.javakaffee.web.msm.TranscoderService.deserializeAttributes(TranscoderService.java:159)
    at de.javakaffee.web.msm.TranscoderService.deserialize(TranscoderService.java:116)
    at de.javakaffee.web.msm.MemcachedSessionService.loadFromMemcached(MemcachedSessionService.java:1027)
    at de.javakaffee.web.msm.MemcachedSessionService.findSession(MemcachedSessionService.java:578)
    at de.javakaffee.web.msm.MemcachedBackupSessionManager.findSession(MemcachedBackupSessionManager.java:196)
    at org.apache.catalina.connector.Request.doGetSession(Request.java:2841)
    at org.apache.catalina.connector.Request.getSession(Request.java:2307)
    at org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:897)
    at javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:229)
    at org.springframework.security.web.context.HttpSessionSecurityContextRepository$SaveToSessionResponseWrapper.createNewSessionIfAllowed(HttpSessionSecurityContextRepository.java:339)
    at org.springframework.security.web.context.HttpSessionSecurityContextRepository$SaveToSessionResponseWrapper.saveContext(HttpSessionSecurityContextRepository.java:280)
    at org.springframework.security.web.context.HttpSessionSecurityContextRepository.saveContext(HttpSessionSecurityContextRepository.java:104)
    at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:87)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
    at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
    at com.redbox.digital.proxy.security.WSFedRememberMeAuthenticationFilter.doFilter(WSFedRememberMeAuthenticationFilter.java:104)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
    at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
    at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
    at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:201)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
    at com.redbox.digital.proxy.security.WSFedRequestParamPreAuthenticationFilter.doFilter(WSFedRequestParamPreAuthenticationFilter.java:195)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
    at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
    at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
    at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:173)
    at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
    at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:224)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169)
    at de.javakaffee.web.msm.SessionTrackerValve.invoke(SessionTrackerValve.java:126)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:581)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:927)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:987)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:579)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:309)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)
Caused by: com.esotericsoftware.kryo.SerializationException: Unable to 
deserialize object of type: 
org.springframework.security.core.context.SecurityContextImpl
    at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:571)
    at com.esotericsoftware.kryo.serialize.MapSerializer.readObjectData(MapSerializer.java:129)
    at com.esotericsoftware.kryo.Serializer.readObject(Serializer.java:61)
    at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:589)
    ... 52 more
Caused by: com.esotericsoftware.kryo.SerializationException: Serialization 
trace:
authorities (org.springframework.security.core.userdetails.User)
principal 
(org.springframework.security.authentication.UsernamePasswordAuthenticationToken
)
authentication (org.springframework.security.core.context.SecurityContextImpl)
    at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:238)
    at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.readObjectData(ReferenceFieldSerializer.java:81)
    at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:220)
    at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.readObjectData(ReferenceFieldSerializer.java:81)
    at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:220)
    at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.readObjectData(ReferenceFieldSerializer.java:81)
    at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:566)
    ... 55 more
Caused by: java.lang.ClassCastException: 
org.springframework.security.core.authority.SimpleGrantedAuthority cannot be 
cast to java.lang.Comparable
    at java.util.TreeMap.compare(TreeMap.java:1188)
    at java.util.TreeMap.put(TreeMap.java:531)
    at java.util.TreeSet.add(TreeSet.java:255)
    at com.esotericsoftware.kryo.serialize.CollectionSerializer.readObjectData(CollectionSerializer.java:113)
    at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:566)
    at de.javakaffee.kryoserializers.UnmodifiableCollectionsSerializer.readObjectData(UnmodifiableCollectionsSerializer.java:84)
    at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:220)
    ... 61 more

What version of the product are you using? On what operating system?
1.6.1. Linux and Windows

Please provide any additional information below.

Can I upgrade Kyro and will this still work?

Original issue reported on code.google.com by monster...@gmail.com on 20 Jun 2012 at 6:51

• Blocked on: #189

[GoogleCodeExporter]

AFAICS the issue is that the spring User class uses a TreeSet initialized with 
a comparator (in User.sortAuthorities) and that kryo's CollectionSerializer 
creates just a new TreeSet (without comparator). This issue with the TreeSet 
might be interesting for the kryo project, although it probably won't be fixed 
for kryo1 as kryo2 is just hot and fresh (do you want to push it there?).

A solution for would be to write a custom serializer for the spring User object 
an register it as customConverter (msm config). You can check existing custom 
converters + kryo-serializers to see how to do this, e.g.:

https://github.com/magro/memcached-session-manager/blob/master/kryo-serializer/s
rc/main/java/de/javakaffee/web/msm/serializer/kryo/JodaDateTimeRegistration.java

and

https://github.com/magro/kryo-serializers/blob/master/src/main/java/de/javakaffe
e/kryoserializers/jodatime/JodaDateTimeSerializer.java

If you need any help you may ask on the mailing list.

This serializer might also be a nice contribution to kryo-serializers / msm :-)

Original comment by martin.grotzke on 21 Jun 2012 at 1:25

[GoogleCodeExporter]

Ill see if I can get to it :)

Worked around it for now with a new spring user object.

Kevin Clark (monster910)

On Wed, Jun 20, 2012 at 8:25 PM,
<memcached-session-manager@googlecode.com>wrote:

Original comment by monster...@gmail.com on 27 Jun 2012 at 2:40

[GoogleCodeExporter]

Hi Kevin/Martin,

How have you done the workaround?
If I will use other serializer will it work?

Thanks,
Netanel

Original comment by netane...@gmail.com on 28 Aug 2013 at 1:33

[GoogleCodeExporter]

Hi Netanel,

the custom serializer for the spring User class has not yet found its way into 
kryo-serializers, so you could contribute it :-)

Do you still have questions regarding the custom serializer?

I'm not sure if the problem also exists with the default (java) serializer, you 
could try it (just remove the transcoderFactoryClass attribute).

Cheers,
Martin

Original comment by martin.grotzke on 28 Aug 2013 at 3:37

[GoogleCodeExporter]

I hacked a customConverter for the Spring Security User class: 
https://gist.github.com/magro/6370466

This can be added to the msm config like this:

...
customConverter="de.javakaffee.web.msm.serializer.kryo.SpringSecurityUserRegistr
ation"
transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFact
ory"
...

Use it on your own risk, I haven't tested it! Feedback welcome.

Original comment by martin.grotzke on 28 Aug 2013 at 8:00

[GoogleCodeExporter]

Hello Martin,

Thank you for writing this class. We currently are having exactly the same 
problem with ConcurrentHashMap and Tomcat MSM. We tried using your class, but 
still are getting the same Error. We have created a jar file of this 
SpringSecurityUserRegistration class, added it to Tomcat and registered the 
class in the manager as shown in your post. But unfortunately, nothing has 
changed. Is there anything else that needs attention for? We also already tried 
to register just the ConcurrentHashMap with the following class:

package tv.mashero.customkryo;
import java.util.concurrent.ConcurrentHashMap;
import com.esotericsoftware.kryo.Kryo;
import com.esotericsoftware.kryo.serialize.MapSerializer;
import de.javakaffee.web.msm.serializer.kryo.KryoCustomization;

public class CustomKryoRegistration implements KryoCustomization {

    public void customize(Kryo kryo) {
        kryo.register( ConcurrentHashMap.class, new MapSerializer(kryo) );  
    }

}

This example does not work, because a NoSuchMethodError for Kryo.register gets 
thrown:

SEVERE: Could not execute customization 
customkryo.CustomKryoRegistration@1d91eb90
java.lang.NoSuchMethodError: 
com.esotericsoftware.kryo.Kryo.register(Ljava/lang/Class;Lcom/esotericsoftware/k
ryo/Serializer;)Lcom/esotericsoftware/kryo/Registration;
    at tv.mashero.customkryo.CustomKryoRegistration.customize(CustomKryoRegistration.java:14)
    at de.javakaffee.web.msm.serializer.kryo.KryoTranscoder.createKryo(KryoTranscoder.java:214)
    at de.javakaffee.web.msm.serializer.kryo.KryoTranscoder.<init>(KryoTranscoder.java:117)
    at de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory.getTranscoder(KryoTranscoderFactory.java:61)
    at de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory.createTranscoder(KryoTranscoderFactory.java:47)
    at de.javakaffee.web.msm.MemcachedSessionService.createTranscoderService(MemcachedSessionService.java:474)
    at de.javakaffee.web.msm.MemcachedSessionService.startInternal(MemcachedSessionService.java:450)
    at de.javakaffee.web.msm.MemcachedBackupSessionManager.startInternal(MemcachedBackupSessionManager.java:128)
    at de.javakaffee.web.msm.MemcachedBackupSessionManager.start(MemcachedBackupSessionManager.java:637)

Has this to do with the old version of Kryo that MSM is using (currently only 
version 1)? Why MSM is still using the old Kryo 1 version and does not update 
Kryo to version 2? Do you have any ideas how to solve the problem?

Original comment by steffen...@googlemail.com on 29 Aug 2013 at 12:54

[GoogleCodeExporter]

Have you built your CustomKryoRegistration with kryo1? If you have 
msm-kryo-serializer in your project/webapp you should have the correct kryo 
version (as transitive dependency if you're using maven) in place already. You 
can keep the CustomKryoRegistration besides your regular app classes - it must 
be loaded by the webapp classloader as it's the case for msm-kryo-serializer + 
kryo.

So msm-kryo-serializer + kryo etc. must be in WEB-INF/lib, your 
CustomKryoRegistration.class should be in WEB-INF/classes.

Is this the case?

Regarding kryo1 vs. kryo2: the kryo-serializers providing additional 
serializers is already ported to kryo2, but the tests for CGLibSerializer (or 
JdkProxySerializer, not sure) failed and I couldn't figure out what's the 
reason for this. This should be fixed, and then msm-kryo-serializer can be 
ported to kryo2.

Original comment by martin.grotzke on 29 Aug 2013 at 4:16

[GoogleCodeExporter]

It has been awhile but I create a new user object and rewrote the methods 
failing.

/**
 * XXXX user details implementation. Pseudo model of the IIA Party-ContactPoint model.
 * 
 * @author Kevin Clark
 *
 */

@JsonIgnoreProperties({"authorities", "username", "password", 
"accountNonExpired", "accountNonLocked", "credentialsNonExpired", "enabled"})
public class ZoeUser implements UserDetails, CredentialsContainer {

I rewrote the sortAuthorities method to not use TreeSet

    private static Set<GrantedAuthority> sortAuthorities(Collection<? extends GrantedAuthority> authorities) {
        Assert.notNull(authorities, "Cannot pass a null GrantedAuthority collection");
        // Ensure array iteration order is predictable (as per UserDetails.getAuthorities() contract and SEC-717)
        List<GrantedAuthority> sortedAuthorities =  new ArrayList<GrantedAuthority>(authorities);

        for (GrantedAuthority grantedAuthority : authorities) {
            Assert.notNull(grantedAuthority, "GrantedAuthority list cannot contain any null elements");
            sortedAuthorities.add(grantedAuthority);
        }
        Collections.sort(sortedAuthorities, new AuthorityComparator());

        return new HashSet<GrantedAuthority>(sortedAuthorities);
    }

Original comment by monster...@gmail.com on 29 Aug 2013 at 6:04

[GoogleCodeExporter]

Hello Martin,

The classes are not laying inside the WEB-INF/classes folder, because Tomcat 
needs to use it in a global context. On our Tomcat several Applications are 
running, not only the one using Spring Security. Thats why we created a jar 
file for the class and added it to Tomcat, not to the Application. If we use 
the class in the Application itself and register it in Tomcats context.xml, 
then there is the problem Tomcat can not start anymore, because then Tomcat can 
not find the class.

The NoSuchMethodError for the CustomKryoRegistration class occured propably 
because we were using a Kryo 2 Version, so it wasnt compatible. We tried it 
again now using Kryo 1, created a jar file of this CustomKryoRegistration 
class, added it to Tomcat and registered the class within the customConverter 
attribute in the Tomcats context.xml. This is our context.xml we do use:

<Context>
    <WatchedResource>WEB-INF/web.xml</WatchedResource>
    <Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
      memcachedNodes="devapp0:10.10.1.14:11211 devapp1:10.10.1.34:11211"
      sticky="false"
      lockingMode="auto"
      requestUriIgnorePattern=".*\.(png|gif|jpg|css|js)$"
      customConverter="customkryo.CustomKryoRegistration"
      transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory"
    />
</Context>

The class CustomKryoRegistration is, how i sayd, packed into a jar file named 
CustomKryo.jar that is added to the Tomcat. The NoSuchMethodError is gone, but 
the main problem continues, the Error "Unable to deserialize object of type: 
java.util.concurrent.ConcurrentHashMap" still occures. Somehow it looks like 
the class maybe just dont get used. Is there anything we are doing wrong or 
something else we have to pay attention for?

Original comment by steffen...@googlemail.com on 30 Aug 2013 at 12:53

[GoogleCodeExporter]

@monster910: as you're using a HashMap you can even remove all the sorting 
stuff. Alternatively use a LinkedHashMap to retain sort order.

Original comment by martin.grotzke on 30 Aug 2013 at 11:01

[GoogleCodeExporter]

@steffen.sz I just noticed that your CustomKryoRegistration covers 
ConcurrentHashMap, that doesn't change anything. The ConcurrentHashMap is not 
the problem.
You need to configure the serializer/registration I provided. If it does not 
work please post the actually used configuration, logs from tomcat start with 
debug logging enabled and the full stacktrace. 

Original comment by martin.grotzke on 30 Aug 2013 at 11:11

[GoogleCodeExporter]

Hello Martin,

Thank you very much for your help. We used the serializer class 
SpringSecurityUserRegistration you provided. Only i had to remove the @Override 
on the customize function since this created an compile Error in Java:
 - The method customize(Kryo) of type SpringSecurityUserRegistration must override a superclass method
 - implements de.javakaffee.web.msm.serializer.kryo.KryoCustomization.customize
I guess it wasnt really your intention to override the customize function there?

The class SpringSecurityUserRegistration is packaged to a jar file named 
"KryoUserSerializer.jar", and is added to the Tomcat together with all other 
jar files. But without any success. Since the logs and stacktrace etc. are very 
large, i made a Text file and added it as attachement to this post. It should 
contain all information. Thanks again very much for your help.

Original comment by steffen...@googlemail.com on 2 Sep 2013 at 1:00

Attachments:

• LogsAndConfiguration.txt

[GoogleCodeExporter]

Re @Override annotation: sounds as if you're building with java5, with java6 
the @Override is valid for overridden methods from an interface.

From the attached logs it seems as if the SpringSecurityUserRegistration is not 
used to deserialized the user object. Did you start with a clean memcached, so 
that it's sure that msm does not try to deserialize a session serialized 
without the registered SpringSecurityUserRegistration?
If that was the case can you provide a very simple sample app that allows me to 
reproduce/debug this issue?

Original comment by martin.grotzke on 3 Sep 2013 at 8:57

[GoogleCodeExporter]

Hi Martin,

We also cleaned the memcached, but its the same result. I will try to get an 
example app during the next days.

Original comment by steffen...@googlemail.com on 4 Sep 2013 at 7:43

[GoogleCodeExporter]

Ok

Original comment by martin.grotzke on 4 Sep 2013 at 7:50

[GoogleCodeExporter]

Hello Martin,

Sorry it toke a while, but finaly we worked out a good solution. We figured out 
byself where the error was coming from. When using the 
SpringSecurityUserRegistration class you provided, it introduces direct and 
indicect dependencies to the spring-security packages and others, which must be 
included within the tomcat server lib. The error still occured because some 
neccessary packages were missing there. However, moving all the jar from the 
war archive to the server-library seems to be an ugly side effect, because it 
breaks the servlet-container/application architecure to separate server and 
application code /libraries.

Since this could be problematic for us and may cause other problems, we gave up 
on using the SpringSecurityUserRegistration class and just fixed the bug in the 
kryo1 version itself, which is been the main source of the problem at all. We 
updated the kryo1 package, creating a new version 1.04.1 of kryo. With this, 
the serializing bug is fixed now, the error is not occuring anymore.

We added the fixed kryo1 version as attachment, so if possible, you can make a 
test with it, checking whether it is working still correct when using it 
together with MSM. The attachment includes the final kryo-1.04.1.jar and its 
sources as complete project that can be build with maven. 

Also it would be nice to put the patch into the kryo repository, to help others 
who are having the same issue. What you think about it?

Original comment by steffen...@googlemail.com on 8 Oct 2013 at 9:15

Attachments:

• kryotest-package.zip

[GoogleCodeExporter]

Great analysis! Can you submit an issue with a patch in the kryo issue tracker?
Regarding the fix: In Kryo.newSerializer I'd use
  TreeMap.class.isAssignableFrom(type)
instead of
  type.getName().equals(TreeMap.class.getName()).

Is there a reason why you chose the latter way?

Original comment by martin.grotzke on 9 Oct 2013 at 7:31

[GoogleCodeExporter]

Btw, I've added TreeMap/SetSerializer for kryo1 (and TreeSetSerializer to 
kryo2, because kryo2 already had a TreeMapSerializer), and will cut a new kryo1 
release soon. I'll drop a not here once this is available.

Here's the change in kryo 1: http://code.google.com/p/kryo/source/detail?r=419

Original comment by martin.grotzke on 11 Oct 2013 at 12:36

• Changed state: Started

[GoogleCodeExporter]

Hello Martin,

Thank you for adding the update to kryo. Regarding to the fix: using 
type.getName().equals(TreeMap.class.getName() avoids serializing subclasses for 
which specific serializers might be requird. 

When using the method isAssignableFrom(mytype) for a derived TreeMap class with 
additional fields, serializing the additional fields would not happen, and no 
error would be produced. The resulting error would be quite similar to the 1.04 
implementation, which made a cast of the TreeMap to a collection and missed to 
serialize the comparator field.

Do you think this could be a problem?

Original comment by steffen...@googlemail.com on 14 Oct 2013 at 10:13

[GoogleCodeExporter]

Original comment by martin.grotzke on 20 Dec 2013 at 10:55

• Now blocked on: #189