search

RuedigerMoeller / fast-serialization

FST: fast java serialization drop in-replacement
Apache License 2.0
1.59k stars 247 forks source link

Java 8 support #7

Closed apotapov closed 10 years ago

apotapov commented 10 years ago

Hello,

First off, much appreciate your efforts on this awesome library.

We are currently using fast-serialization indirectly through https://github.com/debop/hibernate-redis. And we recently migrated our project to Java 8 to take advantage of all that lambda function goodness. Currently FST doesn't seem to support Java 8. I'm seeing the attached stack trace when trying to deserialize an enum that is part of a hibernate bean.

Any plans to support Java 8 in the near future?

Much thanks.

22:48:59.295 [application-akka.actor.default-dispatcher-9] WARN o.h.c.r.s.FstRedisSerializer.deserialize 53 - Fail to deserialize bytes. java.io.IOException: java.lang.RuntimeException: CLASSNAME:com.bvg.models.enums.Gender at de.ruedigermoeller.serialization.FSTObjectInput.readObject(FSTObjectInput.java:168) ~[fst-1.53.jar:na] at org.hibernate.cache.redis.serializer.FstRedisSerializer.deserialize(FstRedisSerializer.java:51) ~[hibernate-redis-1.5.10.jar:na] at org.hibernate.cache.redis.serializer.SnappyRedisSerializer.deserialize(SnappyRedisSerializer.java:41) [hibernate-redis-1.5.10.jar:na] at org.hibernate.cache.redis.jedis.JedisClient.deserializeValue(JedisClient.java:475) [hibernate-redis-1.5.10.jar:na] at org.hibernate.cache.redis.jedis.JedisClient.get(JedisClient.java:165) [hibernate-redis-1.5.10.jar:na] at org.hibernate.cache.redis.regions.RedisTransactionalDataRegion.get(RedisTransactionalDataRegion.java:75) [hibernate-redis-1.5.10.jar:na] at org.hibernate.cache.redis.strategy.AbstractReadWriteRedisAccessStrategy.get(AbstractReadWriteRedisAccessStrategy.java:52) [hibernate-redis-1.5.10.jar:na] at org.hibernate.engine.internal.CacheHelper.fromSharedCache(CacheHelper.java:55) [hibernate-core-4.3.4.Final.jar:4.3.4.Final] at org.hibernate.engine.internal.CacheHelper.fromSharedCache(CacheHelper.java:67) [hibernate-core-4.3.4.Final.jar:4.3.4.Final] at org.hibernate.event.internal.DefaultLoadEventListener.loadFromSecondLevelCache(DefaultLoadEventListener.java:597) [hibernate-core-4.3.4.Final.jar:4.3.4.Final] at org.hibernate.event.internal.DefaultLoadEventListener.doLoad(DefaultLoadEventListener.java:451) [hibernate-core-4.3.4.Final.jar:4.3.4.Final] at org.hibernate.event.internal.DefaultLoadEventListener.load(DefaultLoadEventListener.java:212) [hibernate-core-4.3.4.Final.jar:4.3.4.Final] at org.hibernate.event.internal.DefaultLoadEventListener.proxyOrLoad(DefaultLoadEventListener.java:274) [hibernate-core-4.3.4.Final.jar:4.3.4.Final] at org.hibernate.event.internal.DefaultLoadEventListener.onLoad(DefaultLoadEventListener.java:150) [hibernate-core-4.3.4.Final.jar:4.3.4.Final] at org.hibernate.internal.SessionImpl.fireLoad(SessionImpl.java:1070) [hibernate-core-4.3.4.Final.jar:4.3.4.Final] at org.hibernate.internal.SessionImpl.access$2000(SessionImpl.java:176) [hibernate-core-4.3.4.Final.jar:4.3.4.Final] at org.hibernate.internal.SessionImpl$IdentifierLoadAccessImpl.load(SessionImpl.java:2551) [hibernate-core-4.3.4.Final.jar:4.3.4.Final] at org.hibernate.internal.SessionImpl.get(SessionImpl.java:955) [hibernate-core-4.3.4.Final.jar:4.3.4.Final] at com.bvg.models.daos.DaoImpl.get(DaoImpl.java:78) [classes/:na] at com.bvg.models.daos.DaoImpl.getWithRetry(DaoImpl.java:83) [classes/:na] at com.bvg.actors.DbBasedActor.execute(DbBasedActor.java:52) [classes/:na] at com.bvg.actors.DbBasedActor.execute(DbBasedActor.java:16) [classes/:na] at com.bvg.models.session.HibernateSessionHelper.executeTransactionally(HibernateSessionHelper.java:82) [classes/:na] at com.bvg.actors.DbBasedActor.onReceive(DbBasedActor.java:37) [classes/:na] at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:167) [akka-actor_2.10.jar:2.2.0] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498) [akka-actor_2.10.jar:2.2.0] at akka.actor.ActorCell.invoke(ActorCell.scala:456) [akka-actor_2.10.jar:2.2.0] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237) [akka-actor_2.10.jar:2.2.0] at akka.dispatch.Mailbox.run(Mailbox.scala:219) [akka-actor_2.10.jar:2.2.0] at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386) [akka-actor_2.10.jar:2.2.0] at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [scala-library.jar:na] at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [scala-library.jar:na] at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [scala-library.jar:na] at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [scala-library.jar:na] Caused by: java.lang.RuntimeException: CLASSNAME:com.bvg.models.enums.Gender at de.ruedigermoeller.serialization.FSTClazzNameRegistry.classForName(FSTClazzNameRegistry.java:198) ~[fst-1.53.jar:na] at de.ruedigermoeller.serialization.FSTClazzNameRegistry.classForName(FSTClazzNameRegistry.java:178) ~[fst-1.53.jar:na] at de.ruedigermoeller.serialization.FSTClazzNameRegistry.decodeClass(FSTClazzNameRegistry.java:161) ~[fst-1.53.jar:na] at de.ruedigermoeller.serialization.FSTObjectInput.readClass(FSTObjectInput.java:879) ~[fst-1.53.jar:na] at de.ruedigermoeller.serialization.FSTObjectInput.instantiateEnum(FSTObjectInput.java:335) ~[fst-1.53.jar:na] at de.ruedigermoeller.serialization.FSTObjectInput.instantiateSpecialTag(FSTObjectInput.java:272) ~[fst-1.53.jar:na] at de.ruedigermoeller.serialization.FSTObjectInput.readObjectWithHeader(FSTObjectInput.java:254) ~[fst-1.53.jar:na] at de.ruedigermoeller.serialization.FSTObjectInput.readArray(FSTObjectInput.java:816) ~[fst-1.53.jar:na] at de.ruedigermoeller.serialization.FSTObjectInput.instantiateArray(FSTObjectInput.java:325) ~[fst-1.53.jar:na] at de.ruedigermoeller.serialization.FSTObjectInput.instantiateSpecialTag(FSTObjectInput.java:287) ~[fst-1.53.jar:na] at de.ruedigermoeller.serialization.FSTObjectInput.readObjectWithHeader(FSTObjectInput.java:254) ~[fst-1.53.jar:na] at de.ruedigermoeller.serialization.FSTObjectInput.readObjectFields(FSTObjectInput.java:522) ~[fst-1.53.jar:na] at de.ruedigermoeller.serialization.FSTObjectInput.instantiateAndReadNoSer(FSTObjectInput.java:402) ~[fst-1.53.jar:na] at de.ruedigermoeller.serialization.FSTObjectInput.readObjectWithHeader(FSTObjectInput.java:261) ~[fst-1.53.jar:na] at de.ruedigermoeller.serialization.FSTObjectInput.readObjectInternal(FSTObjectInput.java:230) ~[fst-1.53.jar:na] at de.ruedigermoeller.serialization.FSTObjectInput.readObject(FSTObjectInput.java:210) ~[fst-1.53.jar:na] at de.ruedigermoeller.serialization.FSTObjectInput.readObject(FSTObjectInput.java:165) ~[fst-1.53.jar:na] ... 33 common frames omitted Caused by: java.lang.ClassNotFoundException: com.bvg.models.enums.Gender at java.net.URLClassLoader$1.run(URLClassLoader.java:372) ~[na:1.8.0_05] at java.net.URLClassLoader$1.run(URLClassLoader.java:361) ~[na:1.8.0_05] at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_05] at java.net.URLClassLoader.findClass(URLClassLoader.java:360) ~[na:1.8.0_05] at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_05] at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_05] at java.lang.Class.forName0(Native Method) ~[na:1.8.0_05] at java.lang.Class.forName(Class.java:340) ~[na:1.8.0_05] at de.ruedigermoeller.serialization.FSTClazzNameRegistry.classForName(FSTClazzNameRegistry.java:184) ~[fst-1.53.jar:na] ... 49 common frames omitted

RuedigerMoeller commented 10 years ago

I am currently working on 2.0. However jdk 1.8 ofc should work. My standard testcases did work with 1.8 so it must be a special case. Can you isolate the problematic Enum so I can reproduce ? Additonally you should upgrade to 1.55.

RuedigerMoeller commented 10 years ago

on a first glance it looks like a classpath/classloader problem.

apotapov commented 10 years ago

Thanks for taking a look. As I mentioned we are using the library indirectly, but I'll mess around with it a bit and see if I can get a reproducible test case going. I'll also attempt to update a local copy of hibernate-redis with latest version of FST and see if that solves the issue.

RuedigerMoeller commented 10 years ago

The upgrade probably won't help. Best thing would be a reproducer :-). I am also fine with a complet project copy ..

RuedigerMoeller commented 10 years ago

no news here. Given that all testcases and bigger apps work with 1.8+FST, I assume it was a classpath problem as predicted.

apotapov commented 10 years ago

You were absolutely right. This has nothing to do with Java 8, but in fact a classloader issue.

The problem is that I'm using your library through hibernate-redis, and I'm a Play Framework app.

Here's the problem I'm seeing. If I use regular Class.forName("com.feud.enums.Gender") everything resolves fine. However when I use the classloader fromFSTConfiguration.class.getClassLoader()` it can't seem to resolve Gender. Here's the code:

        Class<?> clazz = null;
        try {
            ClassLoader loader = FSTConfiguration.class.getClassLoader();
            clazz = Class.forName("com.feud.enums.Gender", false, loader);
        } catch (Exception e) {
            log.error("Not on classpath", e);
        }

I've been trying to research why this is the case, and why the FSTConfiguration class loader cannot access my classes, and I'm stumped. Any insight would be much appreciated.

I'm using sbt build system that comes with play framework. And the only thing I can think of is that it's not adding my classes to the classpath when the app initially starts up, but loads them in later?? I have no idea. This bug is really pushing the boundaries of my java knowledge.

Thanks in advance.

apotapov commented 10 years ago

Ugh, looks like it's an issue with Play Framework and their fancy classloaders...

https://github.com/playframework/playframework/issues/2847

Apologies for the confusion.

RuedigerMoeller commented 10 years ago

Thanks for your feedback. Half decent Classloader mechanisms frequently are a cause of problems. Note you can set the Classloader to be used at FSTConfiguration.setClassloader. If you need to dynamically switch Classloaders, using/returning a threadlocal classloader might provide a workaround (requires some subclassing).

apotapov commented 10 years ago

Yep, unfortunately it's not so easy to inject a different classloader since I'm using this library by proxy (hibernate-redis). Oh, life's little challenges.

RuedigerMoeller commented 10 years ago

I'll add a FSTConfiguration factory so an application has the chance to modify/subclass FSTConfigurations in v2.0.

apotapov commented 10 years ago

Sounds great. Thank you.