study: serialization

[oldratlee]

🍺 serialization framework

Kryo

• repo: https://github.com/EsotericSoftware/kryo
• EsotericSoftware/kryo: Java binary serialization and cloning: fast, efficient, automatic.
• Kryo is a fast and efficient binary object graph serialization framework for Java.
• Additional serializers(for some jdk types and some external libs like e.g. joda time) are available in a separate project on github, kryo-serializers.

JBoss Marshalling

• repo: https://github.com/jboss-remoting/jboss-marshalling
• an alternative serialization API that
  • fixes many of the problems found in the JDK serialization API while remaining fully compatible with java.io.Serializable and its relatives,
  • and adds several new tunable parameters and additional features, all of which are pluggable via factory configuration (externalizers, class/instance lookup tables, class resolution, and object replacement, to name a few).
• this framework was inspired by the need for certain features unavailable with the standard Object*Stream classes
  • Pluggable class resolvers, making it easy to customize classloader policy, by implementing a small interface (rather than having to subclass the Object*Stream classes)
  • Pluggable object replacement (also without subclassing)
  • Pluggable predefined class tables, which can dramatically decrease stream size and serialization time for stream types which frequently use a common set of classes
  • Pluggable predefined instance tables, which make it easy to handle remote references
  • Pluggable externalizers which may be used to serialize classes which are not Serializable, or for which an alternate strategy is needed
  • Customizable stream headers
  • Each marshaller instance is highly configurable and tunable to maximize performance based on expected usage patterns
  • A generalized API which can support many different protocol implementations, including protocols which do not necessarily provide all the above features
  • Inexpensive instance creation, beneficial to applications where many short-lived streams are used
  • Support for separate class and instance caches, if the protocol permits; useful for sending multiple messages or requests with a single stream, with separate object graphs but retaining the class cache
• home page: http://jbossmarshalling.jboss.org/
• maintenance is not active...
  • although personally I like the product of jboss and it's high quality 😩

🍺 serialization specification

Protocol Buffers

• repo: https://github.com/protocolbuffers/protobuf
• most widely used, especially when need support multi-languages.
• Protocol Buffers (a.k.a., protobuf) are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data.
• home page: https://developers.google.com/protocol-buffers/

protostuff

improvement/enhancement of java; independent implementation of java.

• Java serialization library, proto compiler, code generator
• repo: https://github.com/protostuff/protostuff
• official docs: https://protostuff.github.io/docs/

[oldratlee]

Kryo

分析

• Kryo为什么比Hessian快 https://x-rip.iteye.com/blog/1555344 有Kryo生成的 二进制字节 分析
• Kryo 使用指南 https://www.cnblogs.com/hntyzgn/p/7122709.html 提到使用注意的问题，如
  • 缺省的设置 不支持 增加或删除 Bean 中的字段，使用的是FieldSerializer
  • 注册机制生成ClassId
  • Kryo 对象不是线程安全的
• Kryo 序列化说明 https://my.oschina.net/u/725800/blog/823467
• Strategy for registering classes with kryo https://stackoverflow.com/questions/7780707
• Kryo Serialization Type Detection https://stackoverflow.com/questions/22079054
• Kryo Class Registration https://stackoverflow.com/questions/31576169
  • Since Kryo use Minlog library for logging, you can enable the trace level log by adding com.esotericsoftware.minlog.Log.TRACE(); to the source code. Enabling TRACE level log will print the details about all the classes which are registered with Kryo.
• Kryo read and write generically https://stackoverflow.com/questions/30184019
• Dealing with an incompatible version change of a serialization framework https://stackoverflow.com/questions/16083263
• Spark 2.0.2, double[], 使用Kyro序列化加速，和手动注册类名 https://facaiy.com/misc/2017/01/21/spark-kyro.html
• Kryo官方文档翻译
  • https://blog.csdn.net/fanjunjaden/article/details/72823866
  • https://github.com/chengdedeng/blog/issues/9

注意的问题

• 字段 + 循环引用下有的一个问题： CompatibleFieldSerializer fails with IndexOutOfBoundsException on field removal https://github.com/EsotericSoftware/kryo/issues/286#issuecomment-74870545
• 使用kryo做序列化会遇到的几个坑 https://blog.csdn.net/paoma_1008/article/details/79827803
  • com.esotericsoftware.kryo.KryoException: Class cannot be created (missing no-arg constructor): java.util.Collections$SynchronizedRandomAccessList
  • com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID: XXXX