jerrinot / subzero

SubZero - Fast Serialization for Hazelcast
Apache License 2.0
50 stars 13 forks source link

Custom Kryo serializer for unmodificable collections #46

Open llaros opened 3 years ago

llaros commented 3 years ago

Hello,

I've read https://github.com/jerrinot/subzero/issues/8 and therefore placed subzero-serializers.properties file with UnmodifiableCollectionsSerializer as its content.

Project dependencies

                  <dependency>
            <groupId>info.jerrinot</groupId>
            <artifactId>subzero-all</artifactId>
            <version>0.10</version>
        </dependency>
        <dependency>
            <groupId>de.javakaffee</groupId>
            <artifactId>kryo-serializers</artifactId>
            <version>0.45</version> <!-- tried also with 0.49 same problem -->
        </dependency>

hazelcast.xml with following element

    <serialization>
        <serializers>
            <global-serializer override-java-serialization="true">info.jerrinot.subzero.Serializer</global-serializer>
        </serializers>
    </serialization>

But got following exception on hz node startup:

Caused by: java.lang.IllegalStateException: Serializer UnmodifiableCollectionsSerializer does not have expected method 'registerSerializers()': 
obh-cache-hz-cluster_1                    |     at info.jerrinot.subzero.internal.PropertyUserSerializer.addNewSpecialSerializer(PropertyUserSerializer.java:90)
obh-cache-hz-cluster_1                    |     at info.jerrinot.subzero.internal.PropertyUserSerializer.readLineAndRegister(PropertyUserSerializer.java:73)
obh-cache-hz-cluster_1                    |     at info.jerrinot.subzero.internal.PropertyUserSerializer.initCustomSerializers(PropertyUserSerializer.java:54)
obh-cache-hz-cluster_1                    |     at info.jerrinot.subzero.internal.PropertyUserSerializer.<init>(PropertyUserSerializer.java:41)
obh-cache-hz-cluster_1                    |     at info.jerrinot.subzero.Serializer.<init>(Serializer.java:10)

I've decompiled UnmodifiableCollectionsSerializer and assured that it has

 public static void registerSerializers(Kryo kryo) {
        UnmodifiableCollectionsSerializer serializer = new UnmodifiableCollectionsSerializer();
        UnmodifiableCollectionsSerializer.UnmodifiableCollection.values();
        UnmodifiableCollectionsSerializer.UnmodifiableCollection[] var2 = UnmodifiableCollectionsSerializer.UnmodifiableCollection.values();
        int var3 = var2.length;

        for(int var4 = 0; var4 < var3; ++var4) {
            UnmodifiableCollectionsSerializer.UnmodifiableCollection item = var2[var4];
            kryo.register(item.type, serializer);
        }

    }
llaros commented 3 years ago

IMHO the problem lies with the expected Kryo object or more precisely its package com.esotericsoftware.kryo.Kryo vs info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.Kryo

llaros commented 3 years ago

Problem solved with following dependencies

<dependency>
            <groupId>info.jerrinot</groupId>
            <artifactId>subzero-core</artifactId>
            <version>0.10</version>
        </dependency>
        <dependency>
            <groupId>de.javakaffee</groupId>
            <artifactId>kryo-serializers</artifactId>
            <version>0.45</version>
        </dependency>
        <dependency>
            <groupId>com.esotericsoftware</groupId>
            <artifactId>kryo</artifactId>
            <version>4.0.2</version>
        </dependency>
llaros commented 3 years ago

I've reopened just because I don't know whether or notshould subzero-all work out-of-the box with kryo-serializers.

jerrinot commented 3 years ago

hi, thanks for reporting this, your analysis looks correct. I'll check what to do about it.

jerrinot commented 3 years ago

@llaros I'm afraid there is no good solution. The serializer plugins expect Kryo in the original package. I could either:

  1. Not relocate Kryo at all. That's already the case with subzero-core.
  2. Include (and relocate) selected serialized plugins inside subzero-all but it does not feel good.

Any other idea?

jerrinot commented 3 years ago

Some more thoughts: Subzero-All could relocate Kryo dependencies, but keep Kryo itself in its original packages. Why? Kryo depends on some fairly popular low-level infrastructure libraries, such as Asm or Objenesis. Not relocating them could lead to a bad JAR hell. Kryo itself is less popular and the risk of conflicts is lower. Not relocating it will allow 3rd party serializers to work. Thanks to @viliam-durina for the idea!