When entity add a field extends Collection, CompatibleFieldSerializer lost compatibility.

[iicreator]

Describe the bug

When serializing, the entity used has one additional field of type List compared to the entity used for deserialization, which throw an Exception during deserialization. I think the CompatibleFieldSerializer is supposed to handle the addition of new fields, so this situation does not meet expectations. After briefly examining the Kryo source code, I found that the CollectionSerializer, when serializing, does not write the elementType if it has access to the elementSerializer, and uses kryo.writeObject to write the element. However, during deserialization, kryo.getGenerics().nextGenericClass() returns null and the elementSerializer cannot be obtained, so kryo.readClassAndObject is used to read, which seems to be causing the problem.

To Reproduce Please delete PoJo.list when deserializing

// KryoFactory
public class CompatibleKryoFactory {

    private static final int MAX_CAPACITY = 8;
    private final Pool<Kryo> kryoPool;
    /**
     * using zero args constructor as default
     * if not exists, fallback using ASM
     */
    private static final InstantiatorStrategy STRATEGY =
            new DefaultInstantiatorStrategy(new StdInstantiatorStrategy());

    public CompatibleKryoFactory() {
        kryoPool = new Pool<Kryo>(true, true, MAX_CAPACITY) {
            @Override
            protected Kryo create() {
                Kryo kryo = new Kryo();
                init(kryo);
                return kryo;
            }
        };
    }

    public Kryo obtain() {
        return kryoPool.obtain();
    }

    public void free(Kryo kryo) {
        kryoPool.free(kryo);
    }

    protected void init(Kryo kryo) {
        kryo.setReferences(false);
        kryo.setRegistrationRequired(false);
        kryo.setInstantiatorStrategy(STRATEGY);
        kryo.setDefaultSerializer(CompatibleFieldSerializer.class);
    }
}

// entity
@Data
public class PoJo {
    private String str;
    private int i;
    // delete when deserializing
    private List<String> list;
}

// Serializer
public class SerializerTest {

    public static final CompatibleKryoFactory FACTORY = new CompatibleKryoFactory();

    public static void main(String[] args) {
        PoJo poJo = new PoJo();
        poJo.setI(1);
        poJo.setStr("test");
        List<String> list = new ArrayList<>();
        list.add("hello");
        poJo.setList(list);

        Kryo kryo = FACTORY.obtain();

        try (ByteArrayOutputStream outputStream = new ByteArrayOutputStream()) {
            Output output = new Output(outputStream);
            kryo.writeClassAndObject(output, poJo);
            output.flush();
            System.out.println(Base64.getEncoder().encodeToString(outputStream.toByteArray()));
            output.close();
        } catch (Exception e) {
            log.error("failed", e);
        } finally {
            FACTORY.free(kryo);
        }

    }
}

// Deserializer
public class DeserializerTest {
    public static void main(String[] args) {
        Kryo kryo = SerializerTest.FACTORY.obtain();

        String str = "AQBjb20udGVzdC5Qb0rvA4JpbGlz9HN08gICAQFqYXZhLnV0aWwuQXJyYXlMaXP0AmhlbGzvA3Rlc/Q=";
        byte[] bytes = Base64.getDecoder().decode(str);
        try (ByteArrayInputStream inputStream = new ByteArrayInputStream(bytes);) {
            Input input = new Input(inputStream);
            PoJo poJo = (PoJo) kryo.readClassAndObject(input);
            System.out.println(poJo);
        } catch (Exception e) {
            log.error("failed", e);
        } finally {
            SerializerTest.FACTORY.free(kryo);
        }
    }

}

Environment:

• OS: win10
• JDK Version: 11
• Kryo Version: 5.6.0

Additional context Add any other context about the problem here.

[theigl]

Thanks for the detailed report.

What error message are you seeing? Is your issue the same as #907?

[iicreator]

Thanks for the detailed report.

What error message are you seeing? Is your issue the same as #907?

com.esotericsoftware.kryo.KryoException: Unable to read unknown data, type: java.util.ArrayList (com.test.PoJo#null) at com.esotericsoftware.kryo.serializers.CompatibleFieldSerializer.read(CompatibleFieldSerializer.java:157)

yes, it's the same. However, I think this is irrelevant to ChunkedEncoding, probably a bug.

[theigl]

Can you test if the problem goes away with chunked encoding?

You could also try disabling generics optimization with kryo.setOptimizedGenerics(false);.

This is definitely a bug, but one that is difficult to resolve and that I already spent a couple of hours on in the past without success. I can't spend any more time on this at the moment but I'd be happy to accept a pull request.

[iicreator]

Can you test if the problem goes away with chunked encoding?

You could also try disabling generics optimization with kryo.setOptimizedGenerics(false);.

This is definitely a bug, but one that is difficult to resolve and that I already spent a couple of hours on in the past without success. I can't spend any more time on this at the moment but I'd be happy to accept a pull request.

Thanks for help. Chunked Encoding is useless, but kryo.setOptimizedGenerics(false) is useful.