EsotericSoftware / kryo

Java binary serialization and cloning: fast, efficient, automatic
BSD 3-Clause "New" or "Revised" License
6.21k stars 827 forks source link

Recommendation for CompatibleFieldSerializer #684

Open maxisvest opened 5 years ago

maxisvest commented 5 years ago

I read the example in #643, and I see that in @NathanSweet last comment says when decode SomeClass#anotherClass but AnotherClass's structure is missing, it will cause decode failure. Because CompatibleFieldSerializer serialize is great depandency in order. When reading, the AnotherClass class doesn't exist, Kryo will skip AnotherClass 's chunked input buffer. See CompatibleFieldSerializer#read's readUnknownTagData part down below.

Registration registration;
try {
    registration = kryo.readClass(fieldInput);
} catch (KryoException ex) {
    if (!chunked)
        throw new KryoException("Unable to read unknown data (unknown type). (" + getType().getName() + ")", ex);
    if (DEBUG) debug("kryo", "Unable to read unknown data (unknown type).", ex);
        inputChunked.nextChunk();
        continue;
}

But the information of SomeClass#value is still int that buffer, so if we continue read the remaing buffer that we can cache the SomeClass#value in SomeClass#anotherClass's chunked input. So I simply do that:

Registration registration;
try {
    registration = kryo.readClass(fieldInput);
} catch (KryoException ex) {
    if (!chunked)
        throw new KryoException("Unable to read unknown data (unknown type). (" + getType().getName() + ")", ex);
    if (DEBUG) debug("kryo", "Unable to read unknown data (unknown type).", ex);
        //inputChunked.nextChunk();
        //continue;
               registration = kryo.getRegistration(Object.class);
}

If Kryo can't find the Class so we give it a default Class that has no field in it, then Kryo successfully read the remaing byte in buffer and cache the SomeClass#value in SomeClass#anotherClass's chunked input.

This change is usefull in my case, but I don't know it will have impact for other case? Or you consider add this change (maybe with configurable) in futrue release?

Looking forward to your reply.

NathanSweet commented 4 years ago

Sorry for the very late response!

You lost me at:

But the information of SomeClass#value is still int that buffer,

Each field is written chunked, so when we can't read a field and inputChunked.nextChunk();, we only skip the bytes for that field.

Maybe you can explain more how you are using your proposed change?

maxisvest commented 4 years ago

@NathanSweet

is still int that buffer

is a wrong spelling,I mean

But the information of SomeClass#value is still in that buffer

which means in serializingSomeClass#anotherClass chunk is contains SomeClass#value Class infomation and cache this information as a index to serialize SomeClass#value, but in deserializng, inputChunked.nextChunk() will skip this thing, so when deserialize to SomeClass#value, kryo find that the index it can not mapping or mismatch the true Class infomation.

This change is try to resolve this problem, force kryo read the next chunk is safely, if kryo.getRegistration(Object.class) throw an execption, that's mean this chunk is truly broken and should be skip, if not, we may cache some Class information which possibly used by the next few chunk.

This change is already in our project and works fine, so you can consider merge this PR #686

NathanSweet commented 1 year ago

Sorry it's been so long.

While I don't have time right now to run the code in the PR, it doesn't appear valid to use the Object registration when kryo.readClass fails. Why would the serializer for Object be able to deserialize the bytes for some other class?