google / guava

Google core libraries for Java
Apache License 2.0
50.02k stars 10.86k forks source link

BloomFilter Serialization #6068

Open matebase opened 2 years ago

matebase commented 2 years ago

BloomFilter size keeps increasing when serializing and deserializing with kryo ?

Below is the relevant code

import com.esotericsoftware.kryo.kryo5.Kryo;
import com.esotericsoftware.kryo.kryo5.io.Input;
import com.esotericsoftware.kryo.kryo5.io.Output;
import com.esotericsoftware.kryo.kryo5.objenesis.strategy.StdInstantiatorStrategy;
import com.esotericsoftware.kryo.kryo5.util.DefaultInstantiatorStrategy;
import com.esotericsoftware.kryo.kryo5.util.Pool;
import com.google.common.hash.BloomFilter;
import com.google.common.hash.Funnels;

protected byte[] kryoSerialize(Object obj) {
    Kryo kryo = kryoPool.obtain();
    try {
        ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
        Output output = new Output(byteArrayOutputStream);
        kryo.writeObject(output, obj);
        output.close();
        return byteArrayOutputStream.toByteArray();
    } catch (Exception e) {
        return null;
    } finally {
        kryoPool.free(kryo);
    }
}

protected <T> T kryoDeserialize(byte[] bytes, Class<T> clazz) {
    Kryo kryo = kryoPool.obtain();
    try {
        kryo.setInstantiatorStrategy(new DefaultInstantiatorStrategy(new StdInstantiatorStrategy()));
        ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes);
        Input input = new Input(byteArrayInputStream);
        input.close();
        return (T) kryo.readObject(input, clazz);
    } catch (Exception e) {
        return null;
    } finally {
        kryoPool.free(kryo);
    }
}

Pool<Kryo> kryoPool = new Pool<Kryo>(true, false, 8) {
    @Override
    protected Kryo create() {
        Kryo kryo = new Kryo();
        kryo.setReferences(true);
        kryo.setRegistrationRequired(false);
        return kryo;
    }
};
amalloy commented 2 years ago

Could you be clearer about how you're detecting this size increase? I see you've included adapter code to plug into Kryo, but nothing that specifically reproduces an issue. I also suggest that your minimum reproducible example use normal Java serialization (Object(Input|Output)Stream), rather than Kryo. If you can't reproduce the problem with normal serialization, then the issue would appear to be with Kryo rather than with BloomFilter.