Cysharp / MemoryPack

Zero encoding extreme performance binary serializer for C# and Unity.
MIT License
3.29k stars 193 forks source link

Need help adding formatter for object type in dictionary<int, object> serialization #186

Closed Giresharu closed 7 months ago

Giresharu commented 1 year ago

I have a class that has a dictionary of type Dictionary<int, object>. It is used to store some data of various types. Now I want to deserialize the stored data, but I get the error message:

MemoryPackSerializationException: System.Object is not registered in this provider.
MemoryPack.MemoryPackSerializationException.ThrowNotRegisteredInProvider (System.Type type) (at Assets/Packages/MemoryPack/Runtime/MemoryPack.Core/MemoryPackSerializationException.cs:84)

I have looked up similar issue and found out that object is not supported by default and I need to add a formatter for it. But I'm not quite sure how I should add a formatter for the object type. Should I add a formatter for Dictionary<int, object> or is there a more complete example to refer to?

Here is my minimal code in Unity 2021.3 Environment:

using MemoryPack;
using System.Collections.Generic;
using System.IO;
using UnityEditor;
using UnityEngine;

[MemoryPackable]
public partial class MemoryClass {
    public Dictionary<int, object> dictionary;
}

public class TestMemory {

    [MenuItem("Tools/Test MemoryPack")]
    public static void Save() {
        using (FileStream stream = new FileStream(Path.Combine(Application.persistentDataPath, "Memory.bin"), FileMode.Create)) {
            stream.Write(MemoryPackSerializer.Serialize(new MemoryClass() {
                dictionary = new Dictionary<int, object>() { { 1, "2" }, { 2, true }, {3, new int[]{1,2,3,4,5}} }
            }));
        }
    }
}

I've been researching for a few days and wrote the following code:

public class ObjectFormatter : MemoryPackFormatter<object> {
    public override void Serialize(ref MemoryPackWriter writer, ref object value) {
        if (value == null) {
            writer.WriteNullObjectHeader();
            return;
        }
        if (value is int i) {
            writer.WriteVarInt(i);
            return;
        }
        if (value is int[] ints) {
            writer.WriteArray(ints);
            return;
        }

        if (value is string str) {
            writer.WriteString(str);
            return;
        }
        if (value is string[] strs) {
            writer.WriteArray(strs);
            return;
        }

        // CustomType with [MemoryPackable] Attribute
        if (value is CustomClass custom) {
            writer.WritePackable(custom);
            return;
        }

        // How to WriteBool or WriteDictionary ?

    }
    public override void Deserialize(ref MemoryPackReader reader, ref object value) {
        if (reader.PeekIsNull()) {
            reader.Advance(1);
            value = null;
            return;
        }

        value = reader.ReadVarIntInt32();
        // reader.ReadValue(ref value); 
        // Use ReadValue(ref value) will throw a StackOverflowException: The requested operation caused a stack overflow.

        // How to make the reader determine the type and assign a value?
    }
}

This code can only serialize/deserialize objects with the actual type of int. I don't know how to write the corresponding serialization for types like bool or Dictionary, etc. Also, during deserialization, I'm not sure how to determine what type the reader should currently read. I hope to receive some guidance.

YCZA commented 11 months ago

I have the same problem, can anyone offer some help?

tedbarth commented 8 months ago

Same issue here: https://github.com/Cysharp/MemoryPack/issues/225

hadashiA commented 7 months ago

MemoryPack's binary specification does not allow retrieval of data structures without a C# type definition.

So it seems difficult to support this. ( By the way, this is a similar restriction to binary specifications such as protobuf.

If you want to dump binary data dynamically, it seems to me that MessagePack or JSON is probably the way to go .

Also, if you know the type variations in advance, please try Union.