max-ieremenko / ServiceModel.Grpc

Code-first for gRPC
https://max-ieremenko.github.io/ServiceModel.Grpc/
Apache License 2.0
90 stars 11 forks source link

Support for MemoryPack #197

Closed De-Crypted closed 2 weeks ago

De-Crypted commented 8 months ago

Is your feature request related to a problem? Please describe. I was trying to implement custom marshaller for MemoryPack (https://github.com/Cysharp/MemoryPack) but I quickly learned it wouldn't work. The internally used Message class could not be serialized and the reason for this is that MemoryPack does not support [DataContract] attribute nor do they plan on supporting it (https://github.com/Cysharp/MemoryPack/issues/188).

What MemoryPack requires is to make the class partial and annotate it with [MemoryPackable] attribute.

Describe the solution you'd like Could the Message class be exposed as partial class so that it could be annotated with [MemoryPackable], I think this should allow Source Generator to do it's things (not 100% sure).

Or maybe there is some other way to solve this?

Anyways, thank you for this library. I'm enjoying it very much.

max-ieremenko commented 8 months ago

https://github.com/Cysharp/MemoryPack/issues/188 ... Unlike protobuf-net and MessagePack for C#, you first need to make it partial ...

Making Messages classes partial won't help: for your project Message(<,>) is external-type. see Serialize external types.

Or maybe there is some other way to solve this?

An example of the MemoryPackMarshaller draft:

public sealed class MemoryPackJsonMarshallerFactory : IMarshallerFactory
{
    public static readonly IMarshallerFactory Default = new MemoryPackJsonMarshallerFactory();

    static MemoryPackJsonMarshallerFactory()
    {
        MemoryPackFormatterProvider.RegisterGenericType(typeof(Message<>), typeof(MessageMemoryPackFormatter<>));
        // TODO: implement MemoryPackFormatterProvider.RegisterGenericType(typeof(Message<,>), typeof(MessageMemoryPackFormatter<,>));
    }

    public Marshaller<T> CreateMarshaller<T>() => new(Serialize, Deserialize<T>);

    private static void Serialize<T>(T value, SerializationContext context)
    {
        var bufferWriter = context.GetBufferWriter();
        MemoryPackSerializer.Serialize(bufferWriter, value);
        context.Complete();
    }

    private static T Deserialize<T>(DeserializationContext context)
    {
        return MemoryPackSerializer.Deserialize<T>(context.PayloadAsReadOnlySequence())!;
    }
}

// TODO implement internal readonly partial struct SerializableMessage<T1, T2>
[MemoryPackable]
internal readonly partial struct SerializableMessage<T>
{
    internal readonly Message<T> Message;

    [MemoryPackInclude]
    public T Value => Message.Value;

    [MemoryPackConstructor]
    public SerializableMessage(T value)
    {
        Message = new Message<T>(value);
    }

    public SerializableMessage(Message<T> message)
    {
        Message = message;
    }
}

// TODO implement internal sealed class MessageMemoryPackFormatter<T1, T2>
internal sealed class MessageMemoryPackFormatter<T> : MemoryPackFormatter<Message<T>>
{
    public override void Serialize<TBufferWriter>(ref MemoryPackWriter<TBufferWriter> writer, scoped ref Message<T>? value)
    {
        ArgumentNullException.ThrowIfNull(value);

        writer.WritePackable(new SerializableMessage<T>(value));
    }

    public override void Deserialize(ref MemoryPackReader reader, scoped ref Message<T>? value)
    {
        if (reader.PeekIsNull())
        {
            throw new NotSupportedException();
        }

        var wrapped = reader.ReadPackable<SerializableMessage<T>>();
        value = wrapped.Message;
    }
}

Test

MemoryPackFormatterProvider.RegisterGenericType(typeof(Message<>), typeof(MessageMemoryPackFormatter<>));

var input = new Message<string>("foo");

var payload = MemoryPackSerializer.Serialize(input);
var actual = MemoryPackSerializer.Deserialize<Message<string>>(payload);

actual.ShouldNotBeNull();
actual.Value.ShouldBe("foo");
De-Crypted commented 8 months ago

This does look like good solution. If you manage to craft it I can give it a good test run.

max-ieremenko commented 8 months ago

This does look like good solution. If you manage to craft it I can give it a good test run.

I invested my time in helping you, have you tried my code example to solve your issue?

De-Crypted commented 8 months ago

Yes I got to try it today and it works great. I also tested making wrappers for Message (empty message?) and Message<T1, T2> which worked as expected.

Do you want PR for this?

max-ieremenko commented 8 months ago

The approach will work for limited functionality: operation with a maximum of 3 input parameters.

Task DoSomething(p1, p2, p3) // will
Task DoSomething(p1, p2, p3, p4) // will not

You are welcome to create PR Examples/MemoryPackMarshaller, it would be helpful for others, something like Examples/CustomMarshaller.

max-ieremenko commented 2 weeks ago

The new NuGet package ServiceModel.Grpc.MemoryPackMarshaller has been released in version 1.10.1 of ServiceModel.Grpc.

Example of how to use MemoryPackMarshaller.