Serialization Guidance for Source-Generated Fields (Vogen)

[cmeyertons]

It would be fantastic if Orleans could support Value Objects source-generated by Vogen.

There is currently an outstanding issue on the Vogen library here:

https://github.com/SteveDunn/Vogen/issues/385

We can't simply source generate the [GenerateSerializer] and [Id] attributes because the Orleans codegen won't pick them up at its source generation time.

Is there any guidance around minimally structuring user-land code to better inform the Orleans serializer how to serialize the value type?

Here's the intial approach I came up w/ -- it uses the assumption that each ValueObject will have one and only one field, Value (which Vogen enforces during compilation)

edit: did some more testing and need to add a implementation of IFieldCodec as well.

[ValueObject<string>]
[Immutable, RegisterSerializer]
public readonly partial record struct ValueObjectTest : IValueSerializer<ValueObjectTest>, IFieldCodec<ValueObjectTest>
{
    [GenerateSerializer]
    public readonly record struct Surrogate(string Value);

    public void WriteField<TBufferWriter>(ref Writer<TBufferWriter> writer, uint fieldIdDelta, Type expectedType, ValueObjectTest value)
        where TBufferWriter : IBufferWriter<byte>
    {
        StringCodec.WriteField(ref writer, fieldIdDelta, value.Value);
    }

    public ValueObjectTest ReadValue<TInput>(ref Reader<TInput> reader, Field field)
    {
        var value = StringCodec.ReadValue(ref reader, field);
        return From(value);
    }

    public void Serialize<TBufferWriter>(ref Writer<TBufferWriter> writer, scoped ref ValueObjectTest value) where TBufferWriter : IBufferWriter<byte>
    {
        StringCodec.WriteField(ref writer, 0, value.Value);
    }

    public void Deserialize<TInput>(ref Reader<TInput> reader, scoped ref ValueObjectTest value)
    {
        var header = reader.ReadFieldHeader();
        var stringValue = StringCodec.ReadValue(ref reader, header);
        value = From(stringValue);
    }
}

[jsteinich]

It's generally possible to chain source generates through intermediate projects. For your case you'd have the Vogen SG run on one project and then create a second project that references the first and applies the Orleans SG. You'll also need a small link so that Orleans knows which code to generate. Having [assembly:Orleans.GenerateCodeForDeclaringAssembly(typeof(<MyType>))] in an otherwise empty cs file is enough (likely other ways possible).

We are using this approach to run an in-house SG before the Orleans SG.

[cmeyertons]

That's good feedback, and probably is do-able if you are following a hexagonal architecture approach and have your value objects in the domain layer w/o any direct dependency on Orleans.

Expecting all users to split code into two projects seems like a pretty heavy requirement for a library to ask for proper serialization to be generated though :/

What are your thoughts on the approach above using a [RegisterSerializer] on the value object itself? It seems to work and is simple because it leverages the assumptions of the ValueObject.

In Vogen's readme, it declares that strongly typed serializers will only be generated for the known primitive types, and will use object otherwise, so this is something we could theoretically generate a serializer permutation per primitive type?

[jsteinich]

Expecting all users to split code into two projects seems like a pretty heavy requirement for a library to ask for proper serialization to be generated though :/

I agree. https://github.com/dotnet/roslyn/issues/57239 would offer an alternative path to handling multiple source generators.

What are your thoughts on the approach above using a [RegisterSerializer] on the value object itself? It seems to work and is simple because it leverages the assumptions of the ValueObject.

The docs push more towards using a converter. Writing a codec by hand is tricky. I believe what you have is functional, but it is missing some additional ceremony found in other codecs. My understanding is that the additional pieces cover things like corruption and version tolerance. The converter route would handle those concerns for you.

[cmeyertons]

I completely agree - I wanted to do the converter-based approach as well. It doesn’t work in a turnkey fashion due to the source generator ordering again. The surrogate has to be defined and decorated in user-land for Orleans to pick it up, which I would prefer to avoid forcing on every single value object that wants this behavior.

Is there an approach using a converter that I’m missing? Where the surrogate does not have to be separately defined in user-land?

[jsteinich]

You could manually create one surrogate type (with attribute) for every basic value object backing type (could still create definition like example for other backing types). Then could create a source generator to create the converter types. The generator can also generate a method that can be called to add to the configuration at runtime (I believe just adding to TypeManifestOptions).

[truegoodwill]

I solved this problem while using the StronglyTypedId source generator for Value Objects. The StronglyTypedId source generator (prerelease 1.x.x, be sure to get the latest pre-release version) lets you define custom templates using text files, so it has a certain flexibility that other source generators of this kind don't have.

In its original form, the StronglyTypedId generator would have you write this:

[GenerateSerializer, Immutable]
[StronglyTypedId(Template.Int)]
public readonly partial struct SomeId;

It would then generate code in a partial class containing the value property and the constructor like so.

partial struct SomeId { 
  public int Value { get; }
  public SomeId(int value) => Value = value;

  // ... plus 58 million lines of useful stuff
}

The problem was that the [Id(0)] attribute was not added to the Value property. Then, when I added the Id attribute to the custom template to be included in the source generation, the Orleans serializer did not pick it up, because as stated by @cmeyertons, the Orleans serializer won't "notice" the source-generated parts of the code.

My solution was to remove the value property and the constructor from the text template used by the source generator, and manually add them to original code that I typed out myself. So in the end I typed the little extra shown below, while the source generator still took care of the other 58 million lines of source-generated code, and Orleans was very happy.

[GenerateSerializer, Immutable]
[StronglyTypedId(Template.Int)]
public readonly partial struct SomeId 
{
  [Id(0)] public int Value { get; }
  public SomeId(int value) => Value = value; 
}