WCF Migration

[freica]

Are there any Serialization Defaults like in WCF?

[mgravell]

No. The problem is that protobuf is an ordinal rather than nominal format, which means: if we did something like "generate numbers by ordering the fields alphabetically", it would be actively dangerous, as adding a new property/field could catastrophically change the assumed data format. We try really hard not to cause data corruption / data loss issues, which this would make really common. There are ways of configuring types that are outside your control, if that is what you need.

[freica]

The current types are not outside of my control, but it's a cluster of perhaps 150 classes/types sometimes with deep inheritance. This is really a lot of work to do. So I am seaching for an easier way for this.

[mgravell]

That may be, but it is still where my recommendation lies. Note: it is possible to use partial classes and some one-time reflection code to generate all the additional attributes in separate files - or we could in theory write an analyzer tool that offers to generate the relevant attributes for you (although that would be quite a lot of work, too). Any of those viable?

[menaheme]

Pardon the hijack, how do you handle types outside your control?

On Wed, Feb 23, 2022, 5:07 PM Marc Gravell @.***> wrote:

That may be, but it is still where my recommendation lies. Note: it is possible to use partial classes and some one-time reflection code to generate all the additional attributes in separate files - or we could in theory write an analyzer tool that offers to generate the relevant attributes for you (although that would be quite a lot of work, too). Any of those viable?

— Reply to this email directly, view it on GitHub https://github.com/protobuf-net/protobuf-net.Grpc/issues/226#issuecomment-1048881650, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLOIH3NLQGSHOZXYEVCAFLU4TZ4TANCNFSM5O6SZIYQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you are subscribed to this thread.Message ID: @.***>

[mgravell]

@menaheme if they're completely outside of your control, you can still tell the system about them at runtime (rather than via attributes); from RuntimeTypeModel (usually RuntimeTypeModel.Default) you can access the configuration API to tell it about types, sub-types, members, etc - everything that exists in attributes

[freica]

Does some kind of documentation or a sample exist on how to do this job (rather than via attributes) at runtime? But at the moment it sounds not as this way would reduce the work significant.

May be you will laughing at me, but I've already thought about making a json serialization of my data and transfer that result (simple string/byte-array) over gRPC to avoid this overhead of work. On the other hand, this idea doesn't seem so far-fetched, at least in the java world you can also find considerations like this one.

[mgravell]

Absolutely not laughing at you. Right; there's two things we can consider here:

1. configuring protobuf-net at runtime
2. using any other serializer (of your convenience), but via code-first gRPC

The second: isn't very hard, if you're used to serializers. Here's an example of 2 using BinaryFormatter, but please read the notes: you absolutely shouldn't do that! But if, for example, you want to use DataContractSerializer - it might be more expensive in terms of bytes than protobuf, but... is that your blocker? I don't know! But: protobuf-net.Grpc isn't precious about which serializer (/marshaller) you use - it defaults to protobuf-net, but if you want something else: that's fine! You just pass your MarshallerFactory to the relevant APIs (I can assist there).

As for 1: here's a really lazy and hacky approach, which might bite you when somebody comes and adds properties (as it will change the order): tell it to assume alphabetical order. You could do this by annotating each type simply with [ProtoContract(ImplicitFields = ImplicitFields.AllPublic)], but if that is also too much overhead, you can hook a callback API to configure something similar at runtime:

        RuntimeTypeModel.Default.BeforeApplyDefaultBehaviour += (sender, args) =>
        {
            if (!Attribute.IsDefined(args.Type, typeof(ProtoContractAttribute)))
            {   // seize control of anything that *isn't* [ProtoContract]
                args.ApplyDefaultBehaviour = false;
                // get all the public properties, order them alphabetically, and
                // call it a day
                var propNames = (from prop in args.Type.GetProperties(BindingFlags.Public | BindingFlags.Instance)
                             orderby prop.Name
                             select prop.Name).ToArray();
                args.MetaType.Add(propNames);
            }
        };

Here, RuntimeTypeModel.Default is the configuration model used by the Serializer API (technically you can have separate isolated configuration models on the same types, but this is rare, so usually: you just want RuntimeTypeModel.Default). The BeforeApplyDefaultBehaviour event gets fired the first time the engine considers a new type (there's also an "after" API); we're going to check whether the type has been annotated, and if not: disable the default behaviour, and add our own configuration manually. Note that this won't work with inheritance (or rather: you'd need to figure out some deterministic mechanism of discovering and ordering all sub-types, and add them).

[freica]

Marc, both sounds good!

1. configuring protobuf-net at runtime Are there any extensions necessary for derived classed, which normaly have to be declared with [ProtoInclude(7, typeof(SomeDerivedType))] ?

2. using any other serializer (of your convenience), but via code-first gRPC Here I need a code snippet how to assign the custom MarshallerFactory using protobuf-net.Grpc.Native.

But I think first I will give Point 1 a chance. Hope I wouldn't get exceptions at runtime with status=Unknown (Issue #225). Here it will be a hard way to find the matter, because I can't use the analyzer.

[freica]

I tried it out (configuring protobuf-net at runtime), but get an exception at client-side: But there is no event fired for the type NestedClass but for the enum SdbTaskID. So client runs into the exception

Exception: System.InvalidOperationException: No serializer defined for type: StockPriceLib.NestedClass
   bei ProtoBuf.Meta.ValueMember.BuildSerializer() in C:\Code\protobuf-net\src\protobuf-net\Meta\ValueMember.cs:Zeile 523.

Model:

  public class NestedClass
  {
    public string Name { get; set; }
    public int Count { get; set; }

    public new string ToString()
    {
      return $"({Name}, {Count})";
    }
  }

  public enum SdbTaskID
  {
    App1 = 55,
  }

[DataContract]
  public class SubscribeRequest
  {
    [DataMember(Order = 1)]
    public string[] Symbols { get; set; }

    [DataMember(Order = 2)]
    SdbTaskID taskId;

    [DataMember(Order = 3)]
    string name;

    [DataMember(Order = 4)]
    int language;

    [DataMember(Order = 5)]
    NestedClass nestedClass;

    public SdbTaskID TaskId
    {
      get { return this.taskId; }
    }

    public string Name
    {
      get { return this.name; }
    }

    public int Language
    {
      get { return this.language; }
    }

    public NestedClass NestedClass
    {
      get { return this.nestedClass; }
    }

    // Wichtig ansonsten Exception mit Status = 2 !!!
    public SubscribeRequest()
    {
    }

    public SubscribeRequest(string name, int language, SdbTaskID taskId, NestedClass nestedClass)
    {
      this.name = name;
      this.language = language;
      this.taskId = taskId;
      this.nestedClass = nestedClass;
    }
  }
  [ServiceContract]
  public interface IStockTickerService
  {
    [OperationContract]
    IAsyncEnumerable<StockTickerUpdate> SubscribeAsync(SubscribeRequest request, CallContext context = default);
  }

RuntimeInitializer:

  public static class RuntimeInitializer
  {
    public static void Initialize()
    {
      // add all types which are not "contracted" at runtime
      RuntimeTypeModel.Default.BeforeApplyDefaultBehaviour += (sender, args) =>
      {
        Console.WriteLine($"BeforeApplyDefaultBehaviour() called (type={args.Type})");
        if (!Attribute.IsDefined(args.Type, typeof(ProtoContractAttribute)) && !Attribute.IsDefined(args.Type, typeof(DataContractAttribute)))
        {   // seize control of anything that *isn't* [ProtoContract] or [DataContract]
          args.ApplyDefaultBehaviour = false;
          // get all the public properties, order them alphabetically, and
          // call it a day
          var propNames = (from prop in args.Type.GetProperties(BindingFlags.Public | BindingFlags.Instance)
                           orderby prop.Name
                           select prop.Name).ToArray();
          args.MetaType.Add(propNames);
          Console.WriteLine($"  MetaType added (propNames={propNames})");
        }
      };
    }
  }

[freica]

Today I made some further tests (configuring protobuf-net at runtime): RuntimeTypeModel.Default.AutoAddMissingTypes = true; does not solve the problem, perhaps it's the default-setting. Don't know why my type is not automatically added?

If I manually call RuntimeTypeModel.Default.Add(typeof(NestedClass), true); the serialization succeeds.

[mgravell]

That should "just work". I'd need to find a moment to try to repro. I don't have an immediate answer.

[freica]

I've reconsidered your suggestion some steps above:

using any other serializer (of your convenience), but via code-first gRPC

Do you mean with "code-first gRPC" "protobuf-net.Grpc"? Why should I not use the original grpc for this?

[mgravell]

well, I assumed that if you were talking about regular gRPC: you'd be asking over there :)

also, if you're using vanilla gRPC, you need the binding code; normally, that binding code is emitted as part of the marshaller output (via protoc) - so: if you're using a custom serializer, you'll also need to replicate all the binding setup. If you're fine with that: no problem, have fun! but again, that's the kind of thing that code-first gRPC aims to simplifiy.

[freica]

ok, but if I understand this article right, I can have both regular gRPC modules + code first (no protoc), isn't it? In the linked sample I only have to replace the Bond serializer/marshaller by DataContractSerializer So at this moment I don't understand your motivation to invest a lot of effort for your way.

[mgravell]

Well, the linked article doesn't look much like WCF, either. If you want to rewrite all the RPC code manually: fine, no-one is stopping you. What code-first gRPC adds is the bit that makes it operate like WCF does, i.e. interfaces at client and server, with the library worrying about how to make that work.

On Thu, 14 Apr 2022, 07:27 freica, @.***> wrote:

ok, but if I understand this article https://bartoszsypytkowski.com/c-using-grpc-with-custom-serializers/ right, I can have both regular gRPC modules + code first (no protoc), isn't it? In the linked sample https://github.com/Horusiath/GrpcSample/ I only have to replace the Bond serializer/marshaller by DataContractSerializer So at this moment I don't understand your motivation to invest a lot of effort for your way.

— Reply to this email directly, view it on GitHub https://github.com/protobuf-net/protobuf-net.Grpc/issues/226#issuecomment-1098754971, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAEHMEGMFA3JVGGKAUJPUTVE63GPANCNFSM5O6SZIYQ . You are receiving this because you commented.Message ID: @.***>