Question: Possibilities of usage grpc Reflection with custom Marshallers

[GoNextGroup]

Hello.

I'm trying to add reflection (for method discovering). I use my own JsonMarshaller (simple, stupid - yes) like this:

    public class JsonMarshallerFactory : MarshallerFactory
    {
        protected static readonly JsonSerializerSettings jsonSettings;

        static JsonMarshallerFactory()
        {
            jsonSettings = BaseJsonSerializerSettings.Instance;
        }

        protected override bool CanSerialize(Type type) => true;

        protected override byte[] Serialize<T>(T value)
        {
            var json = JsonConvert.SerializeObject(value, jsonSettings);

            return Encoding.UTF8.GetBytes(json);
        }
        protected override T Deserialize<T>(byte[] payload)
        {
            var json = Encoding.UTF8.GetString(payload);

            return JsonConvert.DeserializeObject<T>(json, jsonSettings);
        }
    }

but I have problems with discovering through passing cli commands like this: dotnet grpc-cli ls https://localhost:5001 Here I have an exception in Deserialize method.

So, is it possible to use Reflection discovery with custom json marshaller?

Thank you.

[mgravell]

That factory is going to push everything to be serialized as JSON (from the blind "=> true")... but the gRPC discovery API isn't expecting JSON, it is expecting protobuf

So: take this back a level - what are you trying to achieve with the JSON step? what is is that you are trying to serialize as JSON, and when?

[GoNextGroup]

I need to have json marshaller factory for backward compatibility, unfortunatelly. So, now I have just only an idea to check if incoming message is a valid json and use JsonConverter for valid cases and ProtobufMarshallerFactory.Default.Deserialize via reflection for other cases...

[mgravell]

Backward compatibility with what? Because as it stands: you're serializing everything as JSON, so: that'll break things that aren't expecting JSON. Perhaps use an attribute or marker interface to indicate types that you want to use JSON for. Alternatively, if you make sure that the JSON version is last (rather than first)

[GoNextGroup]

I have some services communicated via gRPC. Some of them were written not by me and they need to pass json between calling and caller methods. What do you mean with phrase "Alternatively, if you make sure that the JSON version is last (rather than first)"? This problem could be solved if ProtobufMarshallerFactory had protected constructor...

[GoNextGroup]

Mark, I'll resume this discussion, because I found usage case for json marshaller.

Let's look at this example:

    [ProtoContract, CompatibilityLevel(CompatibilityLevel.Level300)]
    public record EnumSpecification<TSpecification> where TSpecification : struct, Enum
    {
        [ProtoMember(1)]
        public TSpecification Specification { get; init; }
    }

    [AttributeUsage(AttributeTargets.Class)]
    public class ConventionAttribute<TEnum> : Attribute where TEnum : struct, Enum
    {
        public ICollection<Type> Types { get; }
        public ConventionAttribute(params Type[] types )
        {
            Types = types;
        }
    }

    [Convention<PayServiceTypes>(typeof(GetAgentsRequest), typeof(GetSBPAgentsRequest), typeof(GetCardAgentsRequest), typeof(GetQRAgentsRequest))]
    [JsonConverter(typeof(EnumSpecificationConverter<GetAgentsRequest, PayServiceTypes>))]    
    public record GetAgentsRequest : EnumSpecification<PayServiceTypes> { }

    public record GetSBPAgentsRequest : GetAgentsRequest { }
    public record GetCardAgentsRequest : GetAgentsRequest { }
    public record GetQRAgentsRequest : GetAgentsRequest  { }

    public class EnumSpecificationConverter<TRequest, TSpecification> : JsonCreationConverter<TRequest> where TRequest : EnumSpecification<TSpecification>
                                                                                                        where TSpecification : struct, Enum
    {
        protected static readonly string conventionAttributeName = ReflectionExtensions.GetTruncatedTypeName(typeof(ConventionAttribute<>));
        protected readonly bool shouldSerializeDefaults;

        public override void WriteJson(JsonWriter writer, object? value, JsonSerializer serializer) => throw new NotImplementedException();

        public EnumSpecificationConverter() : this(false) { }

        public EnumSpecificationConverter(bool shouldSerializeDefaults)
        {
            this.shouldSerializeDefaults = shouldSerializeDefaults;
        }

        protected override TRequest Create(Type objectType, JObject jObject)
        {
            var attributes = typeof(TRequest).GetCustomAttributes(true);            
            var conventionAttribute = attributes.First(e => ReflectionExtensions.GetTruncatedTypeName(e.GetType())== conventionAttributeName);

            var constructionTypes = conventionAttribute.GetType().GetProperties().ElementAt(0).GetValue(conventionAttribute) as ICollection<Type>;

            var specificationName = nameof(EnumSpecification<TSpecification>.Specification);
            var specification = jObject.Value<string>(specificationName) ?? jObject.Value<string>(specificationName.ToLower());

            if (!Enum.TryParse<TSpecification>(specification, out var connectionSpecification))
            {
                if (!shouldSerializeDefaults)
                {
                    throw new ServiceException(ServiceError.IncorrectSpecification.GetLabel()); //ToDo: change this to universal wrong specification marker
                }

                connectionSpecification = Enum.GetValues<TSpecification>().FirstOrDefault();
            }

            int constructionIndex = EnumOrder<TSpecification>.IndexOfOrLast(connectionSpecification);

            return (TRequest)Activator.CreateInstance(constructionTypes.ElementAt(constructionIndex));
        }
    }

This is primitive realization for polymorphic contracts. So, is it possible to "teach" Default Protobuf Marshaller to serialize/deserialize these contracts properly?

Because I got all time this error:

warn: ProtoBuf.Grpc.Server.ServicesExtensions.CodeFirstServiceMethodProvider[0]
      Type cannot be serialized; ignoring: DTOModel.Wallets.Wallet_Information.Request.Agents.GetAgentsRequest
warn: ProtoBuf.Grpc.Server.ServicesExtensions.CodeFirstServiceMethodProvider[0]
      Signature not recognized for SharedInterfaces.BusinessCore.Base.IGet`2[[DTOModel.Wallets.Wallet_Information.Request.Agents.GetAgentsRequest, DTOModel, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null],[System.Collections.Generic.IEnumerable`1[[DTOModel.Banks.DTO.AgentsDTO, DTOModel, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]], System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].GetAsync; method will not be bound
info: ProtoBuf.Grpc.Server.ServicesExtensions.CodeFirstServiceMethodProvider[0]
      RPC services being provided by ProtoBuf.Grpc.Reflection.ReflectionService: 1

[mgravell]

I'm not sure I understand... is the point here that you want to use JSON? If so, I can show you how to attach a JSON serializer at the top level. I'm not sure what this example is meant to be showing me, to be honest. What is the expected outcome here?

[GoNextGroup]

Marc, I just want to say that it will be great to have an ability to create factory classes derived from your ProtobufMarshallerFactory (currently it's not possible) and customize serialization logic. For example, on incoming json messages I want to use JsonSerializer (because, for example, I can have polymorphic contract DTOs), and Protobuf serializer otherwise. Currently I've solved problem with json serialization usage with "external" method - I've created JsonMarshallerFactory with CanSerialize() method uses attribute for json serialization and pass BinderConfiguration.Create([ProtobufMarshallerFactory.Default, new JsonMarshallerFactory()). But I still want to use just one factory instead. Sorry, my English isn't well, I know.(((