search

protobuf-net / protobuf-net.Grpc

GRPC bindings for protobuf-net and grpc-dotnet
Other
857 stars 109 forks source link

Integer overflow for uint #316

Closed JonathanBout closed 10 months ago

JonathanBout commented 10 months ago

In my hosted blazor WASM system I have a uint-derived enum. When serialized, this causes problems as it apparently gets sent as a signed int32.

The exception I get is this: System.OverflowException: Value was either too large or too small for an Int32., the full stack trace is at the end of this issue.

I found a workaround by using BitConverter in a private property like this:

public MyEnum Property { get; set; }

[ProtoMember(1)]
private byte[] MyEnumBytes
{
    get => BitConverter.GetBytes((uint)Property);
    set => Property = (MyEnum)BitConverter.ToUInt32(value);
}
Full stack trace 
crit: Microsoft.AspNetCore.Components.WebAssembly.Rendering.WebAssemblyRenderer[100]  
      Unhandled exception rendering component: Status(StatusCode="Internal", Detail="Error starting gRPC call. OverflowException: Value was either too large or too small for an Int32.", DebugException="System.OverflowException: Value was either too large or too small for an Int32.  
         at System.Convert.ThrowInt32OverflowException()  
         at System.Convert.ToInt32(UInt32 value)  
         at System.UInt32.System.IConvertible.ToInt32(IFormatProvider provider)  
         at System.Convert.ToInt32(Object value)  
         at System.Enum.System.IConvertible.ToInt32(IFormatProvider provider)  
         at System.Convert.ToInt32(Object value)  
         at ProtoBuf.Meta.MetaType.NormalizeProtoMember(TypeModel model, MemberInfo member, AttributeFamily family, Boolean forced, Boolean isEnum, BasicList partialMembers, Int32 dataMemberOffset, Boolean inferByTagName, Boolean& hasConflictingEnumValue, MemberInfo backingMember) in C:\Code\protobuf-net\src\protobuf-net\Meta\MetaType.cs:line 981  
         at ProtoBuf.Meta.MetaType.ApplyDefaultBehaviour_AddMembers(TypeModel model, AttributeFamily family, Boolean isEnum, BasicList partialMembers, Int32 dataMemberOffset, Boolean inferTagByName, ImplicitFields implicitMode, BasicList members, MemberInfo member, Boolean& forced, Boolean isPublic, Boolean isField, Type& effectiveType, Boolean& hasConflictingEnumValue, MemberInfo backingMember) in C:\Code\protobuf-net\src\protobuf-net\Meta\MetaType.cs:line 793  
         at ProtoBuf.Meta.MetaType.ApplyDefaultBehaviourImpl() in C:\Code\protobuf-net\src\protobuf-net\Meta\MetaType.cs:line 716  
         at ProtoBuf.Meta.MetaType.ApplyDefaultBehaviour() in C:\Code\protobuf-net\src\protobuf-net\Meta\MetaType.cs:line 518  
         at ProtoBuf.Meta.RuntimeTypeModel.FindOrAddAuto(Type type, Boolean demand, Boolean addWithContractOnly, Boolean addEvenIfAutoDisabled) in C:\Code\protobuf-net\src\protobuf-net\Meta\RuntimeTypeModel.cs:line 598  
         at ProtoBuf.Meta.RuntimeTypeModel.GetEnumMap(Type type) in C:\Code\protobuf-net\src\protobuf-net\Meta\RuntimeTypeModel.cs:line 1731  
         at ProtoBuf.Meta.ValueMember.TryGetCoreSerializer(RuntimeTypeModel model, DataFormat dataFormat, Type type, WireType& defaultWireType, Boolean asReference, Boolean dynamicType, Boolean overwriteList, Boolean allowComplexTypes) in C:\Code\protobuf-net\src\protobuf-net\Meta\ValueMember.cs:line 641  
         at ProtoBuf.Meta.ValueMember.BuildSerializer() in C:\Code\protobuf-net\src\protobuf-net\Meta\ValueMember.cs:line 520  
         at ProtoBuf.Meta.ValueMember.get_Serializer() in C:\Code\protobuf-net\src\protobuf-net\Meta\ValueMember.cs:line 213  
         at ProtoBuf.Meta.MetaType.BuildSerializer() in C:\Code\protobuf-net\src\protobuf-net\Meta\MetaType.cs:line 459  
         at ProtoBuf.Meta.MetaType.get_Serializer() in C:\Code\protobuf-net\src\protobuf-net\Meta\MetaType.cs:line 375  
         at ProtoBuf.Meta.RuntimeTypeModel.Serialize(Int32 key, Object value, ProtoWriter dest) in C:\Code\protobuf-net\src\protobuf-net\Meta\RuntimeTypeModel.cs:line 803  
         at ProtoBuf.Meta.TypeModel.SerializeCore(ProtoWriter writer, Object value) in C:\Code\protobuf-net\src\protobuf-net\Meta\TypeModel.cs:line 192  
         at ProtoBuf.Meta.TypeModel.Serialize(Stream dest, Object value, SerializationContext context) in C:\Code\protobuf-net\src\protobuf-net\Meta\TypeModel.cs:line 221  
         at ProtoBuf.Grpc.Configuration.ProtoBufMarshallerFactory.Serialize[RoleSearchParameters](RoleSearchParameters value) in /_/src/protobuf-net.Grpc/Configuration/ProtoBufMarshallerFactory.cs:line 219  
         at ProtoBuf.Grpc.Configuration.ProtoBufMarshallerFactory.ContextualSerialize[RoleSearchParameters](RoleSearchParameters value, SerializationContext context) in /_/src/protobuf-net.Grpc/Configuration/ProtoBufMarshallerFactory.cs:line 151  
         at Grpc.Net.Client.StreamExtensions.d__9`1[[MyProject.Shared.GrpcParameters.RoleSearchParameters, MyProject.Shared, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]].MoveNext()  
         at Grpc.Net.Client.Internal.PushUnaryContent`2.d__4[[MyProject.Shared.GrpcParameters.RoleSearchParameters, MyProject.Shared, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null],[System.Collections.Generic.IEnumerable`1[[MyProject.Shared.GrpcResults.GrpcRole, MyProject.Shared, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]], System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()  
         at System.Net.Http.HttpContent.g__WaitAsync|56_0(ValueTask copyTask)  
         at System.Net.Http.HttpContent.LoadIntoBufferAsyncCore(Task serializeToStreamTask, MemoryStream tempBuffer)  
         at System.Net.Http.HttpContent.d__82`2[[System.Net.Http.HttpContent, System.Net.Http, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a],[System.Byte[], System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()  
         at System.Net.Http.BrowserHttpHandler.CallFetch(HttpRequestMessage request, CancellationToken cancellationToken, Nullable`1 allowAutoRedirect)  
         at System.Net.Http.BrowserHttpHandler.g__Impl|55_0(HttpRequestMessage request, CancellationToken cancellationToken, Nullable`1 allowAutoRedirect)  
         at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)  
         at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)  
         at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.g__Core|5_0(HttpRequestMessage request, CancellationToken cancellationToken)  
         at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.g__Core|5_0(HttpRequestMessage request, CancellationToken cancellationToken)  
         at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)  
         at Grpc.Net.Client.Internal.GrpcCall`2.d__73[[MyProject.Shared.GrpcParameters.RoleSearchParameters, MyProject.Shared, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null],[System.Collections.Generic.IEnumerable`1[[MyProject.Shared.GrpcResults.GrpcRole, MyProject.Shared, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]], System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()")  
Grpc.Core.RpcException: Status(StatusCode="Internal", Detail="Error starting gRPC call. OverflowException: Value was either too large or too small for an Int32.", DebugException="System.OverflowException: Value was either too large or too small for an Int32.  
   at System.Convert.ThrowInt32OverflowException()  
   at System.Convert.ToInt32(UInt32 value)  
   at System.UInt32.System.IConvertible.ToInt32(IFormatProvider provider)  
   at System.Convert.ToInt32(Object value)  
   at System.Enum.System.IConvertible.ToInt32(IFormatProvider provider)  
   at System.Convert.ToInt32(Object value)  
   at ProtoBuf.Meta.MetaType.NormalizeProtoMember(TypeModel model, MemberInfo member, AttributeFamily family, Boolean forced, Boolean isEnum, BasicList partialMembers, Int32 dataMemberOffset, Boolean inferByTagName, Boolean& hasConflictingEnumValue, MemberInfo backingMember) in C:\Code\protobuf-net\src\protobuf-net\Meta\MetaType.cs:line 981  
   at ProtoBuf.Meta.MetaType.ApplyDefaultBehaviour_AddMembers(TypeModel model, AttributeFamily family, Boolean isEnum, BasicList partialMembers, Int32 dataMemberOffset, Boolean inferTagByName, ImplicitFields implicitMode, BasicList members, MemberInfo member, Boolean& forced, Boolean isPublic, Boolean isField, Type& effectiveType, Boolean& hasConflictingEnumValue, MemberInfo backingMember) in C:\Code\protobuf-net\src\protobuf-net\Meta\MetaType.cs:line 793  
   at ProtoBuf.Meta.MetaType.ApplyDefaultBehaviourImpl() in C:\Code\protobuf-net\src\protobuf-net\Meta\MetaType.cs:line 716  
   at ProtoBuf.Meta.MetaType.ApplyDefaultBehaviour() in C:\Code\protobuf-net\src\protobuf-net\Meta\MetaType.cs:line 518  
   at ProtoBuf.Meta.RuntimeTypeModel.FindOrAddAuto(Type type, Boolean demand, Boolean addWithContractOnly, Boolean addEvenIfAutoDisabled) in C:\Code\protobuf-net\src\protobuf-net\Meta\RuntimeTypeModel.cs:line 598  
   at ProtoBuf.Meta.RuntimeTypeModel.GetEnumMap(Type type) in C:\Code\protobuf-net\src\protobuf-net\Meta\RuntimeTypeModel.cs:line 1731  
   at ProtoBuf.Meta.ValueMember.TryGetCoreSerializer(RuntimeTypeModel model, DataFormat dataFormat, Type type, WireType& defaultWireType, Boolean asReference, Boolean dynamicType, Boolean overwriteList, Boolean allowComplexTypes) in C:\Code\protobuf-net\src\protobuf-net\Meta\ValueMember.cs:line 641  
   at ProtoBuf.Meta.ValueMember.BuildSerializer() in C:\Code\protobuf-net\src\protobuf-net\Meta\ValueMember.cs:line 520  
   at ProtoBuf.Meta.ValueMember.get_Serializer() in C:\Code\protobuf-net\src\protobuf-net\Meta\ValueMember.cs:line 213  
   at ProtoBuf.Meta.MetaType.BuildSerializer() in C:\Code\protobuf-net\src\protobuf-net\Meta\MetaType.cs:line 459  
   at ProtoBuf.Meta.MetaType.get_Serializer() in C:\Code\protobuf-net\src\protobuf-net\Meta\MetaType.cs:line 375  
   at ProtoBuf.Meta.RuntimeTypeModel.Serialize(Int32 key, Object value, ProtoWriter dest) in C:\Code\protobuf-net\src\protobuf-net\Meta\RuntimeTypeModel.cs:line 803  
   at ProtoBuf.Meta.TypeModel.SerializeCore(ProtoWriter writer, Object value) in C:\Code\protobuf-net\src\protobuf-net\Meta\TypeModel.cs:line 192  
   at ProtoBuf.Meta.TypeModel.Serialize(Stream dest, Object value, SerializationContext context) in C:\Code\protobuf-net\src\protobuf-net\Meta\TypeModel.cs:line 221  
   at ProtoBuf.Grpc.Configuration.ProtoBufMarshallerFactory.Serialize[RoleSearchParameters](RoleSearchParameters value) in /_/src/protobuf-net.Grpc/Configuration/ProtoBufMarshallerFactory.cs:line 219  
   at ProtoBuf.Grpc.Configuration.ProtoBufMarshallerFactory.ContextualSerialize[RoleSearchParameters](RoleSearchParameters value, SerializationContext context) in /_/src/protobuf-net.Grpc/Configuration/ProtoBufMarshallerFactory.cs:line 151  
   at Grpc.Net.Client.StreamExtensions.d__9`1[[MyProject.Shared.GrpcParameters.RoleSearchParameters, MyProject.Shared, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]].MoveNext()  
   at Grpc.Net.Client.Internal.PushUnaryContent`2.d__4[[MyProject.Shared.GrpcParameters.RoleSearchParameters, MyProject.Shared, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null],[System.Collections.Generic.IEnumerable`1[[MyProject.Shared.GrpcResults.GrpcRole, MyProject.Shared, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]], System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()  
   at System.Net.Http.HttpContent.g__WaitAsync|56_0(ValueTask copyTask)  
   at System.Net.Http.HttpContent.LoadIntoBufferAsyncCore(Task serializeToStreamTask, MemoryStream tempBuffer)  
   at System.Net.Http.HttpContent.d__82`2[[System.Net.Http.HttpContent, System.Net.Http, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a],[System.Byte[], System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()  
   at System.Net.Http.BrowserHttpHandler.CallFetch(HttpRequestMessage request, CancellationToken cancellationToken, Nullable`1 allowAutoRedirect)  
   at System.Net.Http.BrowserHttpHandler.g__Impl|55_0(HttpRequestMessage request, CancellationToken cancellationToken, Nullable`1 allowAutoRedirect)  
   at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)  
   at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)  
   at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.g__Core|5_0(HttpRequestMessage request, CancellationToken cancellationToken)  
   at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.g__Core|5_0(HttpRequestMessage request, CancellationToken cancellationToken)  
   at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)  
   at Grpc.Net.Client.Internal.GrpcCall`2.d__73[[MyProject.Shared.GrpcParameters.RoleSearchParameters, MyProject.Shared, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null],[System.Collections.Generic.IEnumerable`1[[MyProject.Shared.GrpcResults.GrpcRole, MyProject.Shared, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]], System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()")  
   at ProtoBuf.Grpc.Internal.Reshape.d__16`2[[MyProject.Shared.GrpcParameters.RoleSearchParameters, MyProject.Shared, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null],[System.Collections.Generic.IEnumerable`1[[MyProject.Shared.GrpcResults.GrpcRole, MyProject.Shared, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]], System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext() in /_/src/protobuf-net.Grpc/Internal/Reshape.cs:line 554  
   at MyProject.Client.Pages.Index.OnInitializedAsync() in D:\Projects\GitHub\MyProject\MyProject\Client\Pages\Index.razor:line 55  
   at Microsoft.AspNetCore.Components.ComponentBase.RunInitAndSetParametersAsync()
mgravell commented 10 months ago

Please don't do that (BitConverter) - apart from being ridiculously inefficient, that is also CPU-endian, meaning: you're going to have a bad time on some CPUs

Can I see your MyEnum? I want to put together a full repro

mgravell commented 10 months ago

Also: I think this is already resolved in protobuf-net v3; basically, protobuf-net v2 supported "mapped" enums, where-by the value that was serialized was not always the same as the intrinsic value; a "passthru" mode existed, but you needed to configure that manually - it was a pain and not very useful, and it conflicted with some later proto3 semantics

in protobuf-net v3, "mapped" enums are deprecated, and everything is "passthru" by default; so I believe if you simply update to protobuf-net v3, it will "just work"

Note that protobuf-net and protobuf-net.Grpc are separate packages; protobuf-net.Grpc suggests v2, but if you add an explicit package reference to protobuf-net, you can use v3 (and it can exploit a lot of newer features in v3)

JonathanBout commented 10 months ago

It's a very simple user role enum:

    public enum UserRights : uint
    {
        None = 0,
        Minimal = 1,
        ManageRoles = 2,
        Administrator = ~None
    }

Which I am obviously going to expand it in the future

mgravell commented 10 months ago

Can you try with protobuf-net v3?

if that isn't an option, let me know (and I'll show you how to configure "passthru")

JonathanBout commented 10 months ago

I'll be able to take a look at it in a few hours

JonathanBout commented 10 months ago

Yeah this seems to work, thanks!