dapr / dotnet-sdk

Dapr SDK for .NET
Apache License 2.0
1.11k stars 336 forks source link

Daprd gRPC proxy initialization failure when using PostSharp Logging #899

Open bcraun opened 2 years ago

bcraun commented 2 years ago

Originally reported on Dapr Discord server. I was working with @Pundil for initial diagnosis. I was asked to open the issue here.

Expected Behavior

Dapr gRPC proxying without PostSharp works correctly. Dapr gRPC proxying with PostSharp fails.

Actual Behavior

Dapr gRPC proxying when using PostSharp Logging fails with daprd proxy initialization failure:

Grpc.Core.RpcException: Status(StatusCode="Unknown", Detail="failed to proxy request: proxy not initialized. daprd startup may be incomplete.", DebugException="Grpc.Core.Internal.CoreErrorDetailException: {"created":"@1656347134.097343000","description":"Error received from peer ipv6:[::1]:50050","file":"/tmpfs/altsrc/github/grpc/workspace_csharp_ext_macos_x64/src/core/lib/surface/call.cc","file_line":904,"grpc_message":"failed to proxy request: proxy not initialized. daprd startup may be incomplete.","grpc_status":2}")

Steps to Reproduce the Problem

I have 3 Dapr services that perform inter-service communication using gRPC proxying in the following way:

ASP.NET Core Controller --> Service1 --> Service2

Failure occurs in Service2 gRPC method when invoked by Service1.

Steps to reproduce:

  1. Add PostSharp Logging Diagnostics package to each service: <PackageReference Include="PostSharp.Patterns.Diagnostics" Version="6.10.13" />

  2. In each service's Program.cs, add the following:

using PostSharp.Patterns.Diagnostics;
using PostSharp.Patterns.Diagnostics.Backends.Console;

LoggingServices.DefaultBackend = new ConsoleLoggingBackend();
  1. Add a file named GlobalAspects.cs to each project containing:

    using PostSharp.Patterns.Diagnostics;
    [assembly: Log]
  2. Compile and debug the app.

  3. In my case, the proxied gRPC call from Service1 to Service2 fails with the following output describing an initialization failure in the gRPC proxy by daprd (service names redacted and replaced with 'xxx'):

         at Xxx.Connectivity.Generated.ConnectivityGrpcService.ConnectivityGrpcServiceClient.GetModuleConnectivityState(Empty request, CallOptions options) in /Users/bcraun/Documents/projects/xxx/xxx/xxx-core/Protos/ConnectivityGrpc.cs:line 130

         at Xxx.BayService.Services.BayServiceGrpc.GetConnectedModulesAsync() in /Users/bcraun/Documents/projects/xxx/xxx/xxx-xxx-service/Services/BayServiceGrpc.cs:line 176

Exception thrown: 'Grpc.Core.RpcException' in xxx-xxx-service.dll
Exception thrown: 'Grpc.Core.RpcException' in System.Private.CoreLib.dll

Release Note

RELEASE NOTE: FIX Bug in runtime.

halspang commented 2 years ago

@bcraun - Thanks for opening this! Can you fix your command for Step 1.? I want to make sure I'm trying to reproduce this correctly.

bcraun commented 2 years ago

@halspang done. Thanks for looking into this issue.

bcraun commented 2 years ago

@halspang were you able to repro?

halspang commented 2 years ago

@bcraun - I haven't had a chance to repro it yet, hoping I can get to it soon.

bcraun commented 1 year ago

@halspang any update on this?