dapr / test-infra

Test apps and tools for Dapr
Apache License 2.0
14 stars 24 forks source link

Test applications should `wait()` for SDK to be ready #182

Closed tmacam closed 1 year ago

tmacam commented 1 year ago

Dapr application running in k8s might have failures during initialization if the sidecar is not up by the time they start up. See dapr/dapr#1933

Test applications should wait for the SDK to be ready before initializing to avoid this sort of error and avoid producing error metrics or failures in app start up.

Steps to Reproduce the Problem

This is indeterministic (it depends on how fast the app loads and now fast the sidecar is started up by k8s) but a sample error from feed-generator application:

 Caught Dapr.DaprException: Publish operation failed: the Dapr endpoint indicated a failure. See InnerException for details.                                                                                                                                                                                                                      
  ---> Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error connecting to subchannel.", DebugException="System.Net.Sockets.SocketException (111): Connection refused                                                                                                                                                            
    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)                                                                                                                                                                                                             
    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)                                                                                                                                                                                                             
    at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|281_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)                                                                                                                                                            
    at Grpc.Net.Client.Balancer.Internal.SocketConnectivitySubchannelTransport.TryConnectAsync(CancellationToken cancellationToken)")                                                                                                                                                                                                             
    at Grpc.Net.Client.Balancer.Internal.ConnectionManager.PickAsync(PickContext context, Boolean waitForReady, CancellationToken cancellationToken)                                                                                                                                                                                              
    at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)                                                                                                                                                                                                           
    at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout)                                                                                                                                                                                                                                                
    at Dapr.Client.DaprClientGrpc.MakePublishRequest(String pubsubName, String topicName, ByteString content, Dictionary`2 metadata, String dataContentType, CancellationToken cancellationToken)                                                                                                                                                 
    --- End of inner exception stack trace ---                                                                                                                                                                                                                                                                                                    
    at Dapr.Client.DaprClientGrpc.MakePublishRequest(String pubsubName, String topicName, ByteString content, Dictionary`2 metadata, String dataContentType, CancellationToken cancellationToken)                                                                                                                                                 
    at FeedGenerator.Program.StartMessageGeneratorAsync(Int32 delayInMilliseconds) in /app/feed-generator/Program.cs:line 86     

Expected Behavior

No errors during initialization

Actual Behavior

Potentially? Errors :)