RyanThomas73
commented
5 months ago We've been receiving these QuicException: The connection timed out from inactivity. a lot recently at my day job. We've got a lot of different components but the most common ones this had been occurring for are targetting .NET 8 and using the 19.0.1 version of the google-ads-dotnet nuget package.
It appears to happen consistently at about 1 minute into the request and I believe some changes are needed so we can configure a longer connection idle timeout.
@AnashOommen - Tagging you since you appear to be the previous committer for the code in question that I believe should be updated:
Specifically - The CreateGrpNetClientChannel logic here needs to be switched to using a SocketsHttpHandler and needs to allow consumers a way to pass in a TimeSpan value for the socketsHttpHandler.PooledConnectionIdleTimeout property.
This is necessary because:
-
The HttpClientHandler class does not provide any mechanisms to specify the
PooledConnectionIdleTimeouton theSocketsHttpHandler _underlyingHandlerthat it wraps. This causes the wrapped SocketsHttpHandler to always use the default PooledConnectionIdleTimeout which is 1 minutes on .NET 6 and greater (2 minutes on .NET 5 and lower .NET Core versions). -
The Grpc.Net.Client.GrpcChannel object has it's own ConnectionIdleTimeout property which it will populate off of the SocketsHttpHandler.PooledConnectionIdleTimeout when using that handler type. When using the HttpClientHandler however the GrpcChannel.ConnectionIdleTimeout property is left null which will also result in falling back to a default of 1 minute
Raibaz
AnashOommen
Describe the bug:
Error when testing a Google call on the CI / CD server. I cannot go further if a simple test do not pass. It was perfectly working using useGRPCcore = true. Once i removed the useGrpcCore=true, then i am supposed to use http2, based on the error, it looks like it use Http3. When i do the same test on my local machine using Windows 11, i dont have any issue.
Steps to Reproduce: var bidOverrideRestoreEntities = criterions .Where(crit => crit.CriterionBidType == BiddingBehaviorType.BidOverride) .Select(crit => crit as AdGroupBidOverrideCriterion) .Select(bidOverrides => new AdGroupCriterion { ResourceName = ResourceNames.AdGroupCriterion(googleAccountId, bidOverrides.ProviderEntityId, bidOverrides.ProviderCriterionId), CpcBidMicros = (long)(bidOverrides.Value * (decimal)MicroAmountFactor) }); var bidOverrideRestoreOperations = bidOverrideRestoreEntities.Select(x => new AdGroupCriterionOperation { Update = x, UpdateMask = FieldMasks.AllSetFieldsOf(x) });
foreach (var operationBatch in bidOverrideRestoreOperations.InSetsOf(2000)) { var operationArray = operationBatch.ToArray(); var bidOverrideRequest = new MutateAdGroupCriteriaRequest { CustomerId = googleAccountId.ToString() }; bidOverrideRequest.Operations.AddRange(operationArray); service.MutateAdGroupCriteria(bidOverrideRequest); }
Expected behavior:
No error and the bid modifier applied to the adgroup
Client library version and API version: Client library version: 18 Google Ads API version: 15 .NET version: .Net 7 Operating system (Linux, Windows, ...) and version (if the bug is platform-specific): Windows Server 2022 Standard 20348.2227
Request/Response Logs: GoogleAds.DetailedRequestLogs Information: 1 : [2024-02-21 21:59:40Z] - ---------------BEGIN API CALL---------------
Request
Method Name: /google.ads.googleads.v15.services.AdGroupCriterionService/MutateAdGroupCriteria Host: Headers: { "x-goog-api-client": "gl-dotnet/2.0.0 gapic/18.0.0 gax/4.2.0+a8085e4f36ad24e2747b5e550f11079d4a891e78 grpc/2.46.3 gccl/3.1.0 pb/3.21.5+638779f353731a0a04496bde20d14164684c3d93", "developer-token": "REDACTED", "login-customer-id": "XXXXXXX", "x-goog-request-params": "customer_id=XXXXXXX" }
{ "customerId": "XXXXXXX", "operations": [ { "update": { "resourceName": "customers/XXXXXXXXX/adGroupCriteria/120214107244~10", "bidModifier": 1.1 }, "updateMask": "resourceName,bidModifier" } ] }
Response
Headers: {}
Fault: Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: The connection timed out from inactivity. (googleads.googleapis.com:443) QuicException: The connection timed out from inactivity.", DebugException="System.Net.Http.HttpRequestException: The connection timed out from inactivity. (googleads.googleapis.com:443) ---> System.Net.Quic.QuicException: The connection timed out from inactivity. at System.Net.Quic.QuicConnection.HandleEventShutdownInitiatedByTransport(_SHUTDOWN_INITIATED_BY_TRANSPORT_e__Struct& data) at System.Net.Quic.QuicConnection.HandleConnectionEvent(QUIC_CONNECTION_EVENT& connectionEvent) at System.Net.Quic.QuicConnection.NativeCallback(QUIC_HANDLE connection, Void context, QUIC_CONNECTIONEVENT* connectionEvent) --- End of stack trace from previous location --- at System.Net.Quic.ValueTaskSource.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Quic.QuicConnection.FinishConnectAsync(QuicClientConnectionOptions options, CancellationToken cancellationToken) at System.Net.Quic.QuicConnection.ConnectAsync(QuicClientConnectionOptions options, CancellationToken cancellationToken) at System.Net.Quic.QuicConnection.ConnectAsync(QuicClientConnectionOptions options, CancellationToken cancellationToken) at System.Net.Http.ConnectHelper.ConnectQuicAsync(HttpRequestMessage request, DnsEndPoint endPoint, TimeSpan idleTimeout, SslClientAuthenticationOptions clientAuthenticationOptions, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.ConnectHelper.ConnectQuicAsync(HttpRequestMessage request, DnsEndPoint endPoint, TimeSpan idleTimeout, SslClientAuthenticationOptions clientAuthenticationOptions, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.GetHttp3ConnectionAsync(HttpRequestMessage request, HttpAuthority authority, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.TrySendUsingHttp3Async(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) in //src/Grpc.Net.Client/Balancer/Internal/BalancerHttpHandler.cs:line 114 at Grpc.Net.Client.Internal.GrpcCall
2.RunCall(HttpRequestMessage request, Nullable1 timeout) in //src/Grpc.Net.Client/Internal/GrpcCall.cs:line 481") at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync() in //src/Grpc.Net.Client/Internal/GrpcCall.cs:line 300 at Google.Ads.GoogleAds.Logging.LoggingHandler.HandleAsyncUnaryLogging[TRequest,TResponse](TRequest request, ClientInterceptorContext2 context, Task1 oldTask, AsyncUnaryCall`1 call) ----------------END API CALL----------------