ExecuteWaitAsync command is randomly timing out for ey-tim integration tests.

[miazamrai]

Ey-tim integration tests are randomly failing due to SDK's ExecuteWaitAsync function timing out on the HTTP client. Tried to trace down the actual transaction logs in datadog to see if the transaction was taking long time. But, the transaction was created in a few seconds.

Steps to reproduce: Check with @miazamrai on how to run run ey-tim tests to reproduce the issue. Following stack trace shows the error. EY.TaxCalcEngine.RaiCloud.Tests.AffectedJobsTests.AffectedJobsByEventTests.ScenarioTest(scenarioName: "FormMappingEventScenario") [FAIL] Failed EY.TaxCalcEngine.RaiCloud.Tests.AffectedJobsTests.AffectedJobsByEventTests.ScenarioTest(scenarioName: "FormMappingEventScenario") [1 m 14 s] Error Message: System.Threading.Tasks.TaskCanceledException : The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing. ---- System.TimeoutException : The operation was canceled. -------- System.Threading.Tasks.TaskCanceledException : The operation was canceled. ------------ System.IO.IOException : Unable to read data from the transport connection: Operation canceled. ---------------- System.Net.Sockets.SocketException : Operation canceled Stack Trace: at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts) at System.Net.Http.HttpClient.g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) at RelationalAI.Services.Rest.RequestHelperAsync(String method, String url, Object data, Dictionary2 headers, Dictionary2 parameters) at RelationalAI.Services.Rest.RequestAsync(String method, String url, Object data, Dictionary2 headers, Dictionary2 parameters) at RelationalAI.Services.Client.ExecuteAsync(String database, String engine, String source, Boolean readOnly, Dictionary2 inputs) at RelationalAI.Services.Client.ExecuteWaitAsync(String database, String engine, String source, Boolean readOnly, Dictionary2 inputs) at EY.TaxCalcEngine.RaiCloud.RaiService.SubmitTransactionAsync(String sourceCommand, Guid eyClientId, Dictionary2 inputs) in /home/runner/work/raicode/raicode/ey-tim-kg-broker/src/EY.TaxCalcEngine.RaiCloud/RaiService.cs:line 284 at EY.TaxCalcEngine.RaiCloud.RaiService.SubmitTransactionCheckCompletedAsync(String sourceCommand, Guid eyClientId, Dictionary2 inputs) in /home/runner/work/raicode/raicode/ey-tim-kg-broker/src/EY.TaxCalcEngine.RaiCloud/RaiService.cs:line 304 ----- Inner Stack Trace ----- ----- Inner Stack Trace ----- at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpClient.g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) ----- Inner Stack Trace ----- at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](TIOAdapter adapter) at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](TIOAdapter adapter, Memory`1 buffer) at System.Net.Http.HttpConnection.InitialFillAsync(Boolean async) at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) ----- Inner Stack Trace ----- Standard Output Messages: Relations update time: 55670 ms Calculations export time: 18172 ms Failed! - Failed: 1, Passed: 22, Skipped: 1, Total: 24, Duration: 32 m 3 s - /home/runner/work/raicode/raicode/ey-tim-kg-broker/tests/integrationTests/EY.TaxCalcEngine.RaiCloud.Tests/bin/Debug/net6.0/EY.TaxCalcEngine.RaiCloud.Tests.dll (net6.0)

[billscheidel-rai]

Note: This issue has been migrated to https://relationalai.atlassian.net/browse/RAI-6983.

This link is only accessible to employees of RelationalAI.

[miazamrai]

This is an old issue which had already been fixed.