microsoftgraph / msgraph-sdk-dotnet-core

The core Microsoft Graph client library for .Net. (Microsoft.Graph.Core)
Other
150 stars 50 forks source link

MS Graph Request Timeout Querying while processing 20k records for a single tenant #859

Closed techguy0101 closed 3 months ago

techguy0101 commented 4 months ago

Is your feature request related to a problem? Please describe the problem.

When we are processing 213 records for a tenant that time BatchProcessing job sending too many request errors. Added retry mechanism, still received same error and did some research on it but no luck. Please provide the solution Below is the code: //UserId contains 213 contacts while (userId != null) {

                    var batchRequestContent = new BatchRequestContent();
                    Dictionary<string, string> requestUser = new Dictionary<string, string>();

                    for (int i = 0; i < 15 && userId != null; i++)
                    {                          
                        var request = client.Users[userId].Authentication.Methods.Request().Filter(phoneIdFilter);
                        var requestId = batchRequestContent.AddBatchRequestStep(request);
                        requestUser[requestId] = userId;

                        if (!channel.Reader.TryRead(out userId))
                        {
                            userId = null;
                        }
                    }

                   var retryCount = 1;
                    var retryDelay = TimeSpan.FromSeconds(11);
                    var retryAttempts = 0;
                    var returnedResponse = default(BatchResponseContent);
                    while (retryAttempts < retryCount)
                    {

                       returnedResponse = await client.Batch.Request().PostAsync(batchRequestContent);

                       foreach (KeyValuePair<string, HttpResponseMessage> response
                                                in await returnedResponse.GetResponsesAsync())
                        {
                            if (response.Value.StatusCode != HttpStatusCode.OK)
                            {
                                if (response.Value.Headers.RetryAfter != null)
                                {
                                    logger.Information($"Error occurred {response.Value.Headers.RetryAfter}");
                                }
                                logger.Information($"Error occurred count {Count}");
                                await Task.Delay(retryDelay);
                                retryAttempts++;
                            }                              
                        }
                  }

Describe the solution you'd like.

Would like to process all the records without any error. if you have the solution for this, please share. Thank you in advance

Additional context?

No response

andrueastman commented 3 months ago

Thanks for raising this @techguy0101

Any chance you can confirm the version of the SDK you are using for this? Out of curiousity, what is the Http status code received for the responses. Do you get a value for the retryAfter header?

microsoft-github-policy-service[bot] commented 3 months ago

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment.