search

NuGet / NuGetGallery

NuGet Gallery is a package repository that powers https://www.nuget.org. Use this repo for reporting NuGet.org issues.
https://www.nuget.org/
Apache License 2.0
1.54k stars 643 forks source link

NuGetGallery.SearchClient.RetryingHttpClientWrapperFacts.LoadBalancesBetweenValidUrisForGetStringAsync is non-deterministic #6222

Open scottbommarito opened 6 years ago

scottbommarito commented 6 years ago

I had a CI build fail due to the test without making any changes to it: https://nuget.visualstudio.com/NuGetBuild/NuGetBuild%20Team/_build/results?buildId=36147

2018-07-26T22:48:31.0119517Z ##[error]    NuGetGallery.SearchClient.RetryingHttpClientWrapperFacts.LoadBalancesBetweenValidUrisForGetStringAsync [FAIL]
2018-07-26T22:48:31.0119517Z       The second valid Uri has not been hit within the limit of 25 requests.
2018-07-26T22:48:31.0119517Z       Expected: True
2018-07-26T22:48:31.0119517Z       Actual:   False
2018-07-26T22:48:31.0119517Z       Stack Trace:
2018-07-26T22:48:31.0119517Z         tests\NuGetGallery.Facts\SearchClient\RetryingHttpClientWrapperFacts.cs(98,0): at NuGetGallery.SearchClient.RetryingHttpClientWrapperFacts.<LoadBalancesBetweenValidUrisForGetStringAsync>d__12.MoveNext()
2018-07-26T22:48:31.0119517Z         --- End of stack trace from previous location where exception was thrown ---
2018-07-26T22:48:31.0119517Z            at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
2018-07-26T22:48:31.0119517Z            at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
2018-07-26T22:48:31.0119517Z         --- End of stack trace from previous location where exception was thrown ---
2018-07-26T22:48:31.0119517Z            at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
2018-07-26T22:48:31.0119517Z            at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
2018-07-26T22:48:31.0119517Z         --- End of stack trace from previous location where exception was thrown ---
2018-07-26T22:48:31.0119517Z            at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
2018-07-26T22:48:31.0119517Z            at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

It appears that it uses a weighted random comparer to decide between two endpoints, and expects that over 25 requests it will hit the second endpoint at least once. It seems possible, if extremely unlikely, that it could hit the first endpoint with all 25 requests.

skofman1 commented 6 years ago

we should disable this test if it's not deterministic. Please do so if this happens again.