search

newrelic / newrelic-dotnet-agent

The New Relic .NET language agent.
Apache License 2.0
96 stars 57 forks source link

Integration tests fail after upgrading to XUnit 2.9.0 #2684

Closed tippmar-nr closed 1 month ago

tippmar-nr commented 1 month ago

Description Several integration tests fail consistently after upgrading to XUnit 2.9.0, with errors similar to the following: 

  Error Message:
   System.InvalidOperationException : There is no currently active test.
  Stack Trace:
     at NewRelic.Agent.IntegrationTestHelpers.AgentLogBase.WaitForLogLines(String regularExpression, Nullable`1 timeoutOrZero, Int32 minimumCount) in D:\a\newrelic-dotnet-agent\newrelic-dotnet-agent\tests\Agent\IntegrationTests\IntegrationTestHelpers\AgentLogBase.cs:line 169
   at NewRelic.Agent.IntegrationTestHelpers.AgentLogBase.GetReportingAppLink(Nullable`1 timeoutOrZero) in D:\a\newrelic-dotnet-agent\newrelic-dotnet-agent\tests\Agent\IntegrationTests\IntegrationTestHelpers\AgentLogBase.cs:line 118
   at NewRelic.Agent.IntegrationTestHelpers.AgentLogBase.GetAccountId(Nullable`1 timeoutOrZero) in D:\a\newrelic-dotnet-agent\newrelic-dotnet-agent\tests\Agent\IntegrationTests\IntegrationTestHelpers\AgentLogBase.cs:line 93
   at NewRelic.Agent.IntegrationTests.WCF.Client.WCFClientTestBase.DistributedTracing_Metrics() in D:\a\newrelic-dotnet-agent\newrelic-dotnet-agent\tests\Agent\IntegrationTests\IntegrationTests\WCF\WCFClientTests.cs:line 130

The XUnit release notes for v2.9.0 indicate that they fixed a bug related to re-using instances of TestOutputHelper between tests. Prior to the fix, reusing an instance would (incorrectly) mix log output between tests; after the fix, the InvalidOperationException shown above is thrown.

The errors occur primarily in the DistributedTracing and WCF integration test namespaces, indicating that we need to review the implementation of those tests and fix any reuse of TestOutputHelper instances between tests. This comment on the XUnit issue may be helpful in our analysis.

Expected Behavior All integration tests should pass after upgrading to XUnit v2.9.0

For Maintainers Only or Hero Triaging this bug Suggested Priority (P1,P2,P3,P4,P5): P3 Suggested T-Shirt size (S, M, L, XL, Unknown): M

workato-integration[bot] commented 1 month ago

https://new-relic.atlassian.net/browse/NR-301266