OutOfMemory Exception in Az devops since updated to latest nunitxml.testlogger

[mregen]

Describe the bug

After bumping to 4.0.254, https://github.com/OPCFoundation/UA-.NETStandard/pull/2691, the ci test for opc.Ua.Core.Tests is running out of memory before the testing finishes. Appears to be only with net48:

https://opcfoundation.visualstudio.com/opcua-netstandard/_test/analytics?definitionId=14&contextType=build

It is yet unclear how this can happen , because the failing test project has no dependency to this lib, another test project has. Tested that downgrading the lib in the other project fixes the issue.

Is the log file becoming too big here? Should the xml logger be removed?

Exception System.OutOfMemoryException,    Exception thrown executing tests in D:\a\1\s\Tests\Opc.Ua.Core.Tests\bin\Release\net48\Opc.Ua.Core.Tests.exe
Exception of type 'System.OutOfMemoryException' was thrown.
   at System.Text.StringBuilder.ToString()
   at System.IO.StringWriter.ToString()
   at System.Xml.XmlNode.get_OuterXml()
   at NUnit.Engine.Runners.MasterTestRunner.RunTests(ITestEventListener listener, TestFilter filter)
   at NUnit.Engine.Runners.MasterTestRunner.Run(ITestEventListener listener, TestFilter filter)
   at NUnit.VisualStudio.TestAdapter.NUnitEngine.NUnitEngineAdapter.Run(ITestEventListener listener, TestFilter filter) in C:\repos\nunit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnitEngine\NUnitEngineAdapter.cs:line 101
   at NUnit.VisualStudio.TestAdapter.Execution.Run(TestFilter filter, DiscoveryConverter discovery, NUnit3TestExecutor nUnit3TestExecutor) in C:\repos\nunit\nunit3-vs-adapter\src\NUnitTestAdapter\Execution.cs:line 42
   at NUnit.VisualStudio.TestAdapter.VsTestExecution.Run(TestFilter filter, DiscoveryConverter discovery, NUnit3TestExecutor nUnit3TestExecutor) in C:\repos\nunit\nunit3-vs-adapter\src\NUnitTestAdapter\Execution.cs:line 136
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunAssembly(String assemblyPath, IGrouping`2 testCases, TestFilter filter, String assemblyName) in C:\repos\nunit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 300
  Failed D:\a\1\s\Tests\Opc.Ua.Core.Tests\bin\Release\net48\Opc.Ua.Core.Tests.exe
  Error Message:
   System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.Text.StringBuilder.ToString()
   at System.IO.StringWriter.ToString()
   at System.Xml.XmlNode.get_OuterXml()
   at NUnit.Engine.Runners.MasterTestRunner.RunTests(ITestEventListener listener, TestFilter filter)
   at NUnit.Engine.Runners.MasterTestRunner.Run(ITestEventListener listener, TestFilter filter)
   at NUnit.VisualStudio.TestAdapter.NUnitEngine.NUnitEngineAdapter.Run(ITestEventListener listener, TestFilter filter) in C:\repos\nunit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnitEngine\NUnitEngineAdapter.cs:line 101
   at NUnit.VisualStudio.TestAdapter.Execution.Run(TestFilter filter, DiscoveryConverter discovery, NUnit3TestExecutor nUnit3TestExecutor) in C:\repos\nunit\nunit3-vs-adapter\src\NUnitTestAdapter\Execution.cs:line 42
   at NUnit.VisualStudio.TestAdapter.VsTestExecution.Run(TestFilter filter, DiscoveryConverter discovery, NUnit3TestExecutor nUnit3TestExecutor) in C:\repos\nunit\nunit3-vs-adapter\src\NUnitTestAdapter\Execution.cs:line 136
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunAssembly(String assemblyPath, IGrouping`2 testCases, TestFilter filter, String assemblyName) in C:\repos\nunit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 300
  Stack Trace:
     at System.Text.StringBuilder.ToString()
   at System.IO.StringWriter.ToString()
   at System.Xml.XmlNode.get_OuterXml()
   at NUnit.Engine.Runners.MasterTestRunner.RunTests(ITestEventListener listener, TestFilter filter)
   at NUnit.Engine.Runners.MasterTestRunner.Run(ITestEventListener listener, TestFilter filter)
   at NUnit.VisualStudio.TestAdapter.NUnitEngine.NUnitEngineAdapter.Run(ITestEventListener listener, TestFilter filter) in C:\repos\nunit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnitEngine\NUnitEngineAdapter.cs:line 101
   at NUnit.VisualStudio.TestAdapter.Execution.Run(TestFilter filter, DiscoveryConverter discovery, NUnit3TestExecutor nUnit3TestExecutor) in C:\repos\nunit\nunit3-vs-adapter\src\NUnitTestAdapter\Execution.cs:line 42
   at NUnit.VisualStudio.TestAdapter.VsTestExecution.Run(TestFilter filter, DiscoveryConverter discovery, NUnit3TestExecutor nUnit3TestExecutor) in C:\repos\nunit\nunit3-vs-adapter\src\NUnitTestAdapter\Execution.cs:line 136
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunAssembly(String assemblyPath, IGrouping`2 testCases, TestFilter filter, String assemblyName) in C:\repos\nunit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 300
Results File: D:\a\_temp\VssAdministrator_fv-az863-229_2024-08-05_15_03_25.trx

• Test Framework (choose one): Nunit
• .NET project version, sdk version (choose applicable): .NET Framework 4.8, 4.7.2 etc
• Command line used (please share the test run command): dotnet test --logger:xyz...
• OS: Windows

Expected behavior

Tests run and pass without Outofmemoryexception

Other details

• [ ] Here's a minimal repro project
• [ ] I have attached logs from dotnet test --diag:log.txt to the issue. Please redact any confidential info from the logs before attaching. See https://github.com/spekt/testlogger/wiki/Collecting-vstest-logs

Additional context Add any other context about the problem here.

[codito]

@mregen thank you for reporting this issue. OOM from a logger is indeed strange.

Is the log file becoming too big here? Should the xml logger be removed?

The logger is only used if --logger nunit is specified in dotnet test or vstest.console.exe commandline. I couldn't find any references to the logger getting invoked in the test pipelines. So, may be nunit logger is no longer used in the tests and can be safely removed from this project.

I will try to do some stress tests with large # of tests and see if we can see any memory fluctuations across the two releases of this logger.

[mregen]

Hi @codito, in the meantime I tested to completely remove the nunitxml.testlogger, and in certain cases I reproducibly still hit the outofmemory exception. So I think it was coincidence when bumping the lib that the error occurs. The root cause is probably that some log file gets too big, then nunit loads some huge result file that blows the memory manager. Sorry for the confusion!