DTAExecutionHost.exe failed with code 1 - Error occurred while publishing the test results : OutOfMemoryException

Description

Our Azure pipelines use a custom build agent, and our fairly large solution has about 4000 tests that collectively seem to produce a log file about 6.5 million lines long when the tests are being run. The tests succeed and dump a TRX file of unknown size (never been able to catch it) into the temp agent folder. The test runner then seems to try and read the file back in and runs out of memory while doing so

Steps to reproduce

I've no idea; have a lot of tests that generate huge log output?

Originally I had issues with the log file cutting off after 1609 megabytes (the file just ended mid - line). I reduced the verbosity of the entity framwork logging and cut the file down to about 800 megabytes. Now I see what I think is the full output. I've been unable to capture a full TRX file so far

Expected behavior

Well, really, i'd like to be able to get my PR passed because the tests work, it's just the process that seems to publish the success (where to, i've no idea) crashes

Actual behavior

Here is the tail of the log file, from line 6,483,981 onwards:

2024-06-25T10:53:39.9012848Z NUnit Adapter 4.4.0.0: Test execution complete
2024-06-25T10:54:29.2631367Z Results File: C:\agent\_work\_temp\TestResults\AcmeRubberBuildAgent$_AcmeRubberBuildAgent_2024-06-25_10_42_16.trx
2024-06-25T10:54:29.3219829Z Test Run Successful.
2024-06-25T10:54:29.3220389Z Total tests: 3706
2024-06-25T10:54:29.3224945Z      Passed: 3430
2024-06-25T10:54:29.3225301Z     Skipped: 276
2024-06-25T10:54:29.3245276Z  Total time: 11.5205 Minutes
2024-06-25T10:54:30.1212719Z Vstest.console.exe exited with code 0.
2024-06-25T10:54:30.1213282Z **************** Completed test execution *********************
2024-06-25T10:54:30.1964605Z Test results files: C:\agent\_work\_temp\TestResults\AcmeRubberBuildAgent$_AcmeRubberBuildAgent_2024-06-25_10_42_16.trx
2024-06-25T10:54:37.9690262Z ##[error]Error occurred while publishing test results : System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
2024-06-25T10:54:37.9691501Z    at System.Text.StringBuilder.ToString()
2024-06-25T10:54:37.9691856Z    at System.IO.File.InternalReadAllText(String path, Encoding encoding, Boolean checkHost)
2024-06-25T10:54:37.9692367Z    at Microsoft.TeamFoundation.TestClient.PublishTestResults.TrxResultParser.CreateXmlDocumentFromResultFile(String filePath)
2024-06-25T10:54:37.9692995Z    at Microsoft.TeamFoundation.TestClient.PublishTestResults.TrxResultParser.ParseTestResultFile(TestRunContext runContext, String filePath)
2024-06-25T10:54:37.9693482Z    at System.Linq.Enumerable.WhereSelectArrayIterator`2.MoveNext()
2024-06-25T10:54:37.9693919Z    at System.Linq.Enumerable.WhereEnumerableIterator`1.MoveNext()
2024-06-25T10:54:37.9694361Z    at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
2024-06-25T10:54:37.9694717Z    at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
2024-06-25T10:54:37.9695077Z    at Microsoft.TeamFoundation.TestClient.PublishTestResults.TrxResultParser.ParseTestResultFiles(TestRunContext runContext, IList`1 resultFilePaths)
2024-06-25T10:54:37.9695528Z    at Microsoft.VisualStudio.TestService.TcmTestResultPublisher.TrxTestResultPublisher.ParseTrx(TestRunContext testRunContext, IList`1 trxFilePaths)
2024-06-25T10:54:37.9695943Z    at Microsoft.VisualStudio.TestService.TcmTestResultPublisher.TrxTestResultPublisher.PublishTestRunResults(TestRunModel testRunModel)
2024-06-25T10:54:37.9696297Z    at MS.VS.TestService.VstestConsoleAdapter.PublishTestResult.CreateOrUpdateTestRun(TestRunModel testRunModel)
2024-06-25T10:54:37.9696859Z    at MS.VS.TestService.VstestConsoleAdapter.PublishTestResult.Publish(VstestConsoleRunContext testRunContext)
2024-06-25T10:54:37.9733528Z ##[error]Skipping marking test run as complete as test run id 0 is invalid.
2024-06-25T10:54:38.2040344Z Completed TestExecution Model...
2024-06-25T10:54:38.2136771Z 
2024-06-25T10:54:38.5766128Z ##[warning]Vstest failed with error. Check logs for failures. There might be failed tests.
2024-06-25T10:54:38.5767691Z ##[error]Error: The process 'C:\agent\_work\_tasks\VSTest_ef087383-ee5e-42c7-9a53-ab56c98420f9\2.240.1\Modules\DTAExecutionHost.exe' failed with exit code 1
2024-06-25T10:54:38.5770146Z ##[error]Vstest failed with error. Check logs for failures. There might be failed tests.
2024-06-25T10:54:38.6172971Z ##[section]Finishing: Run tests

Diagnostic logs

I can send this file if you want, but it's a lot of info to mask

Environment

OS Name Microsoft Windows Server 2022 Datacenter Azure Edition Version 10.0.20348 Build 20348 Other OS Description Not Available OS Manufacturer Microsoft Corporation System Name AcmeRubberBuildAgent System Manufacturer Microsoft Corporation System Model Virtual Machine System Type x64-based PC System SKU None Processor Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz, 2095 Mhz, 2 Core(s), 4 Logical Processor(s) BIOS Version/Date Microsoft Corporation Hyper-V UEFI Release v4.1, 5/13/2024 SMBIOS Version 3.1 Embedded Controller Version 255.255 BIOS Mode UEFI BaseBoard Manufacturer Microsoft Corporation BaseBoard Product Virtual Machine BaseBoard Version Hyper-V UEFI Release v4.1 Platform Role Desktop Secure Boot State On PCR7 Configuration Binding Possible Windows Directory C:\Windows System Directory C:\Windows\system32 Boot Device \Device\HarddiskVolume3 Locale United States Hardware Abstraction Layer Version = "10.0.20348.2031" User Name Not Available Time Zone Coordinated Universal Time Installed Physical Memory (RAM) 16.0 GB Total Physical Memory 15.9 GB Available Physical Memory 11.5 GB Total Virtual Memory 19.7 GB Available Virtual Memory 15.0 GB Page File Space 3.77 GB Page File D:\pagefile.sys Kernel DMA Protection Off Virtualization-based security Not enabled Device Encryption Support Reasons for failed automatic device encryption: Feature is not available, Hardware Security Test Interface failed and device is not Modern Standby, WinRE is not configured A hypervisor has been detected. Features required for Hyper-V will not be displayed.

microsoft / vstest