Error while parsing TeamCity service message

[TommyN]

We're getting flaky test reports between TC builds and I'm not sure what the reason is. We have a build step like this:

dotnet test solution.sln --no-build -c Release

We sometimes get +/- 100 unit tests reported and it seems to be related to getting a lot of message like this:

Error while parsing TeamCity service message: Incorrect property name "Unit tests from C:\BuildAgent\work\10b50e516cc998df\Tests\ ...

Status in TC with no related unit test changes

#0.0.24065.10990
Tests passed: 2361, ignored: 304; Build warnings: 0 (+0/-0)

#0.0.24065.10989
Tests passed: 2461, ignored: 316; Build warnings: 0 (+0/-0)

Turning off parallel test execution seems to get rid of all the error messages:

dotnet test -m:1 solution.sln --no-build -c Release

But it increases the test time from 1m 14s to 6m 37s

Packages used in the test projects:

<ItemGroup>
    <PackageReference Include="NUnit" Version="3.13.3" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.9.0" />
    <PackageReference Include="NUnit3TestAdapter" Version="4.5.0" />
    <PackageReference Include="TeamCity.VSTest.TestAdapter" Version="1.0.40" />
</ItemGroup>

[boris-yakhno]

Hello @TommyN, Could you please clarify if dotnet test is run from a .NET build step or from a build step with any other runner (for example, Command line)?

Generally, this is a known problem when targeting .sln and .slnf files with dotnet test – multiple child test processes are launched for solution projects, the stdout output from them gets interleaved and the service messages get corrupted.

In TeamCity 2023.11, we have switched to delivering test reports through files, and test reporting should work correctly when using the .NET build step. If you're running dotnet test manually from a different build step, mitigating the problem would require some manual actions.

[TommyN]

@boris-yakhno, the build step is a Command line build step.

We're on TeamCity 2023.11.3 (build 147512) so I will give the .NET build step a go.

[TommyN]

Switching to a .NET build step with command: test command line parameters: -c Release

Fixed the issue! 🥳 Thanks for the quick response @boris-yakhno!

[koen-lee]

If you're running dotnet test manually from a different build step, mitigating the problem would require some manual actions.

@boris-yakhno We have the same issue, but tests are started from a dotnet test invocation in a Powershell script that has some ceremony to make code coverage work for our case. Could you please point to the manual actions we should take to mitigate the issue?

[boris-yakhno]

@koen-lee Hi! Sure, but let me first reiterate the cause of the problem and list all available options for mitigation. The root of the issue is that the outputs from multiple dotnet test processes become interleaved, corrupting TeamCity service messages. This only happens when targeting .sln or .slnf files. To mitigate this issue, you have several options:

• Switch to the .NET build step. Since TeamCity 2023.11, service messages are delivered via files and should not get corrupted.
• Use the -m:1 parameter for dotnet test to disable any parallelism in dotnet test. This increases the time required to run tests.
• Manually start multiple dotnet test processes that target separate csproj files instead of targeting a single solution file.
• If none of the previous approaches are convenient, manually set up delivering test report service messages via files.

To set up delivery of service messages via files, please do the following:

• Set the TEAMCITY_TEST_REPORT_FILES_PATH environment variable to specify the directory for the files with the test reports.
• Set up set up streaming of these files to the build log with the streamToBuildLog data processor.

Please note that the following:

• The streamToBuildLog data processor will monitor the provided files/paths until the build step finishes.
• Only files under the agent's home, work, checkout and temp directories can be processed with streamToBuildLog.
• Every new instance of the data processor will consume a new OS thread until processing is finished.
• If test reporting is needed in several build steps, you should take care to avoid streaming test reports from previous steps.

Here's an example for PowerShell:

# Set the environment variable for the test report directory directly in the script or in the configuration parameters
$env:TEAMCITY_TEST_REPORT_FILES_PATH = "$env:system_teamcity_build_tempDir\TestReports"

Write-Output "##teamcity[importData type='streamToBuildLog' filePattern='$env:system_teamcity_build_tempDir\TestReports\*.msg' wrapFileContentInBlock='true' quiet='false']"

dotnet test --test-adapter-path /test_adapter_path --logger console --logger teamcity /test_assembly.dll

There was also a similar discussion in our YouTrack.

I hope this solves your issue, please let me know if you need further assitance.

[koen-lee]

Thanks, that works, we now have a consistent number of tests and working test history.