search

AArnott / Library.Template

A template for a NuGet package with tests, stylecop, fxcop, versioning, and Azure Pipelines build ready to go.
MIT License
131 stars 26 forks source link

Feature Suggestion: Parse TRX files to better organize testResults artifacts #170

Open SteveBush opened 2 years ago

SteveBush commented 2 years ago

A trx file contains all of the relevant information to better organize a testResults artifacts folder. Ideally, we could organize the results folder using the same folder structure as /bin or /obj:

Unfortunately, data collectors have no concept of a target framework and can often produce more than one copy of an artifact. The goal would be to do the following:

  1. Deduplicate trx files
  2. For each unique trx file: a) Parse the tag to determine ProjectName, BuildConfiguration, and Target Framework b) Parse the tag to discover associated files by collector agent
  3. Organize the artifacts directory based on this information.

Here's my pseudo code to parse a trx file to figure out a more rational test result structure:

For each unique trx file:
       1) Parse the trx file <UnitTest> element to find:
             a) Project name
             b) Build Configuration
             c) TargetFramework

Example: 
<UnitTest name="TestBase_CreateTestLogger_Type" storage="/users/runner/work/1/s/bin/networkvisor.platform.test.macos.integrationtests/release-macos/net5.0/networkvisor.platform.test.macos.integrationtests.dll" id="b73df19d-c122-b73f-4661-41bdf76843e3">

        2) Parse CollectorDataEntries to determine associated files:
            a) Collector agentName
            b) Hang dump
            c) Sequence
            d) Code Coverage

Example:
 <CollectorDataEntries>
      <Collector agentName="Mac-1655736263462" uri="datacollector://microsoft/TestPlatform/Extensions/Blame/v1" collectorDisplayName="Blame">
        <UriAttachments>
          <UriAttachment>
            <A href="Mac-1655736263462/dotnet_3883_20220620T170125_hangdump.dmp"></A>
          </UriAttachment>
          <UriAttachment>
            <A href="Mac-1655736263462/Sequence_2f7f6ae031cc4ca0a6568c9b07e4639b.xml"></A>
          </UriAttachment>
        </UriAttachments>
      </Collector>
      <Collector agentName="Mac-1655736263462" uri="datacollector://microsoft/CodeCoverage/2.0" collectorDisplayName="Code Coverage">
        <UriAttachments>
          <UriAttachment>
            <A href="Mac-1655736263462/runner_Mac-1655736263462_2022-06-20.16_59_05.cobertura.xml"></A>
          </UriAttachment>
        </UriAttachments>
      </Collector>
    </CollectorDataEntries>

Using this information you could publish testResults-XXX folder that is organized as follows:

<subdirectory>
    <projectname>
          <build-configuration>
               <targetframework>
                   trx file
                   dmp file
                   code coverage file
                   sequence file
                   diag.datacollector file
                   diag.host file