search

microsoft / azure-pipelines-tasks

Tasks for Azure Pipelines
https://aka.ms/tfbuild
MIT License
3.51k stars 2.62k forks source link

Publish .trx Test Results Doesn't Contain Test Owners #5174

Closed yiqzhou closed 7 years ago

yiqzhou commented 7 years ago

VSTS task "Publishing Test Result" doesn't show test owners while the owners information exist in the published .trx file. It began in 1 or 2 hours.

image

yiqzhou commented 7 years ago

Add some details recently got: For "Publish Test Results", it seems recordings of a test's owner fade after hours. If result of the test is published again, even in another build or release, test owner in the older build can be fetched. We've got 3 tests as example: 1.Microsoft.Ads.UcmApi.Database.FunctionalTests.UcmTicketingTests.GetFilteredTicketGrid 2.Microsoft.Ads.UcmApi.FunctionalTests.TicketingWorkspaceTests.TicketingAdminApiTests.TicketingWorkspaceTeamProductDataTable 3.Microsoft.Ads.UcmApi.FunctionalTests.TicketingWorkspaceTests.GetAndSearchTicketApiTests.TicketingWorkspaceBookOfTickets_V2 The 1st test comes from one assembly, while the 2nd and the 3rd come from the other. We got their APIs from webpage and wrote a script to query continuously. Here are the caught results. For the 1st test: image For the 2nd test: image For the 3rd test: image It appears to be that test owner seldom got lost. However, this is because that we have started to run these tests more frequently (due to cloudtest fixed). Last week, these tests got run and published for 3 hours only once and the tests' owner disappeared about 1 hour after result files published.

yiqzhou commented 7 years ago

I've ran the script to detect test owner of Microsoft.Ads.UcmApi.Database.FunctionalTests.UcmTicketingTests.GetFilteredTicketGrid() for 24 hours and here's the result: image

  1. There is a cycle of about 1 hour but not precisely. I guess the storage containing test owner cleans itself once get full?
  2. At each reappearance of test owner, I found a test result publishment record during past 1 minute in our team's records. It's not a "recovery" but a "republishment" from other builds or releases. Hope these may help.
ratrakapil commented 7 years ago

We will investigate this issue and get back by Monday. Thanks for details.

whichrakesh commented 7 years ago

Owner and priority are updated based on the run and in this case, there was a different run that wasn’t publishing the owner info while publishing results from "Publish test results task". The customer has agreed to fix this from his side.