Azure / pytest-azurepipelines

Plugin for pytest that makes it simple to work with Azure Pipelines
MIT License
113 stars 35 forks source link

Can't generate and use JUnit XML results file #63

Closed Codym48 closed 2 years ago

Codym48 commented 2 years ago

After upgrading to pytest-azurepipelines 1.0.3, when I call pytests --junit-xml=test-output.xml as part of the pipeline, I get a JUnit format output file overwriting an NUnit format output file, and the pytest-azuredevops plugin fails when trying to publish the JUnit XML file because it treats it as an NUnit file (#vso[results.publish type=NUnit;])

============================= test session starts ==============================
platform linux -- Python 3.6.15, pytest-7.0.1, pluggy-1.0.0
rootdir: /home/vsts/work/1/s
plugins: azurepipelines-1.0.3, cov-3.0.0, nunit-1.0.0
collected 1 item

Result Attachments will be stored in LogStore
Run Attachments will be stored in LogStore
Failed to parse result files: System.NotSupportedException: Invalid file format.
   at Microsoft.TeamFoundation.TestClient.PublishTestResults.NUnitResultParser.ParseTestResultFile(TestRunContext runContext, String filePath)
   at Microsoft.TeamFoundation.TestClient.PublishTestResults.NUnitResultParser.<>c__DisplayClass1_0.<ParseTestResultFiles>b__0(String file)
   at System.Linq.Enumerable.SelectListIterator`2.MoveNext()
   at System.Linq.Enumerable.WhereEnumerableIterator`1.ToList()
   at Microsoft.TeamFoundation.TestClient.PublishTestResults.NUnitResultParser.ParseTestResultFiles(TestRunContext runContext, IList`1 resultFilePaths)
   at Microsoft.VisualStudio.Services.Agent.Worker.TestResults.Parser.ParseFiles(IExecutionContext executionContext, TestRunContext testRunContext, List`1 testResultsFiles, ITestResultParser testResultParser)

-------- generated Nunit xml file: /home/vsts/work/1/s/test-output.xml ---------
----------- generated xml file: /home/vsts/work/1/s/test-output.xml ------------
============================== 1 passed in 0.05s ===============================

I notice this in the 1.0.0a1 release notes,

FEATURE: Use pytest-nunit instead of Junit XML to have richer details and stack traces in the UI.

Is the intent to only support NUnit, not JUnit? If so, feel free to close the issue.

tonybaloney commented 2 years ago

Only NUnit now.

The legacy JUnit export was out of convenience.

The NUnit format is much better supported in Azure Pipelines, so you'll get tracebacks, you can add attachments, all sorts.

Codym48 commented 2 years ago

Understood. So pytest-azurepipelines 1.0.x is NUnit only, Python 3 only, and pytest >= 5 only, while 0.8.0 still supported JUnit, Python 2.7, and pytest 4 (but not 7.1)

tonybaloney commented 2 years ago

Yes, Pytest dropped support for Python 2.7 in 5.x so Pytest 5> means Python 3.6+ too.

You can pin an old version of this package if you need to test Python 2.7 and an old Pytest version.

Codym48 commented 2 years ago

Understood. Thank you for creating and maintaining this plugin!