Allure.XUnit: Test cases are interpreted as independent new tests

[smolchanovsky]

[//]: # ( . Note: for support questions, please use Stackoverflow or Gitter. . This repository's issues are reserved for feature requests and bug reports. . . In case of any problems with Allure Jenkins plugin please use the following repository . to create an issue: https://github.com/jenkinsci/allure-plugin/issues . . Make sure you have a clear name for your issue. The name should start with a capital . letter and no dot is required in the end of the sentence. An example of good issue names: . . - The report is broken in IE11 . - Add an ability to disable default plugins . - Support emoji in test descriptions )

Hi!

I'm submitting a ...

• bug report
• feature request

What is the current behavior?

Each test case of a parameterized test is interpreted as an independent new test without parameters.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

The minimal demo to reproduce is already in the examples :)

In the generated report for this test, there are two independent tests with different name/fullName and without parameters:

{
  "uuid": "1c3834b2-cdec-4ade-85e1-ce98d530e550-TestTheory(a: 1, b: 1)",
  "name": "TestTheory(a: 1, b: 1)",
  "fullName": "TestTheory(a: 1, b: 1)",
  ...
}

{
  "uuid": "5b7b5215-5a38-404d-b246-6b9a63f32f48-TestTheory(a: 1, b: 2)",
  "name": "TestTheory(a: 1, b: 2)",
  "fullName": "TestTheory(a: 1, b: 2)",
  ...
}

although, here is one test simply with different parameter values.

What is the expected behavior?

I would like to be able to get a report with the same names and with the listed parameters:

{
  "uuid": "1c3834b2-cdec-4ade-85e1-ce98d530e550-TestTheory(a: 1, b: 1)",
  "name": "TestTheory",
  "fullName": "TestTheory",
  "parameters": [
    {
      "name": "a",
      "value": "1"
    },
    {
      "name": "b",
      "value": "1"
    }
  ],
  ...
}

{
  "uuid": "5b7b5215-5a38-404d-b246-6b9a63f32f48-TestTheory(a: 1, b: 2)",
  "name": "TestTheory",
  "fullName": "TestTheory",
  "parameters": [
    {
      "name": "a",
      "value": "1"
    },
    {
      "name": "b",
      "value": "2"
    }
  ],
  ...
}

What is the motivation / use case for changing the behavior?

The current behavior is acceptable when the parameter values are immutable. However, when random generation of test data is used (e.g. AutoFixture), it becomes a problem, because the parameter values are always unique, which means there is no test history, and the number of unique tests increases with each publication of the report.

Please tell us about your environment:

• Test framework: xunit 2.4.1
• Generate report using: Allure.XUnit 2.9.2-preview.1

Other information

To generate random test data, I use AutoFixture.Xunit2 4.17.0

[//]: # ( . e.g. detailed explanation, stacktraces, related issues, suggestions . how to fix, links for us to have more context, eg. Stackoverflow, Gitter etc )

[smolchanovsky]

I created the pull request (https://github.com/allure-framework/allure-csharp/pull/310), where I added parameter filling and two attributes that allow to set custom names for tests. The changes are not breaking

[neparij]

@smolchanovsky Hi! I agree with your point. I reviewed your PR, note my comment please.

Thank you very much for contributing to Allure! ☀️

[1Playerr]

@neparij Hi!

I've added some changes on this PR to resolve PR comments. Could you please check the PR once again?

[delatrie]

The fix is included in the newly released GA version 2.10.0 (it was included in a preview version earlier). If the issue still exists, please, leave a comment here.