[Feature] Make use of custom annotations instead of title for ID linking

[Az8th]

Would it be possible to make use of custom annotations for test plan ID linking ? That way we could avoid polluting test titles ;)

Example :

test('my test title', async ({ page }) => {
  test.info().annotations.push({
    type: 'azureTestPlanID',
    description: '42',
  });
  // ...
});

This is indeed more verbose, but adding custom decorator support would fix this. The same approach can be seen in the playwright-bdd package, and it is very handy !

Example :

@azureTestPlanID(42)
test('my test title', async ({ page }) => {
  // ...
});

What are your thought about this ?

[Az8th]

Hi ! Just a remember that keeping test title clean seems to be encouraged since 1.42 update, as tags and annotations can now be added outside of it. Could the same be applied here ? Without decorators, it could appear like so

test("Title", {
  tag: ['@tag1', '@tag2'],
  annotation: [{ type: 'azureTestPlanID', description: '42' }]
  }, async ({ page }) => {
  //perform actions
});

Please let me know if this repository is still maintained, and if you can envisage to do it by yourself or if I shall do a PR (in this case, I need to know if it should be backward compatible)

[alexneo2003]

Please let me know if this repository is still maintained

yep, it is still maintained) but unfortunately I don't have enough time for this feature

if I shall do a PR

I'm really will be appreciated for you work

if it should be backward compatible

yes please let it be backwards compatible because many people won't want to upgrade

[Az8th]

Hello, sorry but I don't think I will be able to do such a pull request quickly, I realized the time I need to do it, and I think it would be better used by putting those features in an AzureDevOps or VSCode extension directly, especially since the API seems to give some alternatives that I would like to explore

[alexneo2003]

@Az8th I got you Let me write this feature

[Az8th]

Just realized that using the same method for test plan/suite ID could also permit to add support for multiple plans, as annotations can also be made in test.describe() To ensure backwards compatibility, the test plan ID should still be present in the playwright.config.ts and act as a default, that can be overrided when an azureTestPlanID is precised.

Example:

...
annotation: [
  { type: 'azureTestPlanID', description: '1' },
  { type: 'azureTestSuiteID', description: '34' },
  { type: 'azureTestCaseID', description: '42' },
  // { type: 'azureSharedStepID', description: '255' }
]
...

[alexneo2003]

@Az8th I had published beta version this this feature, try it

npm i @alex_neo/playwright-azure-reporter@1.9.0-beta.0

Also you can read README about version

[Az8th]

Nice ! Can't at the moment, but will definitely try by tomorrow ;) I see that you have chosen to use tags over annotations, aren't they exposed ?

[alexneo2003]

@Az8th any updates?

[Az8th]

Tested it in CI for a week, while I encountered no bugs, I think that tags are even worse than titles for ID specifying (titles are somewhat unique, while tags should be used to pack test by categories) We should definitely go for test annotations here ! I don't know if they are exposed in the Playwright API, I will check this later today !

[Az8th]

Seems it is : https://playwright.dev/docs/api/class-testcase#test-case-annotations Can adapt your PR to support annotations instead of tags if you want. Unless you also want to support tags, but I would suggest to add a "id-mode" parameter that could be either title, tag or annotation to avoid unwanted collisions, which defaults to "title" to maintain backwards compatibility