webdriverio-community / wdio-testrail-reporter

MIT License
6 stars 9 forks source link

multiples runs create when oneReport is set to true, maxInstances: 1, and tests are organised in different spec files #397

Open alexandrabusu13 opened 2 months ago

alexandrabusu13 commented 2 months ago

Hello, I'm testing something on a mobile app. I have the test organised in different spec files. When i'm trying to upload the results in testrail, i see that multiple test runs are created, one for each spec file i'm using "webdriverio": "^9.0.7", @wdio/testrail-reporter": "^0.3.0" I have the following in the config file

  reporters: [
    ['testrail', {
      projectId: {pojectId},  // Set your project ID here
      suiteId: 1,    // Set your test suite ID here
      domain: '{testrail domain}',
      username: {testrail username},
      apiToken: {testrail apikey},
      runName: `Automated Test Run ${new Date().toISOString()}`, // Optional run name
      oneReport: true,
      includeAll: false,
      outputDir: './reports'
    }]

is there an issue with this library and "webdriverio": "^9.0.7"?

christian-bromann commented 2 months ago

Thanks for reporting, any contributions would be appreciated.

tomprzadka commented 1 month ago

Encountered same issue recently, had same testrail-reporter setup, too.

In my case, getLastTestRun in api.ts has been returning an empty array, even though new test runs were getting created with each spec file launched.

SashaGo3 commented 1 month ago

@alexandrabusu13 @tomprzadka Found solution.

I passed suiteId as 1, but it required valid suiteId. You can find valid if you open test cases page on your project.

The url looks like this: https://{{domain}}/index.php?/suites/view/18&group_by=cases:section_id&group_order=asc&display_deleted_cases=0 where 18 is the suiteId

After the change I have only one test run for all spec files.

ghost commented 1 month ago

@SashaGo3 for me it didn't work.. this is my url https://{{domain}}/index.php?/suites/view/3&group_by=cases:section_id&group_order=asc&display_deleted_cases=0 and i have this config

reporters: [
    ['testrail', {
      projectId: 3,  // Set your project ID here
      suiteId: 3,    // Set your test suite ID here
      domain: '{{domain}}',
      username: process.env.TESTRAIL_USERNAME,
      apiToken: process.env.TESTRAIL_API_TOKEN,
      runName: `Automated Test Run ${new Date().toISOString()}`, // Optional run name
      oneReport: true,
      includeAll: false,
      outputDir: './reports'
    }]
  ],

and this is the result

Screenshot 2024-10-29 at 17 13 08

maybe i'm doing something else wrong...

SashaGo3 commented 1 month ago

@alexandrabusu13 Your runName is dynamic, maybe that is the reason. Could you try without ${new Date().toISOString()}?

Here is my config:

reporters: [
    [
      'testrail',
      {
        projectId: testData.env.TESTRAIL_PROJECT_ID,
        suiteId: testData.env.TESTRAIL_TEST_SUITE_ID,
        domain: testData.env.TESTRAIL_DOMAIN,
        username: testData.env.TESTRAIL_USERNAME,
        apiToken: testData.env.TESTRAIL_API_TOKEN,
        runName: 'E2E (Android)',
        oneReport: true,
        includeAll: false,
      },
    ],
  ],
ghost commented 1 month ago

@SashaGo3 that was the issue...i set the runName to "Automated Test Run" and it worked Thank you:D