Qytera-Gmbh / cypress-xray-plugin

A plugin for uploading Cypress test results to Xray.
https://qytera-gmbh.github.io
MIT License
22 stars 7 forks source link

Adding parallelization to run tests on Cypress but have several test execution issues created #402

Open ethoreau opened 1 day ago

ethoreau commented 1 day ago

Hi,

Recently, I tried to add parallelization when executing tests with a Gitlab pipeline like that :

tnr-sandbox:
   image: cypress/browsers:node-22.11.0-chrome-130.0.6723.116-1-ff-132.0.1-edge-130.0.2849.68-1
   stage: test
   parallel: 3
   rules:
       - if: $TEST == 'tnr-sandbox'
   script:
       # install dependencies
       - npm ci
       # run Cypress tests
       # - npx cypress run --parallel --browser chrome --config-file cypress-sandbox.config.js --spec "cypress/e2e/**/*" --reporter mocha-junit-reporter 

I have 7 tests to run, they run well with parallelization but there are several test execution issues created through the execution while when the tests run without parallelization I have only one test execution issue created.

So, I would like to know if I misconfigured the plugin, or if the adding of parallelization is not handled.

Maybe someone already managed parallelization with creation of one test execution issue ?

FYI : I'm using the last version of the plugin (7.4.1) and the last version of Cypress (13.15.2)

Thanks Emilie

csvtuda commented 12 hours ago

Hi @ethoreau,

thanks for the issue. Parallelization is something I haven't investigated yet. Do the reporters generate multiple files as well or do they somehow only end up with one?

For now, I think the quick & dirty solution is to:

  1. Manually create a test execution issue in the CI/CD pipeline before Cypress runs, e.g. through a simple Node script:

    // create-issue.js
    const JIRA_URL = "https://<your-domain>.atlassian.net/rest/api/3/issue";
    const USERNAME = "<your-example@example.org>";
    const API_TOKEN = "<your-token>";
    const PROJECT_KEY = "CYP";
    const ISSUE_SUMMARY = "My parallel execution issue";
    const ISSUE_TYPE = "Test Execution";
    
    (async () => {
        const payload = {
            fields: {
                project: {
                    key: PROJECT_KEY
                },
                summary: ISSUE_SUMMARY,
                issuetype: {
                    name: ISSUE_TYPE
                },
            },
        };
        const response = await fetch(JIRA_URL, {
            method: "POST",
            headers: {
                "Content-Type": "application/json",
                "Authorization": `Basic ${Buffer.from(`${USERNAME}:${API_TOKEN}`).toString("base64")}`,
            },
            body: JSON.stringify(payload),
        });
        const data = await response.json();
        if (response.ok && data.key) {
            console.log(data.key);
        } else {
            console.error("Failed to create issue:", data);
            process.exit(1);
        }
    })();

    The script is for Jira Cloud, for Jira Server you need to change the URL and maybe also the authorization.

  2. Pass the created issue to Cypress in the CI:

    ISSUE_KEY=$(node create-issue.js)
    npx cypress run --parallel <...> --env JIRA_TEST_EXECUTION_ISSUE="{'key':'$ISSUE_KEY'}"

I will look into it further as soon as I can. But seeing as this issue is still open after three years, I doubt there's some sort of "please run this plugin exactly once, even in parallel mode" option 😞