Allure plugin doesnt generate report when running parallel execution in jenkins

[nkrishna79]

I have trying to generate reports using allure plugin when running tests in parallel

When looking at the report output, I see nothing getting populated

Provide console output if related. Use --verbose mode for more details.

# paste output here

Provide test source code if related

exports.config = {
    tests: './specs/*_test.js',
    output: './output',
    helpers: {
        WebDriver: {
            url: "http://hub-cloud.browserstack.com/wd/hub",
            browser: "chrome",
            smartWait: 5000,
            restart: true,
            user: "username",
            key: "key",
            capabilities: {
                "os": "Windows",
                "os_version": "10",
                "browserName": "Chrome",
                "browser_version": "72.0",
                "project": "codecept-wdio-qa",
                "build": "Build1.0",
                "browserstack.local": true,
                "browserstack.debug": true
            },
            host: "hub-cloud.browserstack.com"
        }
    },
    plugins: {
        wdio: {
            enabled: true,
            services: ['selenium-standalone']
        },
        allure: {
            enabled: true
        }
    },
    multiple: {
        parallel: {
            // Splits tests into 2 chunks
            chunks: 2
        }
    },
    include: {
        I: './steps_file.js'
    },
    bootstrap: null,
    mocha: {},
    name: 'codeceptjs-e2espec-tests'
};

Jenkins allure report config

output as you see in jenkins

Details

• CodeceptJS version:
• NodeJS Version:
• Operating System: *WebDriverIO

[yusufilkeroguz]

Hi,

I am using more plugins for allure and working for me. My Plugins;

const plugins = {
    allure: {
      outputDir: OUTPUT_DIR
    },
    screenshotOnFail: {
      enabled: true
    },
    stepByStepReport: {
      enabled: true,
      output: OUTPUT_DIR
    }
}

[nkrishna79]

@yusufilkeroguz are you running them on jenkins or on local? Can you also share your config where you have configured it for multiple parallel execution. Thanks

[yusufilkeroguz]

I am using allure on local.

This is my config;

{
    "output": "./output",
    "helpers": {
        "WebDriver": {
            "url": "[URL]",
            "smartWait": 5000,
            "browser": [
                "chrome",
                "safari"
            ],
            "restart": false,
            "windowSize": "1920x1080",
            "user": "[BS_USER]",
            "key": "[BS_PASS]",
            "protocol": "http",
            "host": "hub-cloud.browserstack.com",
            "waitForTimeout": 15000,
            "timeouts": {
                "script": 45000,
                "implicit": 15000,
                "pageLoad": 30000
            },
            "capabilities": {
                "build": "[BuildName]",
                "project": "[ProjectName]",
                "resolution": "1920x1080",
                "os": "OS X",
                "os_version": "Mojave",
                "acceptSslCerts": true,
                "browserstack.debug": true,
                "browserstack.networkLogs": true,
                "browserstack.local": true
            }
        },
        "WebDriverHelper": {
            "require": "./helpers/WebDriver.js"
        }
    },
    "plugins": {
        "allure": {
            "outputDir": "./output"
        },
        "screenshotOnFail": {
            "enabled": true
        },
        "stepByStepReport": {
            "enabled": true,
            "output": "./output"
        },
    },
    "multiple": {
        "basic": {
            "browsers": [
                {
                    "browser": "safari",
                    "desiredCapabilities": {
                        "browserName": "Safari",
                        "browser_version": "12.1",
                        "resolution": "1920x1080",
                        "timeouts": {
                            "script": 45000,
                            "implicit": 15000,
                            "pageLoad": 30000
                        }
                    }
                }
            ]
        }
    },
    "include": {
        "I": "./steps_file.js"
    },
    "mocha": {},
    "bootstrap": false,
    "teardown": null,
    "hooks": [],
    "gherkin": {
        "features": "./features/*.feature",
        "steps": [
            "./features/step_definitions/happy-path.js",
            "./features/step_definitions/landing.js"
        ]
    },
    "tests": "./features/step_definitions/*.js",
    "timeout": 10000,
    "name": "[NAME]"
}

[natkrish]

@yusufilkeroguz I dont have problems running it on local. This issue happens when you run on Jenkins

[GSasu]

@yusufilkeroguz I would highly suggest you setup your plugin in a similar way to this:

        allure: {
            enabled: true,
            outputDir: './output/allure/allure-results',
        },

In which case in your jenkins setup the path would be this output/allure/allure-results

[bhautikbpatel]

Facing the same issue not able to generate allure report when we use multiple: { parallel: { // Splits tests into 2 chunks chunks: 2 } },

[kobenguyent]

@bhautikbpatel are you testing against latest version of codeceptjs. I'm able to see the generated allure report after the tests are executed.

Thanhs-MacBook-Pro:pizza-ui-tests thanhnguyen$ npx codeceptjs run-multiple --all
creating output directory: /Users/thanhnguyen/Desktop/pizza-ui-tests/output/parallel_chunk1_4697d80bdefdef94aaba8756c0032a6f_1
[1.parallel:chunk1:default] CodeceptJS v2.3.5
[1.parallel:chunk1:default] Using test root "/Users/thanhnguyen/Desktop/pizza-ui-tests"

[1.parallel:chunk1:default] Login functionality --
[1.parallel:chunk1:default]   ✔ Login with valid creds @C1 in 1057ms
[1.parallel:chunk1:default]   S Login with valid creds @C2

[1.parallel:chunk1:default]   OK  | 1 passed, 1 skipped   // 3s

[auto-qa-git]

Hi, I've read through all these comments and I still have the issue. Can anyone please help with what it is that I am doing wrong? I may as well mention that I am trying to change the directory where allure reports are stored but when I do, the generated report is empty.

This is the alure.js file:

This is what I see in the IDE console when I type allure serve output:

When I try to specify another folder, I have the problem where the generated report shows nothing.

I get the same screen when using npx codeceptjs run-multiple command.

Can someone please help me with this? Thanks.

[GSasu]

@auto-qa-git if you look in the screenshots that you posted, you can see your problem there but let me guide you through this:

• Allure is a wonderful and most commonly used reporter out there. This doesn't mean that it doesn't have quirks.
• One of the quirks is that the test results needs to be stored in a folder called 'allure-results'. If this is not done, then allure will revert to storing the results in a temporary folder which you can also see in the screenshot you added of the allure serve command output.

Solution:

codecept.conf.js

allure plugin config:

    plugins: {
        allure: {
            enabled: true,
            outputDir: './output/allure/allure-results', // notice the path used here pointing to allure-results
        },

In your jenkins allure plugin config the path needs to be the same as above: "output/allure/allure-results"

I used a groovy file for my jenkins pipeline so here's how my allure reports are generated:

    post {
        always {
            generateAllureReport(env.SUITE)
        }
        success {
            slackSend channel: env.NOTIFICATIONS_CHANNEL ?: '#some-slack-channel',
                      color: 'good',
                      message: getNotificationMessage('Success')
        }
        failure {
            slackSend channel: env.NOTIFICATIONS_CHANNEL ?: '#some-slack-channel',
                      color: 'danger',
                      message: getNotificationMessage('Failed')
        }
    }
}

def getNotificationMessage(buildStatus) {
    return "${env.JOB_NAME} - #${env.BUILD_NUMBER} ${buildStatus} after " +
        "${currentBuild.durationString.replace(' and counting', '')} (<${env.RUN_DISPLAY_URL}|Open>)"
}

def generateAllureReport(suite) {
    if(suite) {
        return script {
                    allure([
                        includeProperties: false,
                        jdk: '',
                        properties: [],
                        reportBuildPolicy: 'ALWAYS',
                        results: [[path: 'output/allure/allure-results']]
                    ])
                }
    }
}

Try this out and let me know if you need more help with this.

[auto-qa-git]

@GSasu Hi, thank you so much for your explanation. Earlier I tried this and it worked: I am using allure on local. No Jenkins at the moment. However, when I change to outputDir: './output/allure/allure-results' I am having the same issue as before. It is a faulty report (showing 0 Test Cases etc... even though several test cases did run). And I still see this line in the console: Is it only in codecept.conf.js that I have to specify the directory? And again - I am running my tests and using allure on local. I would go with what's working now but I am having another problem which might be related: The issue that I am having is that when I run the test with parallel execution on say chrome and firefox, I get the results of only one. I do not get the report showing all test case results with the browser for each. I am aiming to have a report similar to this:

Could you help with this please? Thank you.

[kobenguyent]

allure plugin is now maintained by allure team https://github.com/allure-framework/allure-js/tree/master/packages/allure-codeceptjs