codeceptjs / CodeceptJS

Supercharged End 2 End Testing Framework for NodeJS
http://codecept.io
MIT License
4.11k stars 724 forks source link

run-multiple execution creates output sub folders with alphanumeric value instead of browser names #4265

Open AugustineAutoGit opened 7 months ago

AugustineAutoGit commented 7 months ago

What are you trying to achieve?

For run-multiple execution with multiple browsers like Chrome, Safari, Firefox and Edge, the subfolder in the output folder should have a name like parallel_Chrome_1, parallel_Safari_2, parallel_Firefox_3 etc

What do you get instead?

While executing using run-multiple the subfolder in the output folder should have name like the following: parallel_0096a0c740a073d21e23267bf167f711a9826c9d84addc417bc5b320df11e0bd_1 parallel_f7899c9a407a947a824a4955416619aef2c53bd0fbb18965b53d0a3fb4c207d6_2

instead of browser/channel name some alphanumeric value is present in the folder name.

image

This makes it difficult to do many validations. For eg: Validating the file download using handleDownloads and FileSystem helper, like using methods I.amInPath('output/folderName'); where folderName is parallel_0096a0c740a073d21e23267bf167f711a9826c9d84addc417bc5b320df11e0bd_1. This is only 1 example

Provide console output if related. Use --verbose mode for more details.

# paste output here

Provide test source code if related

// paste test

Details

# paste config here
kobenguyent commented 6 months ago

I think that's supposed to be like that when you don't explicitly define the outputName Screenshot 2024-03-22 at 10 37 29

To overcome your issue, try this approach https://github.com/codeceptjs/CodeceptJS/issues/1722#issuecomment-529903127

"multiple": {
        "dev": {
            "browsers": [
                {
                    browser: "chrome",
                    outputName: "chrome"
                },
                {
                    browser: "firefox",
                    outputName: "firefox"
                }
            ]
        }
}
AugustineAutoGit commented 6 months ago

Thanks @kobenguyent this fixed my issue of random hash value For the case of: Validating the file download using handleDownloads and FileSystem helper, like using methods I.amInPath('output/folderName'); for folderName how should we give? say now output folders are like parallel_output_1, the runId is dynamic I tried this way but not working: I.amInPath('output/paralleloutput*');

AugustineAutoGit commented 6 months ago

Sure

AugustineAutoGit commented 6 months ago

@kobenguyent 1 doubt / suggestion Is appending runId to the outputDir mandatory, like can we avoid it? Instead can we append the browser Since run-mutiple itslef is for parallel testing in multiple browsers can we something like: outputDir = runName + outputName + browser

For eg: for Playwright browser can be :

Hence my doubt / suggestion is the possibility of removing runId append and instead adding browser and channel name if any. Like: outputDir = runName + outputName + browser + channel(only for chromium)

kobenguyent commented 6 months ago

@AugustineAutoGit I think run-multiple command is kinda obsolete and there would be not really active maintenance. It is encouraged to use run-workers command.

AugustineAutoGit commented 6 months ago

@kobenguyent I am using codeceptjs-resemblehelper for visual testing. This helper won't work with run-workers due to inability to execute in multiple threads. Sad to know that run-multiple command is kinda obsolete :( run-multiple was very helpful for me as it is preferred as mentioned in CodeceptJS site for mutliple browser executions. Our paralle execution requirement is for multiple browsers. Also due to budget constraints not able to use codeceptjs-applitoolshelper

kobenguyent commented 6 months ago

Well @AugustineAutoGit you could raise a PR if you have time to fix it.

AugustineAutoGit commented 6 months ago

@kobenguyent sure will give a try from my side. Will you provide access or do I need to do a fork or if it run-multiple command is kinda obsolete do we need to consider this

Will proceed as per your suggestion

kobenguyent commented 6 months ago

@AugustineAutoGit https://github.com/codeceptjs/CodeceptJS/blob/3.x/.github/CONTRIBUTING.md#contributing

github-actions[bot] commented 3 months ago

This issue is stale because it has been open for 90 days with no activity.