Open Pablo-Quiroga-Globant opened 2 years ago
@FRSgit Hello! Anything else I can do to help?
@Pablo-Quiroga-Globant Hey, I think there is at least one issue with cypress-mochawesome-reporter
that needs to be resolved before your issue can be handled. Have a look here.
To sum up - currently any screenshot with any special characters in it's name won't appear in HTML report generated by cypress-mochawesome and we need a PR with a fix there. I unfortunately won't do it this week - so if you're in rush, please have a look there 🙏
I also encounter the same problem, whether it is necessary to use special characters
Hey @luqy, Thanks for the suggestion, but it would be a breaking change for our users - all of the existing screenshots would need to be removed & regenerated once again. Since it's an obvious error in mochawesome package I'd say it's much better to address the issue there
Just an update: cypress-mochawesome-reporter
has been just fixed https://github.com/LironEr/cypress-mochawesome-reporter/issues/106.
Now I can finish up work resolving this issue, stay tuned! 😁
We are facing a similar issue. Apparently, the compared images do not exist in the file system.
Dependencies:
{
"cypress": "10.8.0",
"cypress-mochawesome-reporter": "3.2.3",
"@frsource/cypress-plugin-visual-regression-diff": "3.1.2",
}
Cypress Error
cy.task('cp-visual-regression-diff-compareImages') failed with the following error:
> ENOENT: no such file or directory, open 'report-desktop/all-screenshots/cp-visual-regression-diff_snapshots/cypress/e2e/desktop/screenshot/image_snapshots/screen check1 #0.actual.png.png
Stack trace
From Node.js Internals:
Error: ENOENT: no such file or directory, open 'report-desktop/all-screenshots/__cp-visual-regression-diff_snapshots__/cypress/e2e/desktop/screenshot/__image_snapshots__/screen check1 #0.actual.png.png'
at Object.openSync (node:fs:585:3)
at Object.readFileSync (node:fs:453:35)
at scaleImageAndWrite (<hidden path>/node_modules/@frsource/cypress-plugin-visual-regression-diff/src/image.utils.ts:64:24)
at compareImagesTask (<hidden path>/node_modules/@frsource/cypress-plugin-visual-regression-diff/src/task.hook.ts:74:33)
at invoke (/Users/mikstime/Library/Caches/Cypress/10.8.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/child/run_plugins.js:234:16)
at <unknown> (/Users/mikstime/Library/Caches/Cypress/10.8.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/util.js:59:14)
at tryCatcher (/Users/mikstime/Library/Caches/Cypress/10.8.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/util.js:16:23)
at Function.Promise.attempt.Promise.try (/Users/mikstime/Library/Caches/Cypress/10.8.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/method.js:39:29)
at Object.wrapChildPromise (/Users/mikstime/Library/Caches/Cypress/10.8.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/util.js:58:23)
at RunPlugins.taskExecute (/Users/mikstime/Library/Caches/Cypress/10.8.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/child/run_plugins.js:240:10)
at RunPlugins.execute (/Users/mikstime/Library/Caches/Cypress/10.8.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/child/run_plugins.js:160:21)
at EventEmitter.<anonymous> (/Users/mikstime/Library/Caches/Cypress/10.8.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/child/run_plugins.js:257:12)
at EventEmitter.emit (node:events:390:28)
at EventEmitter.emit (node:domain:475:12)
at process.<anonymous> (/Users/mikstime/Library/Caches/Cypress/10.8.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/util.js:33:22)
at process.emit (node:events:390:28)
at process.emit (node:domain:475:12)
at process.emit.sharedData.processEmitHook.installedValue [as emit] (/Users/mikstime/Library/Caches/Cypress/10.8.0/Cypress.app/Contents/Resources/app/node_modules/@cspotcode/source-map-support/source-map-support.js:745:40)
at emit (node:internal/child_process:917:12)
at processTicksAndRejections (node:internal/process/task_queues:84:21)
Cypress Error
cy.task('cp-visual-regression-diff-compareImages') failed with the following error: > ENOENT: no such file or directory, open 'report-desktop/all-screenshots/cp-visual-regression-diff_snapshots/cypress/e2e/desktop/screenshot/image_snapshots/screen check1 #0.actual.png.png
After taking a screenshot, it is moved to another folder, so that it could be compared with the original one later. How it's implemented:
on('after:screenshot')
handler asynchronously moves the file and returns {path: newPath}
.cy.screenshot(oldPath, {onAfterScreenshot: (el, {path}) => {…}})
is called to retrieve newPath.So onAfterScreenshot
is expected to have newPath
instead of oldPath
. But in practice this does not happen as of cypress 10.10.0.
@FRSgit I wonder if you have faced similar issues during development process?
Sorry @mikstime for lack of responses - I'm knee-deep into issue #173 at the moment as it seems more critical :<
I don't know if I understood your comment correctly. Can you confirm that the "path replacement" that was implemented in a return method of on('after:screenshot')
handler have worked previously but is not anymore since cypress@10.10.0
? If so, we need to file an issue request on cypress repo.
Can you confirm that the "path replacement" that was implemented in a return method of
on('after:screenshot')
handler have worked previously but is not anymore sincecypress@10.10.0
?
I’ll reproduce this issue on a blank project and let you know soon
Turns out there was a problem with custom defined on
event binder which had no support for return values.
Cypress has a limitation of adding a single event handler per event. Since your plugin adds custom 'before:browser:launch' handler, you should consider exporting it.
Apparently, mochawesome reporter can't proceed screenshots outside the report folder. And this plugin currently moves screenshots outside screenshots folder.
It seems like option to disable "path replacement" functionality and slight modification of the comparison logic could do the thing. @FRSgit WDYT?
I found a way to create an adaptor for mochawesome report. But in order to effortlessly maintain it the ability to modify imgPath is needed. Would you mind consider accepting this PR.
Are there any updates on this issue?
Any update on this issue?
Any word on a solution?
Hey @FRSgit, just wondering if you have an update on this.
We're currently using uktrade/cypress-image-diff to perform VR testing for our web app, but the inability to update baseline images all at once is slowing down our development times.
Looking at the other plugins available, this one seems to support the functionality we need. However, as we're a large development team with a range of OS's and browsers, we run VR testing in the pipeline to ensure consistency. This means that producing a diff report is a key feature for us.
Either a native report talked about in discussion #96, or the ability to pull in screenshots through a third-party plugin like mochawesome.
Do you have a rough ETA of when either of these would be implemented, or a workaround we can use in the meantime?
Thanks, Alex
Hey everyone!
The newest status is that mochawesome most probably contains a bug, but it hasn't been pursued.
Priorities (from my perspective):
Describe the bug
Diff images are not being included into mochawesome reports. The directory to place the images is being created, though the path doesn't look right.
To Reproduce Steps to reproduce the behavior:
module.exports = defineConfig({ reporter: 'cypress-mochawesome-reporter', e2e: { baseUrl: 'https://your.website.test/', setupNodeEvents(on, config) { require('cypress-mochawesome-reporter/plugin')(on); visualRegression.initPlugin(on, config); }, }, });
reporter: 'cypress-mochawesome-reporter', reporterOptions: { embeddedScreenshots: true, },
Start generate report process Read and merge jsons from "{root_path}/cypress/reports/html/.jsons" Enhance report An error was thrown in your plugins file while executing the handler for the after:run event.
The error we received was:
Error: ENOENT: no such file or directory, open '{root_path}/cypress/screenshots{root_path}/cypress/integration/__image_snapshots__/Error page should validate there are no visual changes #0.actual.png' at Object.openSync (node:fs:585:3) at Object.readFileSync (node:fs:453:35) at convertImageToBase64 ({root_path}/node_modules/cypress-mochawesome-reporter/lib/enhanceReport.js:111:52) at {root_path}/node_modules/cypress-mochawesome-reporter/lib/enhanceReport.js:104:11 at Array.map ()
at createScreenshotsContextList ({root_path}/node_modules/cypress-mochawesome-reporter/lib/enhanceReport.js:100:22)
at attachScreenshotsToTestContext ({root_path}/node_modules/cypress-mochawesome-reporter/lib/enhanceReport.js:76:9)
at {root_path}/node_modules/cypress-mochawesome-reporter/lib/enhanceReport.js:34:24
at Array.forEach ()
at {root_path}/node_modules/cypress-mochawesome-reporter/lib/enhanceReport.js:32:19
at Array.forEach ()
at attachScreenshotsToSuiteTestsContext ({root_path}/node_modules/cypress-mochawesome-reporter/lib/enhanceReport.js:31:12)
at {root_path}/node_modules/cypress-mochawesome-reporter/lib/enhanceReport.js:16:7
at Array.forEach ()
at enhanceReport ({root_path}/node_modules/cypress-mochawesome-reporter/lib/enhanceReport.js:6:18)
at mergeAndCreate ({root_path}/node_modules/cypress-mochawesome-reporter/lib/generateReport.js:19:3)
at async Promise.all (index 0)
at async generateReport ({root_path}/node_modules/cypress-mochawesome-reporter/lib/generateReport.js:62:22)
at async afterRunHook ({root_path}/node_modules/cypress-mochawesome-reporter/lib/index.js:35:3)
at async Object.handler ({root_path}/node_modules/cypress-mochawesome-reporter/plugin.js:9:5)