search

LironEr / cypress-mochawesome-reporter

Zero config Mochawesome reporter for Cypress with screenshots and videos
MIT License
159 stars 49 forks source link

Fail to include screenshots in report with cypress > 12 #138

Closed laurentfirdion closed 1 year ago

laurentfirdion commented 1 year ago

Environment

- OS: Ubuntu 22.04
- Node: 16.19.1
- cypress-mochawesome-reporter: 3.3.0
- cypress: 12.9.0

What happened?

The report result after a cypress run is well generated by it is missing screenshots of failed tests. I tried with embeddedScreenshots: true, didn't help. No error appear in the logs

Config file

export default defineConfig({
    video: false,
    watchForFileChanges: false,
    defaultCommandTimeout: 10000,
    screenshotOnRunFailure: true,
    e2e: {
        setupNodeEvents(on, config) {
            require('cypress-mochawesome-reporter/plugin')(on)
        },
    },
    viewportHeight: 760,
    viewportWidth: 1550,
    reporter: 'cypress-mochawesome-reporter',
})

Relevant log output

Start generate report process
Read and merge jsons from "/home/firdion/Private/projects/weasel-nuxt/cypress/reports/html/.jsons"
Copy screenshots folder from "/home/firdion/Private/projects/weasel-nuxt/cypress/screenshots" to "/home/firdion/Private/projects/weasel-nuxt/cypress/reports/html/screenshots"
Enhance report
Create HTML report
HTML report successfully created!
/home/firdion/Private/projects/weasel-nuxt/cypress/reports/html/index.html

Anything else?

No response

joakim-sch commented 1 year ago

Hi @laurentfirdion Does the report simply not contain screenshots at all or are the placeholder images there with broken links? If so, can you see what the URL in the broken image link is? Also, could you add the following to your config:

reporterOptions: {
    debug: true
  }

then run the test again. Share the outputted cypress-mochawesome-reporter.log logfile.

laurentfirdion commented 1 year ago

Hello, the report simply doesn't contain screenshots (despite the fact there are well generated and well copied) Here is the outputted cypress-mochawesome-reporter.log logfile (expunged from private information.) There are no clear error other than maybe the "no context" info

start cypress-mochawesome-reporter 3.3.0
cwd: /home/firdion/Private/projects/weasel-nuxt/cypress/config
cypress.json: {"video":false,"watchForFileChanges":false,"defaultCommandTimeout":10000,"screenshotOnRunFailure":true,"env":{"type":"dev","apiUrl":"https://xxxxxx/api","bearer":"xxx","hideXHRInCommandLog":true,"email":"xxxx@xxxx.com","password":"xxx"},"viewportHeight":760,"viewportWidth":1550,"reporter":"cypress-mochawesome-reporter","reporterOptions":{"debug":true,"reportDir":"/home/firdion/Private/projects/weasel-nuxt/cypress/reports/html"},"baseUrl":"https://xxxx/v2/#","testIsolation":false,"specPattern":"cypress/e2e/test_cases/**/*.cy.{js,jsx,ts,tsx}","setupNodeEvents":"[Function setupNodeEvents]","projectRoot":"/home/firdion/Private/projects/weasel-nuxt","projectName":"weasel-nuxt","repoRoot":"/home/firdion/Private/projects/weasel-nuxt","rawJson":{"video":false,"watchForFileChanges":false,"defaultCommandTimeout":10000,"screenshotOnRunFailure":true,"e2e":{"baseUrl":"https://xxx/v2/#","testIsolation":false,"specPattern":"cypress/e2e/test_cases/**/*.cy.{js,jsx,ts,tsx}","setupNodeEvents":"[Function setupNodeEvents]"},"env":{"type":"dev","apiUrl":"xxx/api","bearer":"xxx","hideXHRInCommandLog":true,"email":"xxx@xxxx.xx","password":"xxxx"},"viewportHeight":760,"viewportWidth":1550,"reporter":"cypress-mochawesome-reporter","reporterOptions":{"debug":true},"baseUrl":"https://xxx/v2/#","testIsolation":false,"specPattern":"cypress/e2e/test_cases/**/*.cy.{js,jsx,ts,tsx}","setupNodeEvents":"[Function setupNodeEvents]","envFile":{},"projectRoot":"/home/firdion/Private/projects/weasel-nuxt","projectName":"weasel-nuxt","repoRoot":"/home/firdion/Private/projects/weasel-nuxt"},"configFile":"dev.cypress.config.ts","morgan":false,"isTextTerminal":true,"socketId":"zgzakbqpgq","report":true,"animationDistanceThreshold":5,"arch":"x64","blockHosts":null,"chromeWebSecurity":true,"clientCertificates":[],"downloadsFolder":"/home/firdion/Private/projects/weasel-nuxt/cypress/downloads","execTimeout":60000,"experimentalFetchPolyfill":false,"experimentalInteractiveRunEvents":false,"experimentalRunAllSpecs":false,"experimentalMemoryManagement":false,"experimentalModifyObstructiveThirdPartyCode":false,"experimentalSkipDomainInjection":null,"experimentalOriginDependencies":false,"experimentalSourceRewriting":false,"experimentalSingleTabRunMode":false,"experimentalStudio":false,"experimentalWebKitSupport":false,"fileServerFolder":"/home/firdion/Private/projects/weasel-nuxt","fixturesFolder":"/home/firdion/Private/projects/weasel-nuxt/cypress/fixtures","excludeSpecPattern":"*.hot-update.js","includeShadowDom":false,"keystrokeDelay":0,"modifyObstructiveCode":true,"numTestsKeptInMemory":0,"platform":"linux","pageLoadTimeout":60000,"port":37847,"projectId":null,"redirectionLimit":20,"requestTimeout":5000,"resolvedNodePath":"/usr/local/bin/node","resolvedNodeVersion":"16.19.1","responseTimeout":30000,"retries":{"runMode":0,"openMode":0},"screenshotsFolder":"/home/firdion/Private/projects/weasel-nuxt/cypress/screenshots","slowTestThreshold":10000,"scrollBehavior":"top","supportFile":"/home/firdion/Private/projects/weasel-nuxt/cypress/support/e2e.ts","supportFolder":"/home/firdion/Private/projects/weasel-nuxt/cypress/support","taskTimeout":60000,"trashAssetsBeforeRuns":true,"userAgent":null,"videoCompression":32,"videosFolder":"/home/firdion/Private/projects/weasel-nuxt/cypress/videos","videoUploadOnPasses":true,"waitForAnimations":true,"additionalIgnorePattern":[],"autoOpen":false,"browsers":[{"name":"chrome","family":"chromium","channel":"stable","displayName":"Chrome","version":"110.0.5481.177","path":"google-chrome","minSupportedVersion":64,"majorVersion":"110"},{"name":"electron","channel":"stable","family":"chromium","displayName":"Electron","version":"106.0.5249.51","path":"","majorVersion":106}],"clientRoute":"/__/","cypressBinaryRoot":"/home/firdion/.cache/Cypress/12.9.0/Cypress/resources/app","devServerPublicPathRoute":"/__cypress/src","hosts":null,"isInteractive":true,"namespace":"__cypress","reporterRoute":"/__cypress/reporter","socketIoCookie":"__socket","socketIoRoute":"/__socket","version":"12.9.0","cypressEnv":"production","resolved":{"animationDistanceThreshold":{"value":5,"from":"default"},"arch":{"value":"x64","from":"default"},"baseUrl":{"value":"https://xxx/v2/#","from":"config"},"blockHosts":{"value":null,"from":"default"},"chromeWebSecurity":{"value":true,"from":"default"},"clientCertificates":{"value":[],"from":"default"},"defaultCommandTimeout":{"value":10000,"from":"config"},"downloadsFolder":{"value":"cypress/downloads","from":"default"},"env":{"type":{"value":"dev","from":"config"},"apiUrl":{"value":"https://xxx/api","from":"config"},"bearer":{"value":"xxx","from":"config"},"hideXHRInCommandLog":{"value":true,"from":"config"},"email":{"value":"xxxx@xxxx.xx","from":"config"},"password":{"value":"xxx","from":"config"},"psk":{"value":"xxx","from":"config"},"LOCATION_IQ_TOKEN":{"value":"xxx","from":"config"}},"execTimeout":{"value":60000,"from":"default"},"experimentalFetchPolyfill":{"value":false,"from":"default"},"experimentalInteractiveRunEvents":{"value":false,"from":"default"},"experimentalRunAllSpecs":{"value":false,"from":"default"},"experimentalMemoryManagement":{"value":false,"from":"default"},"experimentalModifyObstructiveThirdPartyCode":{"value":false,"from":"default"},"experimentalSkipDomainInjection":{"value":null,"from":"default"},"experimentalOriginDependencies":{"value":false,"from":"default"},"experimentalSourceRewriting":{"value":false,"from":"default"},"experimentalSingleTabRunMode":{"value":false,"from":"default"},"experimentalStudio":{"value":false,"from":"default"},"experimentalWebKitSupport":{"value":false,"from":"default"},"fileServerFolder":{"value":"","from":"default"},"fixturesFolder":{"value":"cypress/fixtures","from":"default"},"excludeSpecPattern":{"value":"*.hot-update.js","from":"default"},"includeShadowDom":{"value":false,"from":"default"},"keystrokeDelay":{"value":0,"from":"default"},"modifyObstructiveCode":{"value":true,"from":"default"},"nodeVersion":{"from":"default"},"numTestsKeptInMemory":{"value":0,"from":"config"},"platform":{"value":"linux","from":"default"},"pageLoadTimeout":{"value":60000,"from":"default"},"port":{"value":null,"from":"default"},"projectId":{"value":null,"from":"default"},"redirectionLimit":{"value":20,"from":"default"},"reporter":{"value":"cypress-mochawesome-reporter","from":"config"},"reporterOptions":{"value":{"debug":true},"from":"config"},"requestTimeout":{"value":5000,"from":"default"},"resolvedNodePath":{"value":null,"from":"default"},"resolvedNodeVersion":{"value":null,"from":"default"},"responseTimeout":{"value":30000,"from":"default"},"retries":{"value":{"runMode":0,"openMode":0},"from":"default"},"screenshotOnRunFailure":{"value":true,"from":"default"},"screenshotsFolder":{"value":"cypress/screenshots","from":"default"},"slowTestThreshold":{"value":10000,"from":"default"},"scrollBehavior":{"value":"top","from":"default"},"supportFile":{"value":"cypress/support/e2e.{js,jsx,ts,tsx}","from":"default"},"supportFolder":{"value":false,"from":"default"},"taskTimeout":{"value":60000,"from":"default"},"testIsolation":{"value":false,"from":"config"},"trashAssetsBeforeRuns":{"value":true,"from":"default"},"userAgent":{"value":null,"from":"default"},"video":{"value":false,"from":"config"},"videoCompression":{"value":32,"from":"default"},"videosFolder":{"value":"cypress/videos","from":"default"},"videoUploadOnPasses":{"value":true,"from":"default"},"viewportHeight":{"value":760,"from":"config"},"viewportWidth":{"value":1550,"from":"config"},"waitForAnimations":{"value":true,"from":"default"},"watchForFileChanges":{"value":false,"from":"config"},"specPattern":{"value":"cypress/e2e/test_cases/**/*.cy.{js,jsx,ts,tsx}","from":"config"},"browsers":{"value":[{"name":"chrome","family":"chromium","channel":"stable","displayName":"Chrome","version":"110.0.5481.177","path":"google-chrome","minSupportedVersion":64,"majorVersion":"110"},{"name":"electron","channel":"stable","family":"chromium","displayName":"Electron","version":"106.0.5249.51","path":"","majorVersion":106,"isHeadless":true,"isHeaded":false}],"from":"runtime"},"hosts":{"value":null,"from":"default"},"isInteractive":{"value":true,"from":"default"}},"testingType":"e2e","remote":{"origin":"https://xxxx.xx","strategy":"http","fileServer":null,"domainName":"xxx.xx","props":{"port":"443","protocol":"https:","subdomain":"lf","domain":"xx","tld":"xx"}},"browser":null,"specs":[],"proxyUrl":"http://localhost:37847","browserUrl":"https://xxxx.xx/__/","reporterUrl":"https://xxxx.xx/__cypress/reporter","proxyServer":"http://localhost:37847","state":{}}
simple config: {"jsonDir":"/home/firdion/Private/projects/weasel-nuxt/cypress/reports/html/.jsons","reporterOptions":{"debug":true,"reportDir":"/home/firdion/Private/projects/weasel-nuxt/cypress/reports/html"},"screenshotsDir":"/home/firdion/Private/projects/weasel-nuxt/cypress/screenshots","outputDir":"/home/firdion/Private/projects/weasel-nuxt/cypress/reports/html"}
Remove output folder /home/firdion/Private/projects/weasel-nuxt/cypress/reports/html
Start generate report process
Read and merge jsons from "/home/firdion/Private/projects/weasel-nuxt/cypress/reports/html/.jsons"
Copy screenshots folder from "/home/firdion/Private/projects/weasel-nuxt/cypress/screenshots" to "/home/firdion/Private/projects/weasel-nuxt/cypress/reports/html/screenshots"
report before enhance: {}
Enhance report
attach screenshots for test "Testing Login page with credentials Loading the login page"
no context
attach screenshots for test "Creating a recruitment with a non authorized user Loads as a manager on /recruitments"
no context
attach screenshots for test "Creating Recruitment and Job Offer Set default user and go to /recruitments"
no context
attach screenshots for test "Creating Recruitment and Job Offer Creating new recruitment"
no context
attach screenshots for test "Creating Recruitment and Job Offer Creating new job Offer"
no context
attach screenshots for test "Creating Recruitment and Job Offer Filling form"
no context
attach screenshots for test "Creating Recruitment and Job Offer Going to step 2"
no context
attach screenshots for test "Creating Recruitment and Job Offer Going to step 3"
no context
attach screenshots for test "Creating Recruitment and Job Offer Adding half time; Going to step 4"
no context
attach screenshots for test "Creating Recruitment and Job Offer Publish Job Offer"
no context
attach screenshots for test "Creating Recruitment and Job Offer Validate Job Offer Creation"
no context
attach screenshots for test "Creating Recruitment and Job Offer Verify create job is in list"
no context
attach screenshots for test "Creating a recruitment with a non authorized user Loads as a manager on /recruitments"
no context
report after enhance: {"stats":{"suites":4,"tests":13,"passes":6,"pending":0,"failures":7,"testsRegistered":13,"passPercent":46.15384615384615,"pendingPercent":0,"other":0,"hasOther":false,"skipped":0,"hasSkipped":false,"start":"2023-04-17T08:37:04.194Z","end":"2023-04-17T08:38:43.543Z","duration":99349},"results":[{"uuid":"dfabb2d6-6400-45f5-84f6-806b8a0a6bd7","title":"","fullFile":"cypress/e2e/test_cases/login/login.cy.ts","file":"cypress/e2e/test_cases/login/login.cy.ts","beforeHooks":[],"afterHooks":[],"tests":[],"suites":[{"uuid":"d50d82ed-dc04-4b48-934b-a4c27a146bff","title":"Testing Login page with credentials"......., "meta":{"mocha":{"version":"7.0.1"},"mochawesome":{"options":{"quiet":false,"reportFilename":"mochawesome","saveHtml":false,"saveJson":true,"consoleReporter":"spec","useInlineDiffs":false,"code":true},"version":"7.1.3"},"marge":{"options":{"debug":true,"overwrite":false,"html":false,"json":true,"reportDir":"cypress/reports/html/.jsons"},"version":"6.2.0"}}}
Create HTML report
HTML result: ["/home/firdion/Private/projects/weasel-nuxt/cypress/reports/html/index.html",null]
HTML report successfully created!
/home/firdion/Private/projects/weasel-nuxt/cypress/reports/html/index.html
done
joakim-sch commented 1 year ago

Yeah, this is the issue; report before enhance: {} This happens when the register.js isn't imported.

Step 3 of the setup in README Add to cypress/support/e2e.js import 'cypress-mochawesome-reporter/register';

laurentfirdion commented 1 year ago

I did included it in my cypress/support/e2e.ts, I found the issue in looking at cypress-mochawesome-reporter/register.js : it uses a deprecated hook Cypress.on('test:after:run', (test) => {}) which is lo longer triggered with Cypress 12.XX I guess it will take some time to implement the new "after run" API (the object returned is really different than the previous one used by your current version.)

joakim-sch commented 1 year ago

I have to admit @laurentfirdion , I am on cypress 12.xx and not seeing this issue. All the tests in this project run on Cypress 12.3.0. My setup runs on 12.7.0 and both have no problems triggering the test:after:run hook and getting the screenshots in the mochawesome report. I also tried updating the tests in this project to 12.9.0 (latest version atm) and had no problems with the hook there either.

I'm also not finding anything about the hook being deprecated in the Cypress documentation, its listed in the catalogue of events. Also, I'm probably just not seeing it, but I can't find anything in the changelog about this hook being disabled after v12. Could you point to the corresponding documentation?

I did notice that you referred to your support file being a typescript file. I did have an issue myself a while ago that Cypress couldn't find the support file because it was looking for e2e.js. You could try adding a reference to your support file in your cypress.config.ts, that did the trick for me atleast 😃 ;

e2e: {
        supportFile: 'cypress/support/e2e.ts',
        setupNodeEvents(on, config) {
            require('cypress-mochawesome-reporter/plugin')(on);
        },
    },
laurentfirdion commented 1 year ago

Thanks for your time, i stand corrected : the event has not been deprecated (i searched "after run" in the cypress documentation and only found the reference of setupNodeEvents(on, config) {on('after:run', (results) => { /* ... */})}, and thought mistakenly it has replaced the after:run event. I tried your work around but that didn't solve the issue. The problem is narrowed to the fact that none of my Cypress.on('test:after:run',{}) are triggered even if i put one at the top of my e2e.ts file with just a console.log inside. (I know the rest of the e2e.ts file is working fine because it has commands that are well used when executing tests)

LironEr commented 1 year ago

If this is still an issue, please create a repo with the problem and steps to reproduce the issue so I can have a look.

Thanks.

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.