LironEr / cypress-mochawesome-reporter

Zero config Mochawesome reporter for Cypress with screenshots and videos
MIT License
161 stars 51 forks source link

Report doesn't show screenshots with embeddedScreenshots and inlineAssets configuration. #201

Open izaac opened 2 weeks ago

izaac commented 2 weeks ago

Environment

- OS: Ubuntu `20.04`
- Node: `16.20.2`
- cypress-mochawesome-reporter: see below.
- cypress: `11.2.0`
info Direct dependencies"
├─ cypress-commands@3.0.0"
├─ cypress-mochawesome-reporter@3.8.2"
├─ cypress-multi-reporters@1.6.4"
├─ junit-report-merger@7.0.0"
├─ mocha-junit-reporter@2.2.1"
├─ mocha@10.7.3"
├─ mochawesome-merge@4.3.0"
└─ mochawesome-report-generator@6.2.0"

What happened?

I am using the multi-reporter. jUnit works fine. I am not handling images there and is out of scope here but I am mentioning just for informative purposes.

Using only charts: true I end up with the report index.html alongside an screenshots folder. No Images linked. Using inlineAssets: true as the config only the outcome is: just index.html no screenshots folder neither images in the report. As noted, inlineAssets: true and embeddedScreenshots: true no images in the html and no screenshots in the report folder. Using inlineAssets: false and embbeddedScreenshots: true Same no images in report html neither screenshots folder.

I've been trying a few combinations and also placing everything on their default expected folders like in the case of Cypress screenshots in cypress/screenshots. I am using a non-default cypress config, which filled in the Config file section below.

I've tested a few other ReportOptions combinations. The commit history can be checked here.

I'm not sure if I am missing something to get the html standalone report with images embedded (base64) or at least with the images linked from the screenshots folder. Extra info below.

Thanks in advance.


This is how I am running Cypress which is in a cypress-docker factory docker image (part of a script cypress.sh):

yarn add -W mocha cypress-mochawesome-reporter cypress-multi-reporters cypress-commands \
  mochawesome-report-generator mochawesome-merge mocha-junit-reporter junit-report-merger

NO_COLOR=1 CYPRESS_grepTags="CYPRESSTAGS" cypress run --browser chrome --config-file cypress.config.jenkins.ts

echo "CYPRESS EXIT CODE: $?"

The e2e.ts is:

import '@cypress/code-coverage/support';
import './commands/commands';
import './commands/chainable';
import './commands/rancher-api-commands';
import registerCypressGrep from '@cypress/grep/src/support';
import { addCustomCommand } from 'cypress-delete-downloads-folder';
import 'cypress-mochawesome-reporter/register';

registerCypressGrep();
addCustomCommand();

// TODO handle redirection errors better?
// we see a lot of 'error navigation cancelled' uncaught exceptions that don't actually break anything; ignore them here
Cypress.on('uncaught:exception', (err, runnable) => {
  // returning false here prevents Cypress from failing the test
  if (err.message.includes('navigation guard')) {
    return false;
  }
});

Config file

/* eslint-disable no-console */
import { defineConfig } from 'cypress';
import { getSpecPattern } from '@/scripts/cypress';
import { removeDirectory } from 'cypress-delete-downloads-folder';
// Required for env vars to be available in cypress
require('dotenv').config();

/**
 * VARIABLES
 */
// const testDirs = ['priority', 'components', 'setup', 'pages', 'navigation', 'global-ui', 'features', 'extensions'];
const skipSetup = process.env.TEST_SKIP?.includes('setup');
const baseUrl = (process.env.TEST_BASE_URL || 'https://localhost:8005').replace(/\/$/, '');
const DEFAULT_USERNAME = 'admin';
const username = process.env.TEST_USERNAME || DEFAULT_USERNAME;
const apiUrl = process.env.API || (baseUrl.endsWith('/dashboard') ? baseUrl.split('/').slice(0, -1).join('/') : baseUrl);

/**
 * LOGS:
 * Summary of the environment variables that we have detected (or are going ot use)
 * We won't show any passwords
 */
console.log('E2E Test Configuration');
console.log('');
console.log(`    Username: ${ username }`);

if (!process.env.CATTLE_BOOTSTRAP_PASSWORD && !process.env.TEST_PASSWORD) {
  console.log(' ❌ You must provide either CATTLE_BOOTSTRAP_PASSWORD or TEST_PASSWORD');
}
if (process.env.CATTLE_BOOTSTRAP_PASSWORD && process.env.TEST_PASSWORD) {
  console.log(' ❗ If both CATTLE_BOOTSTRAP_PASSWORD and TEST_PASSWORD are provided, the first will be used');
}
if (!skipSetup && !process.env.CATTLE_BOOTSTRAP_PASSWORD) {
  console.log(' ❌ You must provide CATTLE_BOOTSTRAP_PASSWORD when running setup tests');
}
if (skipSetup && !process.env.TEST_PASSWORD) {
  console.log(' ❌ You must provide TEST_PASSWORD when running the tests without the setup tests');
}

console.log(`    Setup tests will ${ skipSetup ? 'NOT' : '' } be run`);
console.log(`    Dashboard URL: ${ baseUrl }`);
console.log(`    Rancher API URL: ${ apiUrl }`);

// Check API - sometimes in dev, you might have API set to a different system to the base url - this won't work
// as the login cookie will be for the base url and any API requests will fail as not authenticated
if (apiUrl && !baseUrl.startsWith(apiUrl)) {
  console.log('\n ❗ API variable is different to TEST_BASE_URL - tests may fail due to authentication issues');
}

console.log('');

/**
 * CONFIGURATION
 */
export default defineConfig({
  projectId:             process.env.TEST_PROJECT_ID,
  defaultCommandTimeout: process.env.TEST_TIMEOUT ? +process.env.TEST_TIMEOUT : 10000,
  trashAssetsBeforeRuns: true,
  chromeWebSecurity:     false,
  retries:               {
    runMode:  2,
    openMode: 0
  },
  env: {
    grepFilterSpecs:     true,
    grepOmitFiltered:    true,
    baseUrl,
    api:                 apiUrl,
    username,
    password:            process.env.CATTLE_BOOTSTRAP_PASSWORD || process.env.TEST_PASSWORD,
    bootstrapPassword:   process.env.CATTLE_BOOTSTRAP_PASSWORD,
    grepTags:            process.env.GREP_TAGS,
    // the below env vars are only available to tests that run in Jenkins
    awsAccessKey:        process.env.AWS_ACCESS_KEY_ID,
    awsSecretKey:        process.env.AWS_SECRET_ACCESS_KEY,
    azureSubscriptionId: process.env.AZURE_AKS_SUBSCRIPTION_ID,
    azureClientId:       process.env.AZURE_CLIENT_ID,
    azureClientSecret:   process.env.AZURE_CLIENT_SECRET,
    customNodeIp:        process.env.CUSTOM_NODE_IP,
    customNodeKey:       process.env.CUSTOM_NODE_KEY
  },
  // Jenkins reporters configuration jUnit and HTML
  //   screenshotsFolder: 'cypress/screenshots',
  reporter:        'cypress-multi-reporters',
  reporterOptions: {
    reporterEnabled:                   'cypress-mochawesome-reporter, mocha-junit-reporter',
    mochaJunitReporterReporterOptions: {
      mochaFile:      'cypress/jenkins/reports/junit/junit-[hash].xml',
      toConsole:      true,
      jenkinsMode:    true,
      includePending: true
    },
    cypressMochawesomeReporterReporterOptions: {
    //   reportDir:           'cypress/jenkins/reports/mochawesome',
      reportPageTitle:     'ui-tests',
      embeddedScreenshots: true,
      inlineAssets:        true,
      saveAllAttempts:     false,
      charts:              false,
      debug:               true
    },
  },
  e2e: {
    setupNodeEvents(on, config) {
      require('cypress-mochawesome-reporter/plugin')(on);
      require('@cypress/grep/src/plugin')(config);
      on('task', { removeDirectory });
    },
    fixturesFolder:               'cypress/e2e/blueprints',
    experimentalSessionAndOrigin: true,
    // specPattern:                  getSpecPattern(testDirs, process.env),
    specPattern:                  [`cypress/e2e/tests/pages/fleet/**/*.spec.ts`],
    baseUrl
  },
  video:               false,
  videoCompression:    25,
  videoUploadOnPasses: false,
});

Relevant log output

No response

Anything else?

The Dockerfile using the cypress/factory image:

FROM cypress/factory

ARG KUBECTL_VERSION=v1.29.8
ARG CURL_VERSION=v8.6.0

RUN wget https://storage.googleapis.com/kubernetes-release/release/$KUBECTL_VERSION/bin/linux/amd64/kubectl && \
    mv kubectl /bin/kubectl && \
    chmod +x /bin/kubectl
RUN mkdir -p /root/.kube

RUN wget https://github.com/moparisthebest/static-curl/releases/download/$CURL_VERSION/curl-amd64 && \
    mv curl-amd64 /bin/curl && \
    chmod +x /bin/curl

COPY dashboard/imported_config /root/.kube/config

ENTRYPOINT ["bash", "dashboard/cypress/jenkins/cypress.sh"]
LironEr commented 5 days ago

Add debug flag and add the log file the reporter creates to here, it will have more information that might help.

It would be best if you create a repo with the problem and steps to reproduce the issue so I can have a look.

Thanks.