Allure libraries are not working with cypress 12.15.0

BarbaraGodayol commented 1 year ago

Current behavior

Currently, when you are using the library "@shelex/cypress-allure-plugin" and configurating cypress with the property "allure = true", at the end of cypress execution, the folder allure-results is not created. In the previous versions, the folder was created with all the data

Desired behavior

In this situacion, cypress has to create a folder "allure-results" with all the data to can be used for other tools to create the real report.

Test code to reproduce

cypress.config.ts:

import setupNodeEvents from './setupNodeEvents';

const dotenv = require('dotenv');
const env = process.env.NODE_ENV?.trim() ?? 'docker';

dotenv.config({ path: `./environment/.env.${env}` });

export default defineConfig({
  e2e: {
    retries: {
      runMode: 0,
      openMode: 0,
    },
    env: {
      api_baseEndpoint: process.env.API_BASE_ENDPOINT,
      web_baseUrl: process.env.WEB_BASE_URL,
      api_password: process.env.API_PASSWORD,
      api_client_secret: process.env.API_CLIENT_SECRET,
      platform_environment: process.env.PLATFORM_ENVIRONMENT,
      db: {
        user: process.env.DB_USER,
        password: process.env.DB_PASSWORD,
        host: process.env.DB_HOST,
        server: process.env.DB_SERVER,
        database: process.env.DB_DATABASE,
        port: process.env.DB_PORT,
        ssl: process.env.DB_SSL
      },
      allure: true,
      allureResultsPath: "allure-results",
      allureReuseAfterSpec: true,
      allureAttachRequests: true,
      allureClearSkippedTests: false,
      allureAddVideoOnPass: false,
      ServiceKey: '#my*S3cr3t'
    },
    specPattern: '**/*.feature',
    chromeWebSecurity: false,
    video: false,
    screenshotOnRunFailure: true,
    defaultCommandTimeout: 8000,
    viewportWidth: 1920,
    viewportHeight: 1080,
    setupNodeEvents
  }
});

Cypress Version

12.15.0

Node version

v18.15.0

Operating System

windows

Debug Logs

No response

Other

No response

mike-plummer commented 1 year ago

Hi @BarbaraGodayol , sorry that you're having trouble. We'll need more information in order to help with this issue:

Are you saying your project using @shelex/cypress-allure-plugin worked on an older version of Cypress and no longer works on upgrading to v12.15.0? Or are you having trouble getting the addon working on a new project using 12.15.0?
Are there any errors reported in the Cypress runner or terminal window? We will need the Debug Logs from your run to diagnose the issue.
Can you provide a reproducible example we can use to troubleshoot? A git repo we can clone and run is preferred over copy-pasted code.

sl-tawfiknouri commented 1 year ago

Hi, I had the same problem with the latest Cypress version 12.15.0. The same project works well with version 12.14.0 and Cypress doesn't report any errors. It seems the latest version has a problem with the Allure report and also doesn't seem to be able to create screenshots and videos automatically.

mike-plummer commented 1 year ago

Thanks for the context @sl-tawfiknouri ! There is a known issue with 12.15.0 that we're looking into that may be causing issues under the covers, so it sounds like a temporary workaround for this is to downgrade to 12.14.0

BarbaraGodayol commented 1 year ago

Hello @mike-plummer, Like sl-tawfiknouri said, cypress is working well in 12.14.0 and about logs, I don' see any error log related with allure

JessefSpecialisterren commented 1 year ago

@mike-plummer Sadly, Cypress 12.16.0 does not seem to fix this issue. Still no folder allure-results after a test run. Downgrading to Cypress 12.14.0 while otherwise keeping the exact same configuration and environment makes things work as expected again

mike-plummer commented 1 year ago

Hi @JessefSpecialisterren , sorry you're still having issues. We addressed an issue that was causing this error in 12.16.0 but it appears you've identified one or more edge cases we missed. Are you able to share a reproduction case we can use to troubleshoot?

samixchoumi commented 1 year ago

Hello @mike-plummer just use "@shelex/cypress-allure-plugin": "2.40.0" and "cypress": "12.15.0" or "cypress": "12.16.0" on your package.json project.

The allure-result folder will not be created. Reproduced on a lot of our project (using API testing or UI testing) / (typescript or javascript).

You can see below one of our project failing after upgrading cypress to 12.15/16 version :

"devDependencies": {
    "@badeball/cypress-cucumber-preprocessor": "^18.0.1",
    "@bahmutov/cypress-esbuild-preprocessor": "^2.2.0",
    "@faker-js/faker": "^8.0.2",
    "@shelex/cypress-allure-plugin": "^2.40.0",
    "cypress": "12.16.0",
    "cypress-cloud": "^1.7.4",
    "cypress-multi-reporters": "^1.6.3",
    "cypress-real-events": "^1.8.1",
    "cypress-recurse": "^1.35.2",
    "cypress-slack-reporter": "^1.5.3",
    "esbuild": "^0.18.10",
    "eslint": "^8.42.0",
    "eslint-plugin-cypress": "^2.13.3",
    "fs-extra": "^11.1.1",
    "mocha": "^10.2.0",
    "mochawesome": "^7.1.3",
    "mochawesome-merge": "^4.3.0",
    "mochawesome-report-generator": "^6.2.0",
    "typescript": "^5.1.3"
  },

In our case we switched to cypress-cloud run instead of cypress run command, but it seem to fail when using cypress run command too.

mike-plummer commented 1 year ago

Reproduction: https://github.com/mike-plummer/cypress-issue-27113

mike-plummer commented 1 year ago

@BarbaraGodayol @JessefSpecialisterren @samixchoumi I have replicated your problem in the latest version of Cypress and will forward this ticket to the appropriate team. They will soon evaluate the priority of this ticket and consider their capacity to pick it up. Please note that this does not guarantee that this issue will be resolved. The ticket will indicate status changes during evaluation, so we ask that you please refrain from asking for updates. Thanks!

casalribeiro commented 1 year ago

Do you have any news on this issue? We're still on v12.14.0 for now.

mysticdevx commented 1 year ago

Hi @JessefSpecialisterren , sorry you're still having issues. We addressed an issue that was causing this error in 12.16.0 but it appears you've identified one or more edge cases we missed. Are you able to share a reproduction case we can use to troubleshoot?

Hi, I have upgraded to v12.17.0 with "@shelex/cypress-allure-plugin": "^2.40.0" and still the problem exists.

dilshan-swivel commented 1 year ago

Done quick test on Cypress version 12.17.1 and @shelex/cypress-allure-plugin: 2.40.0 , Still it is broken

stefan-cooper commented 1 year ago

Checked on 12.17.2 and this issue still persists. Sticking with 12.14.0 for now

Deop commented 1 year ago

Having the same issue, latest working version is 12.14.0. Checked with 12.17.2 and it's not working for me as well.

samixchoumi commented 1 year ago

Having the same issue for 12.17.3. Any news about it @mike-plummer ?

mike-plummer commented 1 year ago

@samixchoumi Unfortunately we are a very small team and can't address every issue immediately. We are always happy to receive contributions from the community if someone is interested in investigating and resolving this issue - see the Contributing Guide for info on setup and process.

The ticket will indicate status changes during evaluation, so we ask that you please refrain from asking for updates. Thanks!

AlexandreTI-AUT commented 1 year ago

Having the same issue for 12.17.3

danfooks commented 1 year ago

I am experiencing the same issue. I downgraded Cypress to 11.0.0 to get Allure reports generating again. I am running 2.40.0 of cypress-allure-plugin.

FerrazLeandro commented 1 year ago

Tendo o mesmo problema para Cypress 12.17.3 e "@shelex/cypress-allure-plugin": "^2.40.0",

Rida55 commented 1 year ago

Okay, this isn't the exact solution, but it's a workaround.Downgrade your Cypress to version "cypress": "^11.0.0" or "cypress": "^12.14.0" by executing 'npm install cypress@11.0.0' OR npm install cypress@^12.14.0 along with the "@shelex/cypress-allure-plugin": "^2.40.0".

amaanbs commented 1 year ago

@mike-plummer Any updates on a fix here? I created a plugin for one of our internal projects & that has been failing since 12.15 (it was working up until version 12.14). I implemented certain event listeners in my support file to listen to cypress command executions wherein I would execute cy.now('task', 'xyz') & I would then listen to this xyz event inside my plugin under setupNodeEvents. This flow doesn't work with Cypress >= v12.15. Posting this here since both Allure & ReportPortal plugins have similar implementations & both seem to be broken since Cypress v12.15. Would appreciate a response. Thanks!

mike-plummer commented 1 year ago

@amaanbs This has not yet been prioritized to be worked by the Cypress team - we would welcome anyone from the community who is interested in taking a look. Information to get started is available in our Contributing Guide.

We ask that everyone please refrain from asking for updates - this GitHub issue will reflect status whenever there are updates to share.

jeremie-val commented 1 year ago

Also waiting here for a fix to upgrade Cypress

lmiller1990 commented 1 year ago

Cy 12.14 and 12.15 diff, only a handful of commits, should not be hard to at least find out what broke: https://github.com/cypress-io/cypress/compare/v12.14.0...v12.15.0

Any error, what's actually "broken" - that would help figure out which commit(s) broke it.

JessefSpecialisterren commented 1 year ago

@lmiller1990 I already did some investigating a while ago and I'm pretty sure https://github.com/cypress-io/cypress/commit/89f0fb64650d1795a912773b5c0660d1aceea0bc is the culprit. cypress-allure-plugin uses cy.now('task' to send report data from the browser to Node, which stopped working after those extra security measures were implemented

We've switched to mochawesome reporting for our project, by the way. The report isn't as good, but at least it works with newer Cypress versions

samixchoumi commented 1 year ago

@JessefSpecialisterren, we switched to sorry-cypress and mochawesome too because of that …

On Thu, Aug 24, 2023 at 09:59 JessefSpecialisterren < @.***> wrote:

@lmiller1990 https://github.com/lmiller1990 I already did some investigating a while ago and I'm pretty sure 89f0fb6 https://github.com/cypress-io/cypress/commit/89f0fb64650d1795a912773b5c0660d1aceea0bc is the culprit. cypress-allure-plugin uses cy.now('task' https://github.com/Shelex/cypress-allure-plugin/blob/8af114383af639be9af1916a7723ab09869a0823/reporter/index.js#L76 to send report data from the browser to Node, which stopped working after those extra security measures were implemented

We've switched to mochawesome reporting for our project, by the way. The report isn't as good, but at least it works with newer Cypress versions

— Reply to this email directly, view it on GitHub https://github.com/cypress-io/cypress/issues/27113#issuecomment-1691198042, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFJFCGBH3MYSFIOMW2ZKVOLXW4CWZANCNFSM6AAAAAAZP667PI . You are receiving this because you were mentioned.Message ID: @.***>

lmiller1990 commented 1 year ago

Sorry Cypress still uses the same binary, just a different cloud service -- switching to that is not going to fix this issue.

samixchoumi commented 1 year ago

Yeah I know but the dashboard is better than allure report

On Thu, Aug 24, 2023 at 23:50 Lachlan Miller @.***> wrote:

Sorry Cypress still uses the same binary, just a different cloud service -- switching to that is not going to fix this issue.

— Reply to this email directly, view it on GitHub https://github.com/cypress-io/cypress/issues/27113#issuecomment-1692459196, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFJFCGDXHXYIMZLQTJLJGS3XW7EC7ANCNFSM6AAAAAAZP667PI . You are receiving this because you were mentioned.Message ID: @.***>

dirkrooijakkers commented 1 year ago

We're having the same issue.

dilshan-swivel commented 1 year ago

Done quick test on Cypress version 13.0.0 and @shelex/cypress-allure-plugin: 2.40.0 , Still it is broken

Tatev777 commented 1 year ago

I have the same issue, cypress version 13.1.0, @shelex/cypress-allure-plugin: 2.40.0 - allure report shows NAN I have downgraded cypress to 12.9.0, here allure works fine.

mariamaslam commented 1 year ago

Hi @mike-plummer is this issue resolved? The allure reports are null even with Cypress 13 and this has not been quite some time now. And I am desperately looking for a fix. Please keep me posted, looking forward to your kind response.

Rogerrenjifo commented 1 year ago

You are right thanks.

Hi, I had the same problem with the latest Cypress version 12.15.0. The same project works well with version 12.14.0 and Cypress doesn't report any errors. It seems the latest version has a problem with the Allure report and also doesn't seem to be able to create screenshots and videos automatically.

AbdJazmawi commented 1 year ago

@mike-plummer is there any updates on this?

mdtmattos commented 1 year ago

Some news about it?

sanukhandev commented 1 year ago

is there a working version i have critical demo suddenly it stopped working I rolled back but stii its not working

this is my current package.json

"@shelex/cypress-allure-plugin": "^2.35.2",
    "allure-commandline": "^2.21.0",
    "cypress": "^12.9.0",

AbdJazmawi commented 1 year ago

is there a working version i have critical demo suddenly it stopped working I rolled back but stii its not working

this is my current package.json
"@shelex/cypress-allure-plugin": "^2.35.2",
    "allure-commandline": "^2.21.0",
    "cypress": "^12.9.0",

@sanukhandev I gave up and used another reporter mochawesome

AndriiZhydkykh commented 1 year ago

@sanukhandev I use cypress@12.14.0 and @shelex/cypress-allure-plugin": "^2.34.0. Also, you can try this plugin https://github.com/mmisty/cypress-allure-adapter. People wrote this plugin works well on all versions of Cypress.

BarbaraGodayol commented 1 year ago

We are using cypress@12.14.0 but we are facing a lot of errors with the new chrome. We are trying to use the new plugin https://github.com/mmisty/cypress-allure-adapter. I'll tell you if it is correct for us.

hasanalpzengin commented 1 year ago

@lmiller1990 I already did some investigating a while ago and I'm pretty sure 89f0fb6 is the culprit. cypress-allure-plugin uses cy.now('task' to send report data from the browser to Node, which stopped working after those extra security measures were implemented

We've switched to mochawesome reporting for our project, by the way. The report isn't as good, but at least it works with newer Cypress versions

The culprit seems to be correct to me. The code now follows the path to runPrivilegedCommand. And cy.state('current') returns null. However I could not find any documentation about cy.state so I am a bit lost here. Thinking that maybe it's related with allure plugin relies on "after:run" hook ?

sanukhandev commented 1 year ago

i have demo this week so i cannot try a new plugin now

@sanukhandev I use cypress@12.14.0 and @shelex/cypress-allure-plugin": "^2.34.0. Also, you can try this plugin https://github.com/mmisty/cypress-allure-adapter. People wrote this plugin works well on all versions of Cypress.

for now I have to use the same settings and plugin I have a demo this week now i have set 12.9.0 on a controlled environment the reports seems to be generating fine now

sanukhandev commented 1 year ago

(Run Starting)

┌────────────────────────────────────────────────────────────────────────────────────────────────┐ │ Cypress: 12.9.0 │ │ Browser: Electron 106 (headless) │ │ Node Version: v16.15.0 (C:\Program Files\nodejs\node.exe) │ │ Specs: 1 found (sample.spec.js) │ │ Searched: cypress/integration/*/.spec.js │ └────────────────────────────────────────────────────────────────────────────────────────────────┘

using this version on a controlled environment inside docker

vallme2003 commented 1 year ago

Any updates on this or any workaround except for downgrading the version?

andrei9669 commented 1 year ago

hello, I'm kinda stuck between hard place and a rock right now. there's a bug that was fixed in v13 but I can't update due to this bug

Edd1525 commented 1 year ago

Any updates?

mmisty commented 1 year ago

Hi, The ones who struggles - I think you can try reporting an issue in shelex/cypress-allure-plugin repo. It can be fixed/workarounded faster in the plugin repo. Or you can use mmisty/cypress-allure-adapter (I'm the author). it should be almost backward compatible with shelex's plugin. Cypress-allure-adapter has improvements for before/after/before all/after all hooks and steps improvements (can be controlled by env vars) Plugin continuously improves.

vallme2003 commented 1 year ago

Hi, The ones who struggles - I think you can try reporting an issue in shelex/cypress-allure-plugin repo. It can be fixed/workarounded faster in the plugin repo. Or you can use mmisty/cypress-allure-adapter (I'm the author). it should be almost backward compatible with shelex's plugin. Cypress-allure-adapter has improvements for before/after/before all/after all hooks and steps improvements (can be controlled by env vars) Plugin continuously improves.

does the project has to be in typescript for cypress-allure-adapter?

mmisty commented 1 year ago

does the project has to be in typescript for cypress-allure-adapter?

@vallme2003, it can be either typescript or javascript

vallme2003 commented 1 year ago

does the project has to be in typescript for cypress-allure-adapter?

@vallme2003, it can be either typescript or javascript

Thanks for the plugin. Had to make a small change for JS style imports ( Used const {configureAllureAdapterPlugins} = require("@mmisty/cypress-allure-adapter/plugins"); instead of import { configureAllureAdapterPlugins } from '@mmisty/cypress-allure-adapter/plugins'; , but pretty much straightforward change to replace the shellex plugin with allure adapter and confirming that the allure adapter works with latest version of cypress

mariamaslam commented 1 year ago

Hi @mmisty Thank you adding this, it is going to be great help. I have added the following script but my reports are still NaN. Can you support

config.js

const { configureAllureAdapterPlugins } = require('@mmisty/cypress-allure-adapter/plugins');

module.exports = {
  e2e: {

    setupNodeEvents(on, config) {
      configureAllureAdapterPlugins(on, config);

      return config;
    },

This is my pacakge.json

"scripts": {
    "tests": "cypress run cypress/e2e/Accounts/*.js && npm run html report",
    "cy:run": "cypress run cypress/e2e/Accounts/*.js — env allure=true",
    "allure:report": "npm run allure generate allure-results — clean -o allure-report && allure open allure-report",
    "beforetest": "npm run clean:folders",
    "test": "npm run cy:run || npm run aftertest",
    "aftertest": "npm run allure:report"
  },
  "author": "Mariam Aslam",
  "license": "Apache-2.0",
  "failOnStatusCode": false,
  "devDependencies": {
    "@bahmutov/cy-api": "^2.2.6",
    "@mmisty/cypress-allure-adapter": "^0.10.7",
    "@shelex/cypress-allure-plugin": "^2.40.0",
    "allure-commandline": "^2.24.0",
    "cypress": "^13.2.0",
    "cypress-mochawesome-reporter": "^3.6.0",
    "cypress-multi-reporters": "^1.6.3",
    "mocha": "^10.2.0",
    "mochawesome": "^7.1.3",
    "mochawesome-merge": "^4.3.0"
  }
}

This is my e2e.js

import './commands'
import '@bahmutov/cy-api';
import 'cypress-mochawesome-reporter/register';
import '@mmisty/cypress-allure-adapter/support';
import '@shelex/cypress-allure-plugin';

I am running the test script "Npm run test" based of the script i have created in package.json but it shows reports as blank

cypress-io / cypress