alecmestroni / cypress-xray-junit-reporter

Npm module for cypress and xray
MIT License
0 stars 0 forks source link

Report generation failing if there are failed tests in subfolders #1

Closed usharkollu closed 2 months ago

usharkollu commented 7 months ago

The reporter plugin is not able to look at subfolders for screenshots. My folder structure is like this: Cypress/e2e/folder/subfolder/test.js. If test.js fails, then xml file is not generated,
ENOENT: no such file or directory, open '.../cypress/screenshots/test.js/Testing Reporting -- testcase (failed).png' ' Cypress saves the screenshot by default at ..../cypress/screenshots/folder/subfolder/test.js/Testing Reporting -- testcase (failed).png

Note: If my test is directly under e2e folder and if the test fails, then there is no issue. junit report is created successfully. But in reality, i need to have folders and subfolders to manage my test cases.

This is how i configured reporter options in cypress.config.js:

    reporter: 'cypress-xray-junit-reporter',
    reporterOptions: {
      mochaFile: './reports/junit_report_[hash].xml',
      xrayMode: true, 
      attachScreenshot: true, 
      shortenLogMode: true,
    },
alecmestroni commented 7 months ago

@usharkollu first of all thanks for you feedback.. unfortunately I'm also working with the subfolder and I have no problems Could you upload a sample repository to github so we can reproduce the bug? And some other info are needed: OS: BROWSER: CYPRESS VERSION: Thank you Alec

usharkollu commented 7 months ago

I recreated the error with this small repo. Can you please try this. https://github.com/usharkollu/cypress_xray_junit

alecmestroni commented 7 months ago

Thanks for provinding a repo Try using chrome or other browser whit "cypress run - b chrome" Because electron is not supported for schreenshot import. (due to a cypress bug that shorten the schreenshot path, deleting subfolders)

usharkollu commented 7 months ago

I tried running with chrome browser and still the same error. Cypress: 13.7.3 │ │ Browser: Chrome 123 (headless) │ │ Node Version: v21.6.1 (/opt/homebrew/Cellar/node/21.6.1/bin/node) │ │ Specs: 2 found (anotherSubfolder/assert_test.js, subfolder/asser_another_test.js) │ │ Searched: cypress/e2e/*/.js

alecmestroni commented 6 months ago

Thank you for providing the repository. I've successfully reproduced the bug and identified the issue.

I was able make everything works as expected, just running the command: "runtests": "npx cypress run -s \"cypress/e2e/folder/anotherSubfolder/assert_test.cy.js\""

However, the problem arises when Cypress runs particular scripts like: "runtests": "npx cypress run -s \"cypress/e2e/*/.cy.js\"" creating subfolders also when creating screenshots.

I'm not entirely sure why this is happening, but it seems to be the root cause of the script failure.

I'll attempt to address this issue on my end.

alecmestroni commented 6 months ago

@usharkollu fixed in cypress-xray-junit-reporter@1.1.5 Please let me know

usharkollu commented 6 months ago

I tried running the tests with 1.1.5. I still see the issue. There is an extra slash but the subfolder names are missing and that is causing an error saying no file exists. This is what the new error says: .../cypress_xray_junit/cypress/screenshots//test -- testcase (failed).png' Error: ENOENT: no such file or directory, open '.../cypress_xray_junit/cypress/screenshots//test -- testcase (failed).png'

alecmestroni commented 6 months ago

Which operating system are you running the tests on?

usharkollu commented 6 months ago

MacBook Pro, macOs Sonoma v14.4.1

alecmestroni commented 6 months ago

@usharkollu Unfortunately, Cypress handles folder paths for screenshots in a rather peculiar manner.

For example, with a directory structure like this:

Cypress
└── e2e
    └── mainFolder
        ├── subfolder1
        └── subfolder2

The screenshots end up being stored under ...screenshot/folder1/.. and ...screenshot/folder2/.. bypassing the mainFolder

However, I've managed to find a workaround for this issue, and it should be resolved in version 1.1.7 (cypress-xray-junit-reporter@1.1.7).

Please let me know if you encounter any further issues, and keep me updated on your progress.

usharkollu commented 6 months ago

It worked with 1.1.7. Now with out abruptly ending, a proper report is generated with a screenshot. Thank you very much.