Open vishallanke opened 4 years ago
It seems agent is not working from Git if test cases gets failed. If test case gets failed, then those launches and attachment wont get uploaded.
I found a launch, when it goes to completed state after 1 day. It has 2 test cases. 1 Pass and 1 failed. Only pass test case gets visible and that after 1 day.
This issue still occurs on latest agent. Any solution ? I am using https://github.com/reportportal/agent-js-cypress [npm install @reportportal/agent-js-cypress] Locally it works fine but not from Git Pipeline. I also tried "@reportportal/agent-js-cypress": "https://github.com/reportportal/agent-js-cypress.git#master"
This issue still exists on latest release. From Git, test cases wont get uploaded to RP.
In previous version (1 month ago), I have observed that if all test cases gets passed, then test script and test cases gets successfully uploaded. If any of the test gets failed, then those test script as well as test cases(passed+failed from test script) wont get uploaded.
However, in this release, I found that irrespective of state of test , it wont get updated.
This issue wont occur if I execute test locally. It occurs only when you execute tests from Gitlab Pipeline.
Tested today on latest release. Report portal hosted on AWS EC2, latest version - STEPS followed are - https://reportportal.io/docs/Deploy-ReportPortal-on
CYPRESS GIT PIPELINE code that triggers test cases
script:
- $(npm bin)/print-env CI
- npm run start:ci &
- npx cypress run
- $(npm bin)/print-env CI
- npx cypress -v
- echo Thanks
Package.json
"dependencies": {
"@bahmutov/print-env": "1.2.0",
"@reportportal/agent-js-cypress": "^5.0.0",
"glob": "7.1.3",
"mocha": "7.0.1",
"node-ipc": "9.1.1"
},
"license": "Apache-2.0",
"devDependencies": {
"cypress": "^4.0.2",
"eslint": "6.5.1",
"eslint-config-airbnb-base": "14.0.0",
"eslint-config-prettier": "6.9.0",
"eslint-plugin-cypress": "2.8.1",
"eslint-plugin-import": "2.20.0",
"eslint-plugin-jest": "^23.8.1",
"eslint-plugin-prettier": "3.1.2",
"jest": "^25.1.0",
"mock-fs": "^4.11.0",
"prettier": "1.19.1"
},
Cypress.json
{
"projectId" : "crossVertical",
"reporter": "@reportportal/agent-js-cypress",
"video" : false,
"reporterOptions": {
"endpoint": "http://dummy.compute.amazonaws.com:8080/api/v1",
"token": "12345678dummy-b074-4a1b-8d72-8a5e55c55f8d",
"launch": "cypress",
"project": "cypress_project",
"description": "END_TO_END_CROSS VERTICAL",
"isLaunchMergeRequired": false,
"autoMerge": false,
"integrationFolder": "cypress/integration",
"screenshotsFolder": "cypress/screenshots",
"fixturesFolder": "cypress/fixtures",
"supportFile": "cypress/support/index.js",
"pluginsFile": "cypress/plugins/index.js"
}
}
LOGS
$ npm run start:ci &
$ npx cypress run
npm ERR! missing script: start:ci
npm ERR! A complete log of this run can be found in:
npm ERR! /builds/Vishal.Lanke/cypress-report-portal-integration/.npm/_logs/2020-06-30T05_02_12_819Z-debug.log
/builds/Vishal.Lanke/cypress-report-portal-integration/cache/Cypress/4.0.2/Cypress/Cypress: /lib/x86_64-linux-gnu/libdbus-1.so.3: no version information available (required by /builds/Vishal.Lanke/cypress-report-portal-integration/cache/Cypress/4.0.2/Cypress/Cypress)
/builds/Vishal.Lanke/cypress-report-portal-integration/cache/Cypress/4.0.2/Cypress/Cypress: /lib/x86_64-linux-gnu/libdbus-1.so.3: no version information available (required by /builds/Vishal.Lanke/cypress-report-portal-integration/cache/Cypress/4.0.2/Cypress/Cypress)
Fontconfig warning: "/etc/fonts/fonts.conf", line 86: unknown element "blank"
/proc/self/exe: /lib/x86_64-linux-gnu/libdbus-1.so.3: no version information available (required by /proc/self/exe)
/proc/self/exe: /lib/x86_64-linux-gnu/libdbus-1.so.3: no version information available (required by /proc/self/exe)
/builds/Vishal.Lanke/cypress-report-portal-integration/cache/Cypress/4.0.2/Cypress/Cypress: /lib/x86_64-linux-gnu/libdbus-1.so.3: no version information available (required by /builds/Vishal.Lanke/cypress-report-portal-integration/cache/Cypress/4.0.2/Cypress/Cypress)
====================================================================================================
(Run Starting)
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Cypress: 4.0.2 │
│ Browser: Electron 78 (headless) │
│ Specs: 4 found (actions.spec.js, aliasing.spec.js, assertions.spec.js, example.spec.js) │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────
Running: actions.spec.js (1 of 4)
/proc/self/exe: /lib/x86_64-linux-gnu/libdbus-1.so.3: no version information available (required by /proc/self/exe)
/builds/Vishal.Lanke/cypress-report-portal-integration/cache/Cypress/4.0.2/Cypress/Cypress: /lib/x86_64-linux-gnu/libdbus-1.so.3: no version information available (required by /builds/Vishal.Lanke/cypress-report-portal-integration/cache/Cypress/4.0.2/Cypress/Cypress)
/proc/self/exe: /lib/x86_64-linux-gnu/libdbus-1.so.3: no version information available (required by /proc/self/exe)
(node:305) Warning: Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certificate verification.
(node:305) Warning: Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certificate verification.
(Results)
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Tests: 14 │
│ Passing: 14 │
│ Failing: 0 │
│ Pending: 0 │
│ Skipped: 0 │
│ Screenshots: 0 │
│ Video: false │
│ Duration: 17 seconds │
│ Spec Ran: actions.spec.js │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────
Running: aliasing.spec.js (2 of 4)
/proc/self/exe: /lib/x86_64-linux-gnu/libdbus-1.so.3: no version information available (required by /proc/self/exe)
/builds/Vishal.Lanke/cypress-report-portal-integration/cache/Cypress/4.0.2/Cypress/Cypress: /lib/x86_64-linux-gnu/libdbus-1.so.3: no version information available (required by /builds/Vishal.Lanke/cypress-report-portal-integration/cache/Cypress/4.0.2/Cypress/Cypress)
(node:480) Warning: Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certificate verification.
(node:480) Warning: Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certificate verification.
/proc/self/exe: /lib/x86_64-linux-gnu/libdbus-1.so.3: no version information available (required by /proc/self/exe)
(Results)
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Tests: 2 │
│ Passing: 2 │
│ Failing: 0 │
│ Pending: 0 │
│ Skipped: 0 │
│ Screenshots: 0 │
│ Video: false │
│ Duration: 3 seconds │
│ Spec Ran: aliasing.spec.js │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────
Running: assertions.spec.js (3 of 4)
/proc/self/exe: /lib/x86_64-linux-gnu/libdbus-1.so.3: no version information available (required by /proc/self/exe)
/builds/Vishal.Lanke/cypress-report-portal-integration/cache/Cypress/4.0.2/Cypress/Cypress: /lib/x86_64-linux-gnu/libdbus-1.so.3: no version information available (required by /builds/Vishal.Lanke/cypress-report-portal-integration/cache/Cypress/4.0.2/Cypress/Cypress)
(node:656) Warning: Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certificate verification.
(node:656) Warning: Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certificate verification.
/proc/self/exe: /lib/x86_64-linux-gnu/libdbus-1.so.3: no version information available (required by /proc/self/exe)
(Results)
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Tests: 9 │
│ Passing: 9 │
│ Failing: 0 │
│ Pending: 0 │
│ Skipped: 0 │
│ Screenshots: 0 │
│ Video: false │
│ Duration: 5 seconds │
│ Spec Ran: assertions.spec.js │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────
Running: example.spec.js (4 of 4)
/proc/self/exe: /lib/x86_64-linux-gnu/libdbus-1.so.3: no version information available (required by /proc/self/exe)
/builds/Vishal.Lanke/cypress-report-portal-integration/cache/Cypress/4.0.2/Cypress/Cypress: /lib/x86_64-linux-gnu/libdbus-1.so.3: no version information available (required by /builds/Vishal.Lanke/cypress-report-portal-integration/cache/Cypress/4.0.2/Cypress/Cypress)
(node:830) Warning: Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certificate verification.
(node:830) Warning: Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certificate verification.
/proc/self/exe: /lib/x86_64-linux-gnu/libdbus-1.so.3: no version information available (required by /proc/self/exe)
/proc/self/exe: /lib/x86_64-linux-gnu/libdbus-1.so.3: no version information available (required by /proc/self/exe)
(Results)
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Tests: 9 │
│ Passing: 6 │
│ Failing: 2 │
│ Pending: 1 │
│ Skipped: 0 │
│ Screenshots: 4 │
│ Video: false │
│ Duration: 16 seconds │
│ Spec Ran: example.spec.js │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
(Screenshots)
- /builds/Vishal.Lanke/cypress-report-portal-integration/cypress/screenshots/examp (115x50)
le.spec.js/Feature Cypress Report Portal Integration Test -- SubFeature Verify
Cypress Examples page -- SubFeature Verify content Cypress example page -- Tes
t Verify page should contain Commands.png
- /builds/Vishal.Lanke/cypress-report-portal-integration/cypress/screenshots/examp (1280x720)
le.spec.js/Feature Cypress Report Portal Integration Test -- SubFeature Verify
Cypress Examples page -- SubFeature Verify content Cypress example page -- Tes
t Verify page should contain gfkjdgkjdfgl (failed) (failed).png
- /builds/Vishal.Lanke/cypress-report-portal-integration/cypress/screenshots/examp (500x200)
le.spec.js/Feature Cypress Report Portal Integration Test -- SubFeature Verify
content of Google Dot Com -- Test Verify contains of google page (failed).png
- /builds/Vishal.Lanke/cypress-report-portal-integration/cypress/screenshots/examp (1280x720)
le.spec.js/Feature Cypress Report Portal Integration Test -- SubFeature Verify
content of Google Dot Com -- Test Verify contains of google page (failed) (fai
led).png
====================================================================================================
(Run Finished)
Spec Tests Passing Failing Pending Skipped
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ ✔ actions.spec.js 00:17 14 14 - - - │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ aliasing.spec.js 00:03 2 2 - - - │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ assertions.spec.js 00:05 9 9 - - - │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✖ example.spec.js 00:16 9 6 2 1 - │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
✖ 1 of 4 failed (25%) 00:41 34 31 2 1 -
Uploading artifacts...
00:02
cypress/screenshots/**/*.png: found 4 matching files
Uploading artifacts to coordinator... ok id=27773739 responseStatus=201 Created token=4bkmyR8U
ERROR: Job failed: exit code 1
Hi @vishallanke , We will reproduce and investigate this issue in nearest future. Thanks for updates!
Hi @ElenaRomanchuk Is there any update on this defect ?
Our team is facing plenty of reliability issues
Due to various reliability and stability issues in current Cypress Agent as specified above, we decide to use JUnit and Cypress together. https://github.com/vishallanke/ReportPortalCypressJUnitAgent
Hello @vishallanke ! I started on this issue and am unable to reproduce it. Tested with GitHub Actions CI in this repo. All launches and their children have finished successfully and do not get stuck. The test run results are fully displayed in the Report Portal Demo instance. Some of the test cases failed, but everything looks fine. Could you provide any other details or conditions on this matter? Thank you in advance.