search

percy / percy-cypress

Visual testing with Cypress and Percy
https://percy.io
MIT License
340 stars 41 forks source link

`cy.percySnapshot()` not uploading screenshots when used via `browserstack-cypress` #474

Closed aresnick closed 2 years ago

aresnick commented 2 years ago

The problem

I've successfully run tests with Cypress and Percy locally, and am now trying to use browserstack-cypress-cli to take screenshots via Percy.

When I run a basic test using cy.percySnapshot(), no screenshot is delivered.

NB: If this should be logged as an issue with browserstack-cypress-cli, let me know.

Environment

Details

When I run my test command, the test completes (successfully appearing in Automate as well), but in Percy no snapshot seems to be uploaded, and although I pass the --verbose option, no logs seem to appear anywhere.

image

Debug logs

These are the logs from my terminal when I run the test; I can't seem to locate any remote logs on Percy

aresnick@Alecs-Mac-mini powderhouse.github.io % yarn run percy exec --verbose -- browserstack-cypress run --sync
yarn run v1.22.17
verbose 0.117260833 Checking for configuration file "/Users/aresnick/Desktop/powderhouse.github.io/.npmrc".
verbose 0.117396916 Checking for configuration file "/Users/aresnick/.npmrc".
verbose 0.117467416 Found configuration file "/Users/aresnick/.npmrc".
verbose 0.11794375 Checking for configuration file "/Users/aresnick/.nvm/versions/node/v14.18.1/etc/npmrc".
verbose 0.118020833 Checking for configuration file "/Users/aresnick/Desktop/powderhouse.github.io/.npmrc".
verbose 0.118082708 Checking for configuration file "/Users/aresnick/Desktop/.npmrc".
verbose 0.118149333 Checking for configuration file "/Users/aresnick/.npmrc".
verbose 0.118190708 Found configuration file "/Users/aresnick/.npmrc".
verbose 0.118282791 Checking for configuration file "/Users/.npmrc".
verbose 0.119080833 Checking for configuration file "/Users/aresnick/Desktop/powderhouse.github.io/.yarnrc".
verbose 0.119143791 Checking for configuration file "/Users/aresnick/.yarnrc".
verbose 0.119197583 Found configuration file "/Users/aresnick/.yarnrc".
verbose 0.119291083 Checking for configuration file "/Users/aresnick/.nvm/versions/node/v14.18.1/etc/yarnrc".
verbose 0.119334666 Checking for configuration file "/Users/aresnick/Desktop/powderhouse.github.io/.yarnrc".
verbose 0.119370208 Checking for configuration file "/Users/aresnick/Desktop/.yarnrc".
verbose 0.119403708 Checking for configuration file "/Users/aresnick/.yarnrc".
verbose 0.119430541 Found configuration file "/Users/aresnick/.yarnrc".
verbose 0.119504291 Checking for configuration file "/Users/.yarnrc".
verbose 0.120731208 current time: 2022-03-03T18:28:22.101Z
warning From Yarn 1.0 onwards, scripts don't require "--" for options to be forwarded. In a future version, any explicit "--" will be forwarded as-is to the scripts.
$ /Users/aresnick/Desktop/powderhouse.github.io/node_modules/.bin/percy exec browserstack-cypress run --sync
[percy] Warning: Missing command separator (--), some command options may not work as expected
[percy] Percy has started!
[percy] Running "browserstack-cypress run --sync"
[3/3/2022, 1:28:23 PM] - info: Reading config from /Users/aresnick/Desktop/powderhouse.github.io/browserstack.json
(node:5180) Warning: Accessing non-existent property 'padLevels' of module exports inside circular dependency
(Use `node --trace-warnings ...` to show where the warning was created)
[3/3/2022, 1:28:23 PM] - info: Validating the config
[3/3/2022, 1:28:24 PM] - info: Creating tests.zip with files in .
[3/3/2022, 1:28:26 PM] - info: Skipping the upload of node_modules since BrowserStack has already cached your npm dependencies that have not changed since the last run.
[3/3/2022, 1:28:26 PM] - info: Uploading the tests to BrowserStack
tests.zip [========================================] 100% | ETA: 0s | Speed: 5.54 kbps | Duration: 52s [36.21 MB]
[3/3/2022, 1:29:18 PM] - info: Uploaded tests successfully (bs://eb639c9641ad9e3bb5224dcb0ce443fcd684190e)
[3/3/2022, 1:29:18 PM] - info: Deleted tests.zip successfully.
[3/3/2022, 1:29:18 PM] - info: Browsers list: Windows 10 (latest), Windows 10 (latest-1)
[3/3/2022, 1:29:18 PM] - info: Local is set to: false (you won't be able to test localhost / private URLs)
[3/3/2022, 1:29:18 PM] - info: Running your tests in headless mode. Use --headed arg to run in headful mode.
[3/3/2022, 1:29:18 PM] - info: Project name is: powderhouse.org
[3/3/2022, 1:29:18 PM] - info: Build name is: 0.01
[3/3/2022, 1:29:18 PM] - info: Parallels limit specified: 1
[3/3/2022, 1:29:19 PM] - info: Success! Build created with build id: 4942f756cbb60725a4d11a73b0607c42e8a4a45a
[3/3/2022, 1:29:19 PM] - info: Visit the Automate dashboard for real-time test reporting: https://automate.browserstack.com/dashboard/v2/builds/4942f756cbb60725a4d11a73b0607c42e8a4a45a
[3/3/2022, 1:30:27 PM] - info: BrowserStack machines are now setting up Cypress with the specified npm dependencies for running your tests. It might take some time before your tests start runnning and showing up below...

All tests:

-------------------------------------------------------------------------------------------------------
 Chrome 97 (Windows 1  :   cypress/integration/visual/home_spec.js ✔                                     
 0)
 Chrome 98 (Windows 1  :   cypress/integration/visual/home_spec.js ✔                                     
 0)
-------------------------------------------------------------------------------------------------------

Failed / skipped test report:
+--------------------------------------------------------------+--------+---------+-------------------------+
| Spec                                                         | Status | Browser | BrowserStack Session ID |
+--------------------------------------------------------------+--------+---------+-------------------------+

Total tests: 2, passed: 2, failed: 0, skipped: 0
Done in 153.692 seconds using 2 machines

[3/3/2022, 1:31:02 PM] - info: Report for build: 4942f756cbb60725a4d11a73b0607c42e8a4a45a was successfully created.
See the entire build report here:
https://automate.browserstack.com/dashboard/v2/builds/4942f756cbb60725a4d11a73b0607c42e8a4a45a
[percy] Finalized build #3: https://percy.io/be8d462e/powderhouse.org/builds/16240496
✨  Done in 161.89s.

On Automate, the logs don't indicate any issue:

────────────────────────────────────────────────────────────────────────────────────────────────────

  Running:  visual/home_spec.js                                                   (NaN of undefined)

  powderhouse.org Home
    √ Loads the homepage (2147ms)

  1 passing (6s)

  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests:        1                                                                                │
  │ Passing:      1                                                                                │
  │ Failing:      0                                                                                │
  │ Pending:      0                                                                                │
  │ Skipped:      0                                                                                │
  │ Screenshots:  0                                                                                │
  │ Video:        true                                                                             │
  │ Duration:     5 seconds                                                                        │
  │ Spec Ran:     visual/home_spec.js                                                              │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘

  (Video)

  -  Started processing:  Compressing to 1 CRF                                                      
  -  Finished processing: cypress_tes     (1 second)
                          t_folder\cypress\videos\visual\home_spec.js.mp4                           

  (Uploading Results)

  - Done Uploading (1/1) cypress_test_folder\cypress\videos\visual\home_spec.js.mp4

Code to reproduce issue

Happy to create a repo if needed, but right now it's tied into our site setup and similar. Hopefully this is enough to work with?

aresnick commented 2 years ago

FYI, here are my run_settings and connection_settings from browserstack.json

"run_settings": {
    "cypress_config_file": "./cypress.json",
    "project_name": "powderhouse.org",
    "build_name": "0.01",
    "exclude": [],
    "parallels": "1",
    "npm_dependencies": {
        "@percy/cli": "^1.0.0-beta.76",
        "@percy/cypress": "^3.1.1",
        "browserslist": "^4.19.3",
        "cypress": "^9.5.0"
    },
    "package_config_options": {},
    "headless": true
},
"connection_settings": {
    "local": false,
    "local_identifier": null,
    "local_mode": null,
    "local_config_file": null
}