search

cypress-io / cypress

Fast, easy and reliable testing for anything that runs in a browser.
https://cypress.io
MIT License
47.02k stars 3.18k forks source link

Cypress is trying to read the cypress config files even when it is invoked programmatically and has the config passed in #26492

Open mohsen1 opened 1 year ago

mohsen1 commented 1 year ago

Current behavior

Cypress reading the config file even when it is invoked programmatically and has the config passed in

I've created a minimal repo to demonstrate the current behavior:

https://github.com/mohsen1/cypress-programming-api-

Desired behavior

When invoking Cypress programmatically using cypress.open or cypress.run and passing the config object we're expecting Cypress to leave the cypress.config.js alone since we've passed the configuration.

Why this is a problem?

We have a complicated flow for running Cypress. We read the config ourselves and do a bunch of modifications to the config before calling cypress.run. When Cypress is trying to read the config file again, we run into unexpected problems. Namely Cypress fails to understand our custom imports.

Test code to reproduce

https://github.com/mohsen1/cypress-programming-api-

Cypress Version

11

Node version

16

Operating System

macOS, Linux

Debug Logs

No response

Other

No response

jordanpowell88 commented 1 year ago

Thank you for providing a repro. Im having a hard time understanding exactly what the problem is though. The screenshot below shows the cypress.config.ts file and the run-cypress.js script which alters the config file and then the terminal output which prints the altered config from the run-cypress.js file. Can you be more specific on where the problem lies?

Image

mohsen1 commented 1 year ago

@jordanpowell88 The problem is the cypress.config.js file is being loaded again (using a require call somewhere in Cypress module)

Console output of node run-cypress.js ``` Trace: cypress.config.js is read at Object. (/Users/mohsen_azimi/code/cypress-programming-api-/cypress.config.js:3:9) at Module._compile (node:internal/modules/cjs/loader:1101:14) at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10) at Module.load (node:internal/modules/cjs/loader:981:32) at Function.Module._load (node:internal/modules/cjs/loader:822:12) at Module.require (node:internal/modules/cjs/loader:1005:19) at require (node:internal/modules/cjs/helpers:102:18) at runTest (/Users/mohsen_azimi/code/cypress-programming-api-/run-cypress.js:4:27) at Object. (/Users/mohsen_azimi/code/cypress-programming-api-/run-cypress.js:10:1) at Module._compile (node:internal/modules/cjs/loader:1101:14) at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10) at Module.load (node:internal/modules/cjs/loader:981:32) at Function.Module._load (node:internal/modules/cjs/loader:822:12) at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:79:12) at node:internal/main/run_main_module:17:47 Trace: cypress.config.js is read at Object. (/Users/mohsen_azimi/code/cypress-programming-api-/cypress.config.js:3:9) at Module._compile (node:internal/modules/cjs/loader:1101:14) at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10) at Module.load (node:internal/modules/cjs/loader:981:32) at Function.Module._load (node:internal/modules/cjs/loader:822:12) at Module.require (node:internal/modules/cjs/loader:1005:19) at require (node:internal/modules/cjs/helpers:102:18) at loadFile (/Users/mohsen_azimi/Library/Caches/Cypress/11.2.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/child/run_require_async_child.js:89:14) at EventEmitter. (/Users/mohsen_azimi/Library/Caches/Cypress/11.2.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/child/run_require_async_child.js:116:38) at EventEmitter.emit (node:events:390:28) at process. (/Users/mohsen_azimi/Library/Caches/Cypress/11.2.0/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/util.js:33:22) at process.emit (node:events:390:28) at emit (node:internal/child_process:917:12) at processTicksAndRejections (node:internal/process/task_queues:84:21) ==================================================================================================== (Run Starting) ┌────────────────────────────────────────────────────────────────────────────────────────────────┐ │ Cypress: 11.2.0 │ │ Browser: Electron 106 (headless) │ │ Node Version: v16.10.0 (/Users/mohsen_azimi/.volta/tools/image/node/16.10.0/bin/node) │ │ 39m │ │ Specs: 1 found (spec.cy.js) │ │ Searched: cypress/e2e/**/*.cy.{js,jsx,ts,tsx} │ └────────────────────────────────────────────────────────────────────────────────────────────────┘ ──────────────────────────────────────────────────────────────────────────────────────────────────── Running: spec.cy.js (1 of 1) My First Test ✓ should visit home page (11977ms) 1 passing (12s) (Results) ┌────────────────────────────────────────────────────────────────────────────────────────────────┐ │ Tests: 1 │ │ Passing: 1 │ │ Failing: 0 │ │ Pending: 0 │ │ Skipped: 0 │ │ Screenshots: 0 │ │ Video: false │ │ Duration: 11 seconds │ │ Spec Ran: spec.cy.js │ └────────────────────────────────────────────────────────────────────────────────────────────────┘ ==================================================================================================== (Run Finished) Spec Tests Passing Failing Pending Skipped ┌────────────────────────────────────────────────────────────────────────────────────────────────┐ │ ✔ spec.cy.js 00:11 1 1 - - - │ └────────────────────────────────────────────────────────────────────────────────────────────────┘ ✔ All specs passed! 00:11 1 1 - - - [42509:0425/141542.067999:ERROR:ssl_client_socket_impl.cc(983)] handshake failed; returned -1, SSL error code 1, net_error -100 { status: 'finished', startedTestsAt: '2023-04-25T12:15:30.025Z', endedTestsAt: '2023-04-25T12:15:42.010Z', totalDuration: 11985, totalSuites: 1, totalTests: 1, totalPassed: 1, totalPending: 0, totalFailed: 0, totalSkipped: 0, runs: [ { stats: [Object], reporter: 'spec', reporterStats: [Object], hooks: [], tests: [Array], error: null, video: null, spec: [Object], shouldUploadVideo: true } ], browserPath: '', browserName: 'electron', browserVersion: '106.0.5249.51', osName: 'darwin', osVersion: '22.4.0', cypressVersion: '11.2.0', config: { supportFile: false, baseUrl: 'https://azimi.me', video: false, screenshotsFolder: '/Users/mohsen_azimi/code/cypress-programming-api-/cypress/screenshots', projectRoot: '/Users/mohsen_azimi/code/cypress-programming-api-', projectName: 'cypress-programming-api-', repoRoot: '/Users/mohsen_azimi/code/cypress-programming-api-', rawJson: { e2e: [Object], supportFile: false, baseUrl: 'https://azimi.me', video: false, screenshotsFolder: 'cypress/screenshots', envFile: {}, projectRoot: '/Users/mohsen_azimi/code/cypress-programming-api-', projectName: 'cypress-programming-api-', repoRoot: '/Users/mohsen_azimi/code/cypress-programming-api-' }, configFile: 'cypress.config.js', morgan: false, isTextTerminal: true, socketId: 's2fapofetg', report: true, animationDistanceThreshold: 5, arch: 'arm64', blockHosts: null, chromeWebSecurity: true, clientCertificates: [], defaultCommandTimeout: 4000, downloadsFolder: '/Users/mohsen_azimi/code/cypress-programming-api-/cypress/downloads', env: {}, execTimeout: 60000, experimentalFetchPolyfill: false, experimentalInteractiveRunEvents: false, experimentalRunAllSpecs: false, experimentalSessionAndOrigin: false, experimentalModifyObstructiveThirdPartyCode: false, experimentalSourceRewriting: false, experimentalSingleTabRunMode: false, experimentalStudio: false, experimentalWebKitSupport: false, fileServerFolder: '/Users/mohsen_azimi/code/cypress-programming-api-', fixturesFolder: '/Users/mohsen_azimi/code/cypress-programming-api-/cypress/fixtures', excludeSpecPattern: '*.hot-update.js', includeShadowDom: false, keystrokeDelay: 0, modifyObstructiveCode: true, numTestsKeptInMemory: 0, platform: 'darwin', pageLoadTimeout: 60000, port: 56229, projectId: null, redirectionLimit: 20, reporter: 'spec', reporterOptions: null, requestTimeout: 5000, resolvedNodePath: '/Users/mohsen_azimi/.volta/tools/image/node/16.10.0/bin/node', resolvedNodeVersion: '16.10.0', responseTimeout: 30000, retries: { runMode: 0, openMode: 0 }, screenshotOnRunFailure: true, slowTestThreshold: 10000, scrollBehavior: 'top', supportFolder: false, taskTimeout: 60000, testIsolation: null, trashAssetsBeforeRuns: true, userAgent: null, videoCompression: 32, videosFolder: '/Users/mohsen_azimi/code/cypress-programming-api-/cypress/videos', videoUploadOnPasses: true, viewportHeight: 660, viewportWidth: 1000, waitForAnimations: true, watchForFileChanges: false, specPattern: 'cypress/e2e/**/*.cy.{js,jsx,ts,tsx}', additionalIgnorePattern: [], autoOpen: false, browsers: [ [Object], [Object], [Object] ], clientRoute: '/__/', cypressBinaryRoot: '/Users/mohsen_azimi/Library/Caches/Cypress/11.2.0/Cypress.app/Contents/Resources/app', devServerPublicPathRoute: '/__cypress/src', hosts: null, isInteractive: true, namespace: '__cypress', reporterRoute: '/__cypress/reporter', socketIoCookie: '__socket', socketIoRoute: '/__socket', version: '11.2.0', xhrRoute: '/xhrs/', cypressEnv: 'production', resolved: { animationDistanceThreshold: [Object], arch: [Object], baseUrl: [Object], blockHosts: [Object], chromeWebSecurity: [Object], clientCertificates: [Object], defaultCommandTimeout: [Object], downloadsFolder: [Object], env: {}, execTimeout: [Object], experimentalFetchPolyfill: [Object], experimentalInteractiveRunEvents: [Object], experimentalRunAllSpecs: [Object], experimentalSessionAndOrigin: [Object], experimentalModifyObstructiveThirdPartyCode: [Object], experimentalSourceRewriting: [Object], experimentalSingleTabRunMode: [Object], experimentalStudio: [Object], experimentalWebKitSupport: [Object], fileServerFolder: [Object], fixturesFolder: [Object], excludeSpecPattern: [Object], includeShadowDom: [Object], keystrokeDelay: [Object], modifyObstructiveCode: [Object], nodeVersion: [Object], numTestsKeptInMemory: [Object], platform: [Object], pageLoadTimeout: [Object], port: [Object], projectId: [Object], redirectionLimit: [Object], reporter: [Object], reporterOptions: [Object], requestTimeout: [Object], resolvedNodePath: [Object], resolvedNodeVersion: [Object], responseTimeout: [Object], retries: [Object], screenshotOnRunFailure: [Object], screenshotsFolder: [Object], slowTestThreshold: [Object], scrollBehavior: [Object], supportFile: [Object], supportFolder: [Object], taskTimeout: [Object], testIsolation: [Object], trashAssetsBeforeRuns: [Object], userAgent: [Object], video: [Object], videoCompression: [Object], videosFolder: [Object], videoUploadOnPasses: [Object], viewportHeight: [Object], viewportWidth: [Object], waitForAnimations: [Object], watchForFileChanges: [Object], specPattern: [Object], browsers: [Object], hosts: [Object], isInteractive: [Object] }, testingType: 'e2e', remote: { origin: 'https://azimi.me', strategy: 'http', fileServer: null, domainName: 'azimi.me', props: [Object] }, browser: null, specs: [], proxyUrl: 'http://localhost:56229', browserUrl: 'https://azimi.me/__/', reporterUrl: 'https://azimi.me/__cypress/reporter', xhrUrl: '__cypress/xhrs/', proxyServer: 'http://localhost:56229', state: {} } } ```

Note the line console.trace('cypress.config.js is read'); was invoked twice even though my code requires this file only once. You can see in the trace that it is being required from within Cypress module for the second time.

For most people this is not a problem but for our case -- since we use Cypress CLI programmatically this is causing issues.

warrensplayer commented 1 year ago

@mohsen1 I was able to reproduce what you are seeing. Thanks for the reproduction. It looks like Cypress needs to see that the configuration is being passed in programmatically and not load the config again.