search

bahmutov / cypress-split

Split Cypress specs across parallel CI machines for speed
MIT License
201 stars 23 forks source link

cypress-split: there are 0 found specs #119

Closed denis-domanskii closed 9 months ago

denis-domanskii commented 9 months ago

Hi! I faced the issue with finding the specs. cypress-split found 0 of them. 

❯ DEBUG=cypress-split,find-cypress-specs SPLIT=1 SPLIT_INDEX=0 cypress run --e2e --config-file tests/cypress/cypress.config.ts --browser chrome --reporter teamcity

DevTools listening on ws://127.0.0.1:57041/devtools/browser/25c7f080-b912-404f-a87c-c51eb6213e55
  cypress-split Cypress config env +0ms
  cypress-split {
  cypress-split   'cypress-plugin-snapshots': '{"autoCleanUp":false,"autopassNewSnapshots":true,"diffLines":3,"excludeFields":[],"formatJson":true,"ignoreExtraArrayItems":false,"ignoreExtraFields":false,"imageConfig":{"createDiffImage":true,"resizeDevicePixelRatio":true,"threshold":0.1,"thresholdType":"percent"},"normalizeJson":true,"prettier":true,"prettierConfig":{"html":{"parser":"html","tabWidth":2,"endOfLine":"lf"}},"screenshotConfig":{"blackout":[],"capture":"fullPage","clip":null,"padding":null,"disableTimersAndAnimations":true,"log":false,"scale":false,"timeout":30000},"serverEnabled":false,"serverHost":"localhost","serverPort":2121,"token":"xxx-token-xxx","updateSnapshots":false,"backgroundBlend":"difference","name":""}'
  cypress-split } +1ms
  find-cypress-specs finding specs of type e2e +0ms
  find-cypress-specs Cypress version 13.3.0 +0ms
  find-cypress-specs treating options as Cypress version 13 +0ms
  find-cypress-specs config has "e2e" property, treating as Cypress v10+ +1ms
  find-cypress-specs findCypressSpecsV10 +0ms
  find-cypress-specs options v10 { specPattern: 'tests/cypress/integration/**/*.spec.ts*', excludeSpecPattern: '**/*.spec.ts.snap' } +0ms
  find-cypress-specs globby options tests/cypress/integration/**/*.spec.ts* { sort: true, ignore: [ '**/*.spec.ts.snap' ], absolute: true } +0ms
  find-cypress-specs found 0 file(s) [] +6ms
  find-cypress-specs ignore patterns [ '**/*.spec.ts.snap' ] +0ms
  find-cypress-specs filtered 0 specs +0ms
  find-cypress-specs  +0ms
cypress-split: there are 0 found specs

I read that cypress-split is based on find-cypress-specs, so I ran it separately from the same dir and it found all the specs:

❯ DEBUG=find-cypress-specs CYPRESS_CONFIG_FILE=tests/cypress/cypress.config.ts npx find-cypress-specs

  find-cypress-specs arguments { _: [] } +0ms
  find-cypress-specs found file tests/cypress/cypress.config.ts +0ms
  find-cypress-specs loading Cypress config from /Users/denis.domanskii/git/my-app/tests/cypress/cypress.config.ts +54ms
  find-cypress-specs returning default export as config from ./tests/cypress/cypress.config.ts +619ms
  find-cypress-specs finding specs of type e2e +0ms
  find-cypress-specs treating options as Cypress version 10 +0ms
  find-cypress-specs config has "e2e" property, treating as Cypress v10+ +0ms
  find-cypress-specs findCypressSpecsV10 +1ms
  find-cypress-specs options v10 { specPattern: 'tests/cypress/integration/**/*.spec.ts*', excludeSpecPattern: '**/*.spec.ts.snap' } +0ms
  find-cypress-specs globby options tests/cypress/integration/**/*.spec.ts* { sort: true, ignore: [ '**/*.spec.ts.snap' ], absolute: false } +0ms
  find-cypress-specs found 38 file(s) [ 'tests/cypress/integration/anchors.spec.ts', 'tests/cypress/integration/basic-types.spec.ts', ... ] +7ms
  find-cypress-specs ignore patterns [ '**/*.spec.ts.snap' ] +10ms
  find-cypress-specs filtered 38 specs +2ms
  find-cypress-specs tests/cypress/integration/anchors.spec.ts
  find-cypress-specs tests/cypress/integration/basic-types.spec.ts
  ...

In both scenarios it definitely reads the custom config file, because specPattern is correct. But something is wrong in cypress-split <-> find-cypress-specs interaction. I tried cypress-on-fix, but nothing changed. Also I tried to explicitly pass CYPRESS_CONFIG_FILE env variable to cypress, but nothing change too.

denis-domanskii commented 9 months ago

Ok, I just found the issue. I have the following project structure:

my-app/
├── frontend
├── tests/
│   └── cypress/
│       ├── integration/
│       │   └── super.spec.ts
│       ├── cypress.config.js
│       └── package.json
└── package.json

My root package.json contains script

    "cy:test": start-server-and-test 'yarn start-test-server' http-get://localhost:7072 'SPLIT=1 SPLIT_INDEX=0 cypress run --e2e --config-file tests/cypress/cypress.config.ts --browser chrome --reporter teamcity'",

And cypress.config.ts have the following specPattern:

    specPattern: 'tests/cypress/integration/**/*.spec.ts*',

With such setup cypress-split doesn't work, showing 0 specs, when running it from the project root dir using the npm script above. And find-cypress-specs works perfect.

But when I change specPattern to

    specPattern: 'integration/**/*.spec.ts*',

'cypress-split' found all the specs and run it. So, for some reason, it's not dir-agnostic and I'd say it's an issue.

bahmutov commented 9 months ago

The problem is when cypress-split executes, it runs inside Cypress. Cypress changes the working directory to the same folder as the config file. So from tests/cypress the path 'tests/cypress/integration/*/.spec.ts*' makes no sense, right? I think Cypress itself has a fallback in this case, but regular globby inside find-cypress-specs does not. When you run find-cypress-specs by itself, you are running from the root folder, so it makes sense to find the specs. I need to think how to better solve this

bahmutov commented 9 months ago

Let me see if I can fix this, I added something in https://github.com/bahmutov/find-cypress-specs/issues/188 for this

github-actions[bot] commented 9 months ago

:tada: This issue has been resolved in version 1.5.4 :tada:

The release is available on:

Your semantic-release bot :package::rocket:

github-actions[bot] commented 9 months ago

:tada: This issue has been resolved in version 1.5.5 :tada:

The release is available on:

Your semantic-release bot :package::rocket: