cypress-io / cypress

Fast, easy and reliable testing for anything that runs in a browser.
https://cypress.io
MIT License
46.71k stars 3.16k forks source link

Cypress Config not using the current value. #27189

Open TomMaher-EUSL opened 1 year ago

TomMaher-EUSL commented 1 year ago

Current behavior

Created a brand new Angular project using Angular CLI: ng new TestCypress Added the Cypress via the Angular Cypress schematic: ng add @cypress/schematic Ran the e2e test via: ng e2e Everything worked as expected. I then modified the cypress configuration to change the base url in the typescript config file cypress.config.ts.

The changes to the settings were not picked up. I stopped the ng e2e and re-ran still the resolved configuration correctly shows the baseUrlas being resolved to cypress.config.ts but the value is wrong.

config cypress

Desired behavior

The updated value in the cypress.config.ts file should be the value that is resolved.

Test code to reproduce

The code to replicate the issue can be found here: Github CypressConfigIssue

Cypress Version

12.16.0

Node version

18.16.1

Operating System

Windows 11

Debug Logs

No response

Other

No response

MikeMcC399 commented 1 year ago

@TomMaher-EUSL

I cloned your repo and it worked for me, also on Windows 11, so I can't explain why it didn't work for you unfortunately.

MikeMcC399 commented 1 year ago

If you need some interactive help, you may be interested in the Cypress technical community on Discord

Discord chat

TomJMaher commented 1 year ago

@MikeMcC399 I pulled the repo on my personal laptop, which is also Windows 11. And was able to run replicate the issue as before.

I'm running the latest LTS version of Node.js, npm and Angular CLI:

image

I then ran the following commands:

npm i
ng serve
ng e2e

image

I then chose the Chrome browser:

image

I then selected Settings / Project settings / Resolved configuration and as you can see it is showing the value baseUrl is set to just http://localhost:4200/ and is set from the cypres.config.ts.

image

However as you can see from the contents of the cypress.config.ts file the value should be http://localhost:4200/Test

image

This example is as basic as you can get. It is a example Angular application generated through the ng new command with cypress added through the schematic. The only change was to the cypress.config.ts.

My guess would be there is something going wrong when Angular CLI is loading Cypress. I initially though I had missed something or done something wrong. However Cypress is saying that the baseUrl is set from the cypress.config.ts file and it does not have the value that is being displayed (and used) within Cypress.

Out of interest what version of Angular CLI are you using when you were not able to replicate the issue?

MikeMcC399 commented 1 year ago

@TomJMaher

My apologies! I was not using exactly your repro steps.

When Cypress is started using npx ng e2e the wrong value of baseUrl is used.

I used instead npx ng serve and in a parallel terminal window session npx cypress open where the desired value of baseUrl appears.

run from cypress

So this problem may be more suited to investigation via Angular rather than Cypress, but I would leave it here for the time being.

TomMaher-EUSL commented 1 year ago

@MikeMcC399 thank you, that's actually really helpful as it provides me with a work around.

I'm currently evaluating Cypress as a replacement for Protractor. Cypress looks to be a very good fit. I was using the following the two articles: Protractor to Cypress and Testing your app

And got as far as setting up the config when I ran in to difficulties. Looking at your suggestions combined with the info from the article Protractor to Cypress I was able to get the correct value from the config by using the following commands:

ng serve ng run TestCypress:cypress-open (or more generically ng run {your-project-name}:cypress-open)

And that worked:

image

Again thank you for your help and hopefully yourselves or somebody on the Angular CLI team can get the ng e2e command working as that is generally the most common workflow for testing in Angular.

TomMaher-EUSL commented 1 year ago

Just an update. I am leaning towards this being an issue with Cypress rather than Angular CLI. The reason for this is I tried using the cypress-run command. And it fails to load the correct baseUrl from the config.

Adding cypress to the Angular template project updates the package.json with two commands:

package.json:

"scripts": {
    "ng": "ng",
    "start": "ng serve",
    "build": "ng build",
    "watch": "ng build --watch --configuration development",
    "test": "ng test",
    "e2e": "ng e2e",
    "cypress:open": "cypress open",
    "cypress:run": "cypress run"
  }

The commands are configured via the angular.json file. angular.json As commented previously the following out the box Angular configuration of cypress open works:

"cypress-run": {
          "builder": "@cypress/schematic:cypress",
          "options": {
            "devServerTarget": "TestCypress:serve"
          },
          "configurations": {
            "production": {
              "devServerTarget": "TestCypress:serve:production"
            }
          }
        },
"cypress-open": {
          "builder": "@cypress/schematic:cypress",
          "options": {
            "watch": true,
            "headless": false
          }
        }

For clarity the key difference here are that using cypress open requires running the site via the ng serve command prior to running cypress open. Also cypress run uses headless by default.

So to rule out any differences between cypress run and cypress open I modified the settings so that they match. And sure enough it fails.

cypress-open as run

"cypress-run": {
          "builder": "@cypress/schematic:cypress",
          "options": {
            "devServerTarget": "TestCypress:serve"
          },
          "configurations": {
            "production": {
              "devServerTarget": "TestCypress:serve:production"
            }
          }
        },
"cypress-open": {
          "builder": "@cypress/schematic:cypress",
          "options": {
            "devServerTarget": "TestCypress:serve"
          },
          "configurations": {
            "production": {
              "devServerTarget": "TestCypress:serve:production"
            }
          }
        }

Removing all the options for the cypress open makes it work again.

cypress-open no options

 "cypress-run": {
          "builder": "@cypress/schematic:cypress",
          "options": {
            "devServerTarget": "TestCypress:serve"
          },
          "configurations": {
            "production": {
              "devServerTarget": "TestCypress:serve:production"
            }
          }
        },
"cypress-open": {
          "builder": "@cypress/schematic:cypress"
        }

You do have to manually use the ng serve command again and that does appear to be the key difference. When using the "cypress run" setup cypress or more accurately the builder cypress/schematic:cypress is starting the dev server target for you. And it is this, that appears to be not loading or causing the config to be ignored (or at least the baseUrl option from it).

MikeMcC399 commented 1 year ago

@TomJMaher

It looks like this is a consequence of Cypress <=> Angular integration, otherwise the file angular.json would have no influence when executing

npx cypress open or npm run cypress:open

In any case you have provided a repo with your issue, so the Cypress / Angular experts can give their opinion.

lmiller1990 commented 1 year ago

Here is another minimal reproduction repro.zip, it is indeed conflicting - I guess the angular schematic is somehow overriding the value. I'm looking into this.

TomMaher-EUSL commented 1 year ago

@TomJMaher

It looks like this is a consequence of Cypress <=> Angular integration, otherwise the file angular.json would have no influence when executing

npx cypress open or npm run cypress:open

In any case you have provided a repo with your issue, so the Cypress / Angular experts can give their opinion.

Yeah no worries. I was assuming the Cypress schematic is part of the Cypress project which I think is where the fault lies.

Btw I am really impressed with the Cypress features in particular the videos produced of the test run. This will definitely come in use in our integrated continuous deployment pipeline. As the tests will always be run headless. So having the ability to retrospectively look at a failing test is a very useful feature and clever idea.

MikeMcC399 commented 1 year ago

@TomJMaher

Btw I am really impressed with the Cypress features in particular the videos produced of the test run. This will definitely come in use in our integrated continuous deployment pipeline. As the tests will always be run headless. So having the ability to retrospectively look at a failing test is a very useful feature and clever idea.

Note that it's planned to disable video collection by default in a future version (see https://github.com/cypress-io/cypress/issues/26157 for details and reasons). That should not prevent you explicitly enabling videos if you migrate to the future version when the time comes.