search

bbc / simorgh

The BBC's Open Source Web Application. Contributions welcome! Used on some of our biggest websites, e.g.
https://www.bbc.com/thai
Other
1.39k stars 219 forks source link

Unable to run UK and non-UK tests against prod in UK #4651

Closed jamesdonoh closed 3 years ago

jamesdonoh commented 4 years ago

Describe the bug Our README says to use CYPRESS_UK when running tests in the UK against test/live environments.

However this seems to not work correctly depending on the service being tested.

To Reproduce Note: this affects many tests, the below is just given as one example.

Running the tests locally from inside the UK against test this passes:

$ CYPRESS_APP_ENV=test CYPRESS_SMOKE=false CYPRESS_ONLY_SERVICE=scotland CYPRESS_UK=true npx cypress run --spec cypress/integration/specialFeatures/cookieBanner/testsForAMPOnly.js

however without CYPRESS_UK set it fails.

Conversely, this passes:

$ CYPRESS_APP_ENV=test CYPRESS_SMOKE=false CYPRESS_ONLY_SERVICE=tamil npx cypress run --spec cypress/integration/specialFeatures/cookieBanner/testsForAMPOnly.js

however with CYPRESS_UK set it fails.

Expected behavior It should be possible to get the entire (SMOKE=false) test suite to pass in the UK.

Additional context The relevant README section should be reviewed/updated as part of this work.

oluoluoxenfree commented 4 years ago

@jamesdonoh Would it be okay to try this solution? https://docs.cypress.io/guides/guides/web-security.html#Disabling-Web-Security

currently researching if there are other implications of turning it off.

oluoluoxenfree commented 4 years ago

The above has already been implemented.

PriyaKR commented 4 years ago

@jamesdonoh By default cypress tests will run as if they were ran outside of the uk as the cypress variable CYPRESS_UK is set to default.If i remember correctly this variable CYPRESS_UK was added to fix the cypress failing locally because of cross domain issues.

Currently cypress doesnt support visiting multiple domains in one test https://github.com/cypress-io/cypress/issues/3985

Few tests are failing because in the same test we are trying to fetch another domain.

Also this issue was being avoided by usingcy.request instead of cy.visit in most of the tests as cypress request command doesn't throw cross domain issues.But i see in case of amp tests apart from making a cy.request before each test is run there is cy.visit being called within the test which causes the tests to fail.Might try replacing them with request instead of visit and see if the tests passes.

PriyaKR commented 4 years ago

I have tried running all the tests on my machine against test and live environments without passing any value to CYPRESS_UK as default its set to false and all the tests passes no cross origin issues one or two issues failed which are not related to whether CYPRESS_UK is set or not.If you wanna run the tests as if they are run from uk then you need to set CYPRESS_UK=true but you will see cross domain issues.

Will there be any use cases where you have to run locally against test env setting CYPRESS_UK=true?

PriyaKR commented 4 years ago

The issues which i noticed irrespective of CYPRESS_UK set to true or false

PriyaKR commented 4 years ago

The above issues needs to be fixed before we fix the env var CYPRESS_ENV issue.