Open otalandim opened 4 years ago
Hello, I have regularly experienced this issue with my Production host.
for the Stage - all works as expected and tests successfully passed. When I'm running tests from a local machine - the same - all fine.
but when the tests are executed with GitLab CI on the Production host - 50% time it just fails with unable to open the requested URL.
Sorry, not an issue in my case. The reason for the unaccessible Production URL is a proxy of NLB which blocked this request. The tests were executed inside of Docker container on GitLab and the external Production URL is unaccessible from that network.
As a solution here - new runner wich is running outside of the container.
I've been able to temporarily patch the problem. The runners in Github Actions and AWS Codebuild would fail on visit but this issue isn't seen nearly as often on local runners or if you make them run in EC2.
Adding retries to my cypress.json
helped me go around this issue
{
"baseUrl": "http://localhost:3000",
"retries": {
"runMode": 2,
"openMode": 0
}
}
It's not ideal and I can see the tests failing one or two times before succeeding due to this but at least they are passing
We face the same problem. We have 17 specs and 6 different environments. From one point on, all upcoming tests are failing due to a 404 error. First we thought the problem was that too many tests were running at the same time, therefore we changed the tests to run the environments after each other. But still, the problem occurs. Less often, but still. If run locally, we never had the issue. Only if we run the tests in gitlab. We checked the logs of our nginx and couldn't find those requests that are shown in cypress as 404. So something is failing or blocking them before it accesses our testsystems.
I was able to almost completely solve this issue by turning chromeWebSecurity in the cypress config. "chromeWebSecurity": true.
Setting chromeWebSecurity to true does not solve the problem for me. Still, after half of my tests I get 404 for all remaining ones when running my tests in gitlab.
We were able to solve this issue for our tests with our specific tech stack. We were using Next.js and our Cypress tests ran with the GitHub action.
We changed from:
with:
start: yarn dev
to
with:
build: npm run build
start: npm start
wait-on: 'http://localhost:3000'
Which caused our tests to run a lot faster and the GitHub action actually waits for the response of the server. We needed BOTH changes because the tests were running on the landing page (which loads fast even in dev mode) before the server was ready, which was fixed by the wait-on
. And running the build helped to stay within the timeout window on the slower pages.
Hey, I fear within this issue two different topics are covered. 1) People having problems with a slow initial load of a page and therefore getting a timeout. I think the 404 is somehow correct here and workarounds have been presented.
2) People getting 404 errors from one point on after running multiple tests in headless mode in gitlab/github and others.
I am struggling with problem 2) as well. We are running around 100 tests, and after a random number of successful tests all upcoming ones are failing with 404 error.
@jennifer-shehane I faced a similar issue and on debugging think the following could be a hypothesis. Whenever i use
cy.visit("http://mysite.com")
I would get an ESOCKETTIMEDOUT, now the remote address of mysite points to akamai. Is it possible akamai has done some form of bot detection and ends up causing the above error. Now the moment i add an etc hosts entry and bypass akamai and send the request to the webserver behind akamai it seems to work. This is just my initial hypothesis. Also is there a way to make cypress not behave in a bot like manner 🤔
Adding this to the cy.visit() function solved it for me:
cy.visit(url, { headers: { "Accept-Encoding": "gzip, deflate" } });
... as referenced here: https://github.com/cypress-io/cypress/issues/943#issuecomment-730705557
Adding this to the cy.visit() function solved it for me:
cy.visit(url, { headers: { "Accept-Encoding": "gzip, deflate" } });
... as referenced here: #943 (comment)
It resolved for me too!
Thanks Adding this to the cy.visit() function solved it for me too
cy.visit(url, { headers: { "Accept-Encoding": "gzip, deflate" } });
... as referenced here: #943 (comment)
@euclidesdry @vjvibhanshu which cypress version are you using?
Hey, is there a way to set it somewhere global or do I have to add the header to each request?
@alangabrielarango my cypress version is:
"dependencies": {
"cypress": "^9.2.0"
}
@sosnet You can put this in your cypress support file:
before(() => {
cy.intercept("*", (req) => {
req.headers["Accept-Encoding"] = "gzip, deflate";
});
});
@jennifer-shehane Hello, my company has been having issues with the performance of cy.visit() and cy.request() timing out when running cypress integration tests on our Vercel preview deploymens in GithubActions. I have implemented timeout props to the options object on both cy methods, as well as the headers "Accept-Encoding" suggestion, but am still sometimes receiving the ESOCKETTIMEDOUT message. I am running Cypress 9.x and am not having this problem when running Cypress on my local machine. We only receive this ESOCKETTIMEDOUT when run through GithubActions. Do you have any other suggestions for a fix for this?
wait-on: 'http://localhost:3000'
"wait-on: 'http://localhost:3000'" worked for me. Thanks a lot!
In case it helps anyone else: I ran into this at work because a cypress test job was running against a frontend served by webpack-dev-server. The dev server will hold requests open until it finishes bundling so as our client builds got longer we started hitting the cypress responseTimeout. Bumped responseTimeout for now though a better fix would be for the job not to start running cypress until the client finishes bundling
wait-on: 'http://localhost:3000'
"wait-on: 'http://localhost:3000'" worked for me. Thanks a lot!
Where you added wait-on?
At the end of the .yml-file
with:
build: yarn build
start: yarn start
wait-on: "http://localhost:3000"
Same error on GitHub CI, I also tried these things , it still gives me the error
pageLoadTimeout
cypress version: 9.5.0
Same error on GitHub CI, I also tried these things , it still gives me the error
- increasing the
pageLoadTimeout
- adding headers
- adding retries
- ... but no luck yet
cypress version: 9.5.0
it worked for me with downgrading cypress to version 8.4.1 and using retries: 2 option
We were able to solve this issue for our tests with our specific tech stack. We were using Next.js and our Cypress tests ran with the GitHub action.
We changed from:
with: start: yarn dev
to
with: build: npm run build start: npm start wait-on: 'http://localhost:3000'
Which caused our tests to run a lot faster and the GitHub action actually waits for the response of the server. We needed BOTH changes because the tests were running on the landing page (which loads fast even in dev mode) before the server was ready, which was fixed by the
wait-on
. And running the build helped to stay within the timeout window on the slower pages.
This works for me. We use next
and apollo-client
. It waited for quite long time to visit the page at dev mode but when I use the built version, no need for timeout. Thank you @janhesters
wait-on: 'http://localhost:3000'
worked for me. Thanks
Hi, we are facing the same issue, tried with:
1 - Longer timeout in visit options (900000) but Cypress seems to ignore it and fails at 30 seconds. 2 - Added option headers: { "Accept-Encoding": "gzip, deflate" } } to visit method.
Hello,
Any update @jennifer-shehane ?
Issue still open after 2 years now...
we're getting ESOCKETTIMEDOUT
error with cypress 10.8.0
using Bitbucket Pipeline OR Jenkins running cypress/browsers
image
No error on local machine
Thx
Perhaps it ma be helpful for some of you, but it my case, it happened because the Angular application was served with the "watch" flag on and for some reason, as soon as it finishes to build and serve, both cypress was starting and the angular app was refreshing, ultimately leading to the ESOCKETTIMEDOUT
. Flipping the watch flag off, fixed my issue.
start-server-and-test
Hello, I have regularly experienced this issue with my Production host. for the Stage - all works as expected and tests successfully passed. When I'm running tests from a local machine - the same - all fine. but when the tests are executed with GitLab CI on the Production host - 50% time it just fails with unable to open the requested URL.
Sorry, not an issue in my case. The reason for the unaccessible Production URL is a proxy of NLB which blocked this request. The tests were executed inside of Docker container on GitLab and the external Production URL is unaccessible from that network.
As a solution here - new runner wich is running outside of the container.
Have you solved it yet?
Hi @jennifer-shehane and @otalandim,
I had the same issue, locally all is working fine but on GitLab CI, one or two tests always fail with the above mentioned error message:
Error: ESOCKETTIMEDOUT
.Currently, I workaround this by using
cypress-plugin-retries
, so when the error happens, the test gets retried.Unfortunately I can't tell you how to reprocedure the whole thing, because it very sporadically always hits different tests. But basically it happens at
cy.visit()
. Previously I usedcypress@3.8.4
and never got this error before. I read more about it in GitHub issues and perhaps this is another "workaround": #350 (comment); but I did not test it.Similar tickets: #6547, #5975 Used Cypress Version: 4.4.1
Have you solved it yet?
I have tried all the solutions presente here, nothing works.
I'm facing the same issue running cypress 10.11.0
with Nx 14
and Nextjs 12
.
My cypress.config.ts
:
const cypressJsonConfig = {
baseUrl: 'http://localhost:4002',
fileServerFolder: '.',
fixturesFolder: './src/fixtures',
video: true,
screenshot: true,
chromeWebSecurity: false,
env: {
failSilently: false,
},
viewportWidth: 1440,
viewportHeight: 900,
requestTimeout: 120000,
pageLoadTimeout: 120000,
defaultCommandTimeout: 120000,
specPattern: 'src/e2e/**/*.cy.{js,jsx,ts,tsx}',
supportFile: 'src/support/e2e.ts',
}
My Github action:
name: Cypress Tests
on:
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
branches:
- develop
jobs:
cypress-run:
name: Cypress test
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
with:
node-version-file: '.nvmrc'
- name: Cache central NPM modules
uses: actions/cache@v3
with:
path: ~/.npm
key: ${{ runner.os }}-node-${{ github.ref }}-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-node-${{ github.ref }}-${{ hashFiles('**/package-lock.json') }}
- name: Cache Cypress binary
uses: actions/cache@v3
with:
path: ~/.cache/Cypress
key: cypress-${{ runner.os }}-cypress-${{ github.ref }}-${{ hashFiles('**/package.json') }}
restore-keys: |
cypress-${{ runner.os }}-cypress-${{ github.ref }}-${{ hashFiles('**/package.json') }}
- name: Cache local node_modules
uses: actions/cache@v3
with:
path: node_modules
key: ${{ runner.os }}-node-modules-${{ github.ref }}-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-node-modules-${{ github.ref }}-
- name: Clean install
run: npm ci
- name: Cypress tests
run: npx nx e2e web-app-e2e:e2e --record
env:
CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
TERM: xterm
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
YACOOBA_ENV: develop
NEXT_PUBLIC_API_URL: https://our-dev-url-api
Error message:
cy.visit() failed trying to load:
http://localhost:4002/events/jazz-concert-c01894a6
We attempted to make an http request to this URL but the request failed without a response.
We received this error at the network level:
> Error: ESOCKETTIMEDOUT
Common situations why this would fail:
you don't have internet access
you forgot to run / boot your web server
your web server isn't accessible
you have weird network configuration settings on your computer
Because this error occurred during a before each hook we are skipping the remaining tests in the current suite: Event Page
For me the issue was the cypress tried to visit the page even before the page was loaded. I faced this issue on a nuxt app which would take around a min to compile in dev mode and yeah I used start-server-and-test still this issue was happening only on circle ci and not in local.
My previous package.json
"scripts": {
"dev": "nuxt",
"build": "nuxt build",
"start": "nuxt start",
"cy-test": "npx cypress run",
"ci": "start-server-and-test dev http://lvh.me:3000 cy-test",
},
I started the app after building it and that fixed the issue for me.
My current package.json
"scripts": {
"dev": "nuxt",
"build": "nuxt build",
"start": "nuxt start",
"build:start": "npm run build && npm run start",
"cy-test": "npx cypress run",
"cy-browser-test": "npx cypress open",
"ci": "start-server-and-test build:start http://lvh.me:3000 cy-test",
},
Hi. My two cents to this: When using Chromium, Firefox or Electron, then I get this random error almost every second test when cy.visit
tries to reach my initially started node instance. After running cy run
two or three times (I assume the pages got compiled on server-side), all `cy.visit
work as expected.
And I get the "ESOCKETTIMEDOUT"-error regardless of how high the timeout is set. I even tried 3 minutes.
FYI: I use a node instance with express next.js in dev-mode, so pages compile on the fly when requested by cypress. I also have to mention that all relevant tests try to visit a page that makes use of a redirection.
Cypress: v12.9.0 Electron 106 Chromium: 113 Firefox: 112 Node: v16.20.0 Next.js: 10.0.3
@teneko please open a new issue with a reproducible example and the Cypress team will be happy to investigate.
@nagash77, I have started experiencing this with Cypress 12.9.0. Although I understand that a reproducible example would make debugging this much easier, it is often not easy for the consumer of Cypress to produce one, however, it should be easier for the maintainers and developers to understand what is going on.
Getting same issue with latest Cypress 13.1.0 on CircleCI runs - locally no issues in headed and headless mode, any official comments on this from Cypress team?
Maybe, it will be helpful for somebody.
We faced the same issue a few days ago while using Github Action. In our case, it turned out that Cypress tests don't actually wait until the server is started. The Angular app began to listen on port only in the middle of testing. Luckily, Cypress action has a solution for this:
- name: "Run Tests"
id: tests
uses: cypress-io/github-action@v6
with:
build: npm run build
start: npm run start
wait-on: "http://localhost:4200" # URL to wait before tests
wait-on-timeout: 300 # How long to wait in seconds
browser: firefox
@jennifer-shehane any news on this, still issue with latest Cypress 13.3 and CIrcleCi
@jennifer-shehane we also been running into this for a while. Any update on this? How can we help? Repro's are a bit hard to deliver for our project, but since so many people seem to run into this shouldn't be that hard for someone to create a repro. For what it's worth: we're using Cloud (parallelism) and have been experiencing this issue for quite some time. Haven't been able to find any logic in why or when it happens. Seems somewhat random.
following
Any update. I have tried all the resolution which is provided in this discussion and none of them are working. Last successfully Gitlab job worked with below mentioned configuration, but that is also not working now.
Cypress: 13.1.0 Browser: Electron 106 (headless) Node Version: v18.16.0 (/usr/local/bin/node) Specs: 1 found (spec.cy.js)
I was getting the ESOCKETTIMEDOUT error in my company, and we solved it with the following solution:
Add on the nginx-test.conf
in the http server
listen 8080;
listen [::]:8080;
Change the ports in the docker-compose.yml
ports:
- '80:8080'
Maybe, it will be helpful for somebody.
We faced the same issue a few days ago while using Github Action. In our case, it turned out that Cypress tests don't actually wait until the server is started. The Angular app began to listen on port only in the middle of testing. Luckily, Cypress action has a solution for this:
- name: "Run Tests" id: tests uses: cypress-io/github-action@v6 with: build: npm run build start: npm run start wait-on: "http://localhost:4200" # URL to wait before tests wait-on-timeout: 300 # How long to wait in seconds browser: firefox
Adding wait-on: "http://localhost:3000"
worked for me!
@nagash77 I'm able to reproduce with 13.7.0
on NextJS by making an async server component that just awaits 31 seconds before rendering anything. I've already tried several things, like setting my .visit()
timeout to 60 seconds or setting turning on retryOnNetworkFailure
. I run into this issue in CI when my app takes more than 30s to build. This causes the initial visit to timeout and throw ESOCKETTIMEDOUT
. This behavior doesn't happen in the browser.
UPDATE
Bumping responseTimeout
in the configs worked for me. It's weird that this option isn't available as part of visit()
. It's also confusing that visit()
takes a generic timeout
param that isn't the same thing.
I am still seeing this error with Cypress in CircleCI using the latest orb. Strangely, it works with a NextJS app with the same setup, but not a React app.
Current behavior:
Desired behavior:
Does not start testing on gitlab, but on local machine it does works
Test code to reproduce
Starts the tests in gitlab
Versions
Cypress: 4.2.0 Image: cypress/base