search

cypress-io / cypress

Fast, easy and reliable testing for anything that runs in a browser.
https://cypress.io
MIT License
46.72k stars 3.16k forks source link

Warning: vkCreateInstance: Found no drivers! warning shown when running Cypress #29085

Closed Rajesh-Kodandam closed 5 months ago

Rajesh-Kodandam commented 6 months ago

Current behavior

Jenkis e2e tests are failing due to Vulkan

====================================================================================================
17:37:39    (Run Starting)
17:37:39    ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
17:37:39    │ Cypress:        13.6.6                                                                         │
17:37:39    │ Browser:        Chrome 118 (headless)                                                          │
17:37:39    │ Node Version:   v20.9.0 (/usr/local/bin/node)                                                  │
17:37:39    │ Specs:          2 found (content-internal-links.cy.ts, sidebar.cy.ts)                          │
17:37:39    │ Searched:       cypress/e2e/**/*                                                               │
17:37:39    └────────────────────────────────────────────────────────────────────────────────────────────────┘
17:37:39  ────────────────────────────────────────────────────────────────────────────────────────────────────
17:37:39                                                                                                      
17:37:39    Running:  content-internal-links.cy.ts                                                    (1 of 2)
17:37:42  Couldn't determine Mocha version
17:37:42     | FDP Documentation |
17:37:42      Frontend Development Platform (FDP) | 
17:38:14        1) Development Environment Stepup
17:38:46        2) Quick Start Guide
17:39:19        3) Tech Stack
17:39:19       Getting started | 
17:39:19         Quick start | 
17:39:37  Warning: vkCreateInstance: Found no drivers!
17:39:37  Warning: vkCreateInstance failed with VK_ERROR_INCOMPATIBLE_DRIVER
17:39:37      at CheckVkSuccessImpl (../../third_party/dawn/src/dawn/native/vulkan/VulkanError.cpp:88)
17:39:37      at CreateVkInstance (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:458)
17:39:37      at Initialize (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:344)
17:39:37      at Create (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:266)
17:39:37      at operator() (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:521)

### Desired behavior

_No response_

### Test code to reproduce

cypress: 13.6.6
contianer image: node-20.11.0-chrome-121.0.6167.184-1-ff-123.0-edge-121.0.2277.128-1

seems my e2e tests are failing with below errors

====================================================================================================
17:37:39    (Run Starting)
17:37:39    ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
17:37:39    │ Cypress:        13.6.6                                                                         │
17:37:39    │ Browser:        Chrome 118 (headless)                                                          │
17:37:39    │ Node Version:   v20.9.0 (/usr/local/bin/node)                                                  │
17:37:39    │ Specs:          2 found (content-internal-links.cy.ts, sidebar.cy.ts)                          │
17:37:39    │ Searched:       cypress/e2e/**/*                                                               │
17:37:39    └────────────────────────────────────────────────────────────────────────────────────────────────┘
17:37:39  ────────────────────────────────────────────────────────────────────────────────────────────────────
17:37:39                                                                                                      
17:37:39    Running:  content-internal-links.cy.ts                                                    (1 of 2)
17:37:42  Couldn't determine Mocha version
17:37:42     | FDP Documentation |
17:37:42      Frontend Development Platform (FDP) | 
17:38:14        1) Development Environment Stepup
17:38:46        2) Quick Start Guide
17:39:19        3) Tech Stack
17:39:19       Getting started | 
17:39:19         Quick start | 
17:39:37  Warning: vkCreateInstance: Found no drivers!
17:39:37  Warning: vkCreateInstance failed with VK_ERROR_INCOMPATIBLE_DRIVER
17:39:37      at CheckVkSuccessImpl (../../third_party/dawn/src/dawn/native/vulkan/VulkanError.cpp:88)
17:39:37      at CreateVkInstance (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:458)
17:39:37      at Initialize (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:344)
17:39:37      at Create (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:266)
17:39:37      at operator() (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:521)

Cypress Version

13.6.6

Node version

20.11.0

Operating System

macOS Sonoma 14.3.1

Debug Logs

No response

Other

No response

roadSurfer commented 6 months ago

We see this as well when running locally in Docker, and in our GitLab Pipelines (via the Docker executor).

Warning: vkCreateInstance: Found no drivers!
Warning: vkCreateInstance failed with VK_ERROR_INCOMPATIBLE_DRIVER
    at CheckVkSuccessImpl (../../third_party/dawn/src/dawn/native/vulkan/VulkanError.cpp:88)
    at CreateVkInstance (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:458)
    at Initialize (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:344)
    at Create (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:266)
    at operator() (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:521)

I am not fully sure if this is a problem or, like with libva error: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null), if the message is benign but there are complaints about tests locking-up similar to issue 1136.

These lock-ups appear to be intermittent and it might just be coincidence that this warning is the last log message emitted.

Rajesh-Kodandam commented 6 months ago

I dont see these errors when I run locally and faced only when I used the container image: node-20.11.0-chrome-121.0.6167.184-1-ff-123.0-edge-121.0.2277.128-1 on jenkins.

roadSurfer commented 6 months ago

I meant locally in a Docker container. Apologies, I should have been clearer.

MikeMcC399 commented 6 months ago

@Rajesh-Kodandam / @roadSurfer

I ran all latest Cypress Docker images including cypress/browsers:node-20.11.0-chrome-121.0.6167.184-1-ff-123.0-edge-121.0.2277.128-1 under GitHub's Ubuntu 22.04 runner and could not reproduce this issue.

Do either of you have public workflow you can share which shows the Vulkan issue?

MikeMcC399 commented 6 months ago

@Rajesh-Kodandam

In your log file you are showing Chrome 118, however you say that you are using node-20.11.0-chrome-121.0.6167.184-1-ff-123.0-edge-121.0.2277.128-1

so your log should also show Chrome 121.

Could you check that you can reproduce the issue with Chrome 121?

Current behavior

Jenkis e2e tests are failing due to Vulkan


====================================================================================================
17:37:39    (Run Starting)
17:37:39    ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
17:37:39    │ Cypress:        13.6.6                                                                         │
17:37:39    │ Browser:        Chrome 118 (headless)                                                          │
17:37:39    │ Node Version:   v20.9.0 (/usr/local/bin/node)                                                  │
17:37:39    │ Specs:          2 found (content-internal-links.cy.ts, sidebar.cy.ts)                          │
17:37:39    │ Searched:       cypress/e2e/**/*                                                               │
17:37:39    └────────────────────────────────────────────────────────────────────────────────────────────────┘
17:37:39  ────────────────────────────────────────────────────────────────────────────────────────────────────
17:37:39                                                                                                      
17:37:39    Running:  content-internal-links.cy.ts                                                    (1 of 2)
17:37:42  Couldn't determine Mocha version
17:37:42     | FDP Documentation |
17:37:42      Frontend Development Platform (FDP) | 
17:38:14        1) Development Environment Stepup
17:38:46        2) Quick Start Guide
17:39:19        3) Tech Stack
17:39:19       Getting started | 
17:39:19         Quick start | 
17:39:37  Warning: vkCreateInstance: Found no drivers!
17:39:37  Warning: vkCreateInstance failed with VK_ERROR_INCOMPATIBLE_DRIVER
17:39:37      at CheckVkSuccessImpl (../../third_party/dawn/src/dawn/native/vulkan/VulkanError.cpp:88)
17:39:37      at CreateVkInstance (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:458)
17:39:37      at Initialize (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:344)
17:39:37      at Create (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:266)
17:39:37      at operator() (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:521)
.
.
.
cypress: 13.6.6
contianer image: node-20.11.0-chrome-121.0.6167.184-1-ff-123.0-edge-121.0.2277.128-1
roadSurfer commented 6 months ago

I tried the cypress/browsers:node-20.11.0-chrome-121.0.6167.184-1-ff-123.0-edge-121.0.2277.128-1 image running in Docker 25.0.3, build 4debf41 on KDE Neon (Ubuntu 22.04 base):

Electon 118, Chrome 121, and Edge 121 all reported the warning; although the tests did not hang.

I am not a Cypress developer, but I will see if we can get a test made public.

roadSurfer commented 6 months ago

Attached is a very crude test that exhibits the issue.

I don't work with Cyrpess and have probably done this the hardest way possible, but it does show the warning after a minute or so with a simple loop that emits a log message every second.

It does not exhibit any hang/delay, but we are seeing this in the GitLab Pipelines where tests run for an hour before being killed due to run time. That might be down to a fault with one (or more) of our specs, I am not sure yet and we are trying to chase down where it is happening.

1022_demo.zip

Rajesh-Kodandam commented 6 months ago

@MikeMcC399 and @roadSurfer: Thank you for response.

Its my bad regarding the versions messedup. But now I did tried as mentioned before

docker image: node-20.11.0-chrome-121.0.6167.184-1-ff-123.0-edge-121.0.2277.128-1

========================================================================================== 12:49:54 (Run Starting) 12:49:54 ──────────────────────────────────────────────────────── 12:49:54 │ Cypress: 13.6.6 │ 12:49:54 │ Browser: Chrome 121 (headless) │ 12:49:54 │ Node Version: v20.11.0 (/usr/local/bin/node) │ 12:49:54 │ Specs: 2 found (content-internal-links.cy.ts, sidebar.cy.ts) │ 12:49:54 │ Searched: cypress/e2e/*/ │ 12:49:54 ──────────────────────────────────────────────────────── 12:49:54 Running: content-internal-links.cy.ts (1 of 2) 12:49:58 Couldn't determine Mocha version 12:49:58 | FDP Documentation | 12:49:58 Frontend Development Platform (FDP) | 12:50:30 1) Development Environment Stepup 12:51:02 2) Quick Start Guide 12:51:35 3) Tech Stack 12:51:35 Getting started | 12:51:35 Quick start | 12:51:53 Warning: vkCreateInstance: Found no drivers! 12:51:53 Warning: vkCreateInstance failed with VK_ERROR_INCOMPATIBLE_DRIVER 12:51:53 at CheckVkSuccessImpl (../../third_party/dawn/src/dawn/native/vulkan/VulkanError.cpp:88) 12:51:53 at CreateVkInstance (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:458) 12:51:53 at Initialize (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:344) 12:51:53 at Create (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:266) 12:51:53 at operator() (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:521)

MikeMcC399 commented 6 months ago

@Rajesh-Kodandam

Rajesh-Kodandam commented 6 months ago

We have app called docs which is our documentation those tests only failing with that reason not all tests. I'll explore more about Mocha version.

MikeMcC399 commented 6 months ago

@roadSurfer

Thanks for your demo. It doesn't actually run out of the box as there is an extra set of closing brackets in the support file.

I corrected the bracket error and ran it locally on Windows 11 and then on Ubuntu 22.04 on GitHub Actions . It did not fail in either case, so it seems that we are chasing a bug which is hard to reproduce. 🙁

MikeMcC399 commented 6 months ago
Rajesh-Kodandam commented 6 months ago

This is issue there in 13.6.4 as well.

Rajesh-Kodandam commented 6 months ago

@Rajesh-Kodandam

  • Does the problem only occur with the test spec content-internal-links.cy.ts or does it happen when you try to run other test specs?
  • Do you know the reason for the error message "Couldn't determine Mocha version"?

I did fixed the mocha version: 10.3.0 now and after that looks like as below with same vkCreateInstance: Found no drivers!

=========================================================================================== 14:56:22 (Run Starting) 14:56:22 ──────────────────────────────────────────────────────── 14:56:22 │ Cypress: 13.6.6 │ 14:56:22 │ Browser: Chrome 121 (headless) │ 14:56:22 │ Node Version: v20.11.0 (/usr/local/bin/node) │ 14:56:22 │ Specs: 2 found (content-internal-links.cy.ts, sidebar.cy.ts) │ 14:56:22 │ Searched: cypress/e2e/*/ │ 14:56:22 ─────────────────────────────────────────────────────── 14:56:22 Running: content-internal-links.cy.ts (1 of 2) 14:56:25 | FDP Documentation | 14:56:25 Frontend Development Platform (FDP) | 14:56:57 1) Development Environment Stepup 14:57:30 2) Quick Start Guide 14:58:02 3) Tech Stack 14:58:02 Getting started | 14:58:02 Quick start | 14:58:20 Warning: vkCreateInstance: Found no drivers! 14:58:20 Warning: vkCreateInstance failed with VK_ERROR_INCOMPATIBLE_DRIVER 14:58:20 at CheckVkSuccessImpl (../../third_party/dawn/src/dawn/native/vulkan/VulkanError.cpp:88) 14:58:20 at CreateVkInstance (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:458) 14:58:20 at Initialize (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:344) 14:58:20 at Create (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:266) 14:58:20 at operator() (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:521)

roadSurfer commented 6 months ago

@MikeMcC399

Thanks for your demo. It doesn't actually run out of the box as there is an extra set of closing brackets in the support file.

Whoops! Sorry about that.

I corrected the bracket error and ran it locally on Windows 11 and then on Ubuntu 22.04 on GitHub Actions . It did not fail in either case, so it seems that we are chasing a bug which is hard to reproduce. 🙁

The test will complete, but that vkCreateInstance warning will be shown 100% of the time when I run it on Docker locally (VBox VM runing KDE Neon on Win10) and when executed via a GitLab Pipeline.

I have never needed to use a GitHub Action, but I can see how hard it is to do.

I am not even sure if this warning is an actual problem or not, we are still trying to chase down where the tests are hanging/delyaing.

roadSurfer commented 6 months ago

I created a GitHub action using runner ubuntu-22.04 and running on image cypress/browsers:node-20.11.0-chrome-121.0.6167.184-1-ff-123.0-edge-121.0.2277.128-1, I received the vkCreateInstance warning.

I deliberately did not use cypress-io/github-action@v6 as mentioned in the docs as we obviously would not have that available in a GitLab Pipeline and I wanted to remain as consistent with our environment as possible.

The output was:

...
Iteration: 114
Warning: vkCreateInstance: Found no drivers!
Warning: vkCreateInstance failed with VK_ERROR_INCOMPATIBLE_DRIVER
    at CheckVkSuccessImpl (../../third_party/dawn/src/dawn/native/vulkan/VulkanError.cpp:88)
    at CreateVkInstance (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:458)
    at Initialize (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:344)
    at Create (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:266)
    at operator() (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:521)

Iteration: 115
...

You can see the workflow here: https://github.com/roadSurfer/cypress_1022/blob/main/.github/workflows/main.yml

Edit: I should add, the run did complete.

MikeMcC399 commented 6 months ago

@roadSurfer

Thanks for your GitHub repo! I revisited the results of running your test in my repo under Ubuntu 22.04 and I found that there was indeed a warning in the logs like the one you are seeing.

Warning: vkCreateInstance: Found no drivers!

I was unfortunately misled by the fact that the workflow showed that it had succeeded.

I did however correctly report about running locally under Windows 11. In this environment I saw no warnings issued.

roadSurfer commented 6 months ago

Phew! It's not just me going crazy. :smile:

As I say, it may not be related to the hangs/delays and just coincidence that it was the last message logged.

MikeMcC399 commented 6 months ago

@roadSurfer

I omitted to say that I see the warning in the GitHub Ubuntu 22.04 runner without using a Cypress Docker image, so it seems that Docker may be a distraction in finding a root cause.

I didn't see any warning locally when I tested under Ubuntu 22.04 running inside VMware Workstation.

jennifer-shehane commented 6 months ago

Going to transfer this issue to the main cypress repo since it seems more related to the App than the docker-image.

roadSurfer commented 6 months ago

@MikeMcC399 I would be very surprised if the backend of GitHub wasn't similar to GitLab in that the runner is just spinning up an ephemeral container to do the build. So Docker still in play,

What's interesting is that your Win11 run didn't show the warning. ~Did you run it direct on Windows, via WSL2 (so, virtualised), or Docker Desktop (which would be a container within a VM)?~ (Sorry, missed a line in yor comment.)

I do get the warning when running via a container inside my VBox GNU/Linux VM (KDE Neon, based on Ubuntu 22.04). I can't use WSL2 nor Docker Desktop due hypervisor incompatibilities, so I cannot recreate what you did.

Out of interest I updated Node etc and ran it directly in my VM (no container) and I saw this:

I really am starting to think that this vkCreateInstance warning is benign (it is a "warning", not an "error") and a total red herring for the specs hanging/delaying. It may well just be an artifact of available host hardware how/if that is passed through to any VM/containerisation, or how a hypervisor is emulating/abstracting GPU hardware ). I wouldn't have the first clue on how to detangle all that!

For those who do have a hang/delay, is it possible to run Cypress with increased logging so more can be seen on what is happening when it comes to a halt?

MikeMcC399 commented 6 months ago

@roadSurfer

Thanks for your comments and thoughts!

Another user, running in Cypress Docker cypress/included:13.6.6 reported

while is running i get the following warning "vkCreateInstance: Found no drivers!" when this happens the server crashes and the test stops running

so for this user, the warning appears not to be simply benign. However they then reported that reverting to Cypress 13.6.4 worked around the issue.

So it seems that the outcomes can be quite varied, as @Rajesh-Kodandam noted that reverting to Cypress 13.6.4 did not resolve the issue there.

I have run your test with DEBUG=cypress:* and it does not provide any additional useful information. I also tried different runners, and reverting to Cypress 13.6.4.

GitHub runner Cypress warning
ubuntu-20.04 13.6.6 yes
ubuntu-22.04 13.6.6 yes
windows-2022 13.6.6 no
macos-14 13.6.6 no
ubuntu-22.04 13.6.4 no

There is no GPU on GitHub runners. Cypress runs using Xvfb and this area has been the source of warnings in previous versions.

I'll see what else I can find out.

MikeMcC399 commented 6 months ago

Vulkan drivers are part of Debian (used for Cypress Docker images) and Ubuntu so it seems like the message "Found no drivers!" should not be occurring. I think I have come to the end of what I can find out here, as I'm not a Linux driver expert. My guess is that the issue may be related to Electron.

roadSurfer commented 6 months ago

I gave cypress/included:13.6.6 a go as well, saw the error but no crash/hang. Possibly my example is just too simple.

ryabtsev commented 6 months ago

Yesterday I faced with issue after update Docker desktop from old version (4.9.1 (81317)) to 4.28.0. I run test with old Docker and cypress/included:13.6.6 and it was OK After docker updating error is appeared: Warning: vkCreateInstance: Found no drivers!

And I've fixed it by Docker downgrading

macOS Sonoma 14.3.1

ryabtsev commented 6 months ago

I gave cypress/included:13.6.6 a go as well, saw the error but no crash/hang. Possibly my example is just too simple.

please tell me the docker version

Rajesh-Kodandam commented 6 months ago

docker image: node-20.11.0-chrome-121.0.6167.184-1-ff-123.0-edge-121.0.2277.128-1

docker image: node-20.11.0-chrome-121.0.6167.184-1-ff-123.0-edge-121.0.2277.128-1

pitgrap commented 6 months ago

maybe this information is helpful too:

We're using the docker image cypress/factory in our CI (Jenkins). We updated it from 

NODE_VERSION=18.18.2
CHROME_VERSION=118.0.5993.70-1

to

NODE_VERSION=20.11.1
CHROME_VERSION=118.0.5993.70-1

and updates cypress from 13.3.1 to 13.6.6. The Docker version stayed the same: Docker version 20.10.17.

Since the update we get the same vkCreateInstance warning, but everything works.

Rajesh-Kodandam commented 6 months ago

After all this, Yes it's not failing my test due to vkCreateInstance

bobbhatti commented 6 months ago

Hi,

We started experiencing this a few weeks ago too.

We have a self-managed instance of gitlab - with the following

gitlab runners (debian) - in aws ec2 (t4g.medium) cypress - 13.6.6 chromium 122 arm64 architecture

we execute the full suite using the following command (so not docker run) CYPRESS_NO_COMMAND_LOG=1 cypress run -b chromium etc (recording to cypress cloud)

and the vkCreateInstance warning started to occur - and at times when a test failed, made us think it was causing tests to fail, which in fact it isn't, we tried to set a launchoption to disable vulkan but it still happens

however we have a another set of tests (smoke tests) - which run in a cypress docker container executed using docker run, currently at 13.6.3 and chromium 120 - and the issue does not occur.

Therefore, we are going to downgrade from chromium 122 to chromium 120 in the gitlab runner that runs our full suite, to see if that resolves, if not we may have to downgrade to 13.6.3 of cypress too - will post an update when we have done so

MikeMcC399 commented 6 months ago

@bobbhatti

Does your self-managed GitLab runner include GPU hardware or is it headless?

Several users have reported that downgrading to cypress@13.6.4 stopped the Vulkan error from appearing. (cypress@13.6.5 updated electron from 25.8.4 to 27.1.3.)

bobbhatti commented 6 months ago

@MikeMcC399

our giitlab runners are headless, however we are going look at downgrading chromium first and if needs be will downgrade cypress

MikeMcC399 commented 6 months ago

@bobbhatti

our giitlab runners are headless, however we are going look at downgrading chromium first and if needs be will downgrade cypress

Thanks for the feedback that your GitLab runners are headless!

I experimented with Vulkan on Ubuntu using:

sudo apt-get install -y vulkan-tools
vulkaninfo --summary

On a GitHub headless Ubuntu 22.04 runner the summary gives the error:

vkCreateInstance failed with ERROR_INCOMPATIBLE_DRIVER

On a local Ubuntu 22.04 system with GPU, executing vulkaninfo --summary locates the GPU and provides detailed information.

The Cypress source code shows no signs that it directly accesses Vulkan. The Electron source code on the other hand has multiple references to vulkan. Cypress includes Electron (for the exact version of Electron execute npx cypress version).

If there is no GPU available, Cypress uses Xvfb (X virtual framebuffer).

My theory is that the reports of the error

vkCreateInstance failed with ERROR_INCOMPATIBLE_DRIVER

appearing on headless systems are due to a bug in Electron erroneously trying to use Vulkan. In a headless system Vulkan shouldn't be used, only Xvfb. The issue is surfacing more often in cypress@13.6.5 and later which use Electron 27.1.3. Unfortunately I couldn't find any related Electron PRs or issues which described the problem outside of Cypress, so at the moment this remains an unproved theory.

bobbhatti commented 6 months ago

@MikeMcC399 thanks for the detailed information above. I've been going back through the gitlab jobs, to figure out when we first noticed it.

From what I can see it started on the 28th Feb 2024 for our gitlab jobs (cypress 13.6.6, chromium 122)

Also, interesting you mentioned Xvfb - we installed all the pre-reqs including Xvfb, and were initially triggering cypress run as xvfb-run -a cypress run-b chromium, and we then removed the xvfb-run -a, but still to no avail

At this moment, as long as it's not causing any of our cypress tests to break, we can live with it. I had my suspicions early on that there might be some cypress/latest version of chromium compatibility issues, and searched the git hub issues list in this repo on the 28th Feb - to be met with nothing being reported as yet back then - maybe I should have reported it back then, and figured someone would report this to cypress at some point. Ideally we want to be on the latest version of cypress and chromium and will only downgrade as a last resort (in the case that this suddenly starts breaking our tests)

MikeMcC399 commented 6 months ago

@bobbhatti

If your tests are not breaking, then I would ignore the error.

l00sed commented 6 months ago

I've started experiencing an issue that is showing some similar error/warning output to some of the others in this thread. I updated to: Cypress. 13.7.0 Node. 20.11.1

The error appears in both Chrome 122 and Electron 118 in headless mode:

Warning: terminator_CreateInstance: Failed to CreateInstance in ICD 1.  Skipping ICD.
Error: Loader Message: setup_loader_term_phys_devs:  Failed to detect any valid GPUs in the current config
Warning: vkEnumeratePhysicalDevices
at GatherPhysicalDevices (../../third_party/dawn/src/dawn/native/vulkan/VulkanInfo.cpp:131)
at Initialize (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:353)
at Create (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:266)
at operator() (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:521)

Subsequently, tests will execute, but they all now timeout. I'm using the --disable-gpu flag with Chrome too.

The build is running through a Bamboo agent in a CI/CD pipeline. AFAIK, there is no dedicated GPU.

UPDATE: I installed vulkan-tools on the build host and ran vulkaninfo --summary with the following output:

# vulkaninfo --summary
WARNING: [Loader Message] Code 0 : terminator_CreateInstance: Failed to CreateInstance in ICD 1.  Skipping ICD.
'DISPLAY' environment variable not set... skipping surface info
error: XDG_RUNTIME_DIR is invalid or not set in the environment.
ERROR: [Loader Message] Code 0 : setup_loader_term_phys_devs:  Failed to detect any valid GPUs in the current config
ERROR at vulkaninfo/vulkaninfo.h:237:vkEnumeratePhysicalDevices failed with ERROR_INITIALIZATION_FAILED

ANOTHER UPDATE: Worked out the reason for our failing Cypress tests (allowing external resources through a company proxy). I am now in the camp that these Vulkan errors can safely be ignored.

yktoo commented 6 months ago

Same problem with 13.7.0:

Warning: vkCreateInstance: Found no drivers!
Warning: vkCreateInstance failed with VK_ERROR_INCOMPATIBLE_DRIVER
    at CheckVkSuccessImpl (../../third_party/dawn/src/dawn/native/vulkan/VulkanError.cpp:88)
    at CreateVkInstance (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:458)
    at Initialize (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:344)
    at Create (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:266)
    at operator() (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:521)
l00sed commented 6 months ago

I had set the Cypress --headless flag and the --disable-gpu flag for Chrome to run in a headless environment. These errors are from Vulkan complaining that no GPU exists or the driver for your GPU doesn't exist. Vulkan has drivers and SDKs that are used so browsers can use your GPU to render content (to my understanding). In my case, it was a red herring. I'm guessing there's some place where Vulkan is still checking for a GPU. Although, that might not be happening in Cypress's code? Enabling the debug logs as suggested in the documentation ended up being helpful enough to point us to the actual problem that was preventing our tests from running correctly.

MetRonnie commented 6 months ago

Getting this error a lot suddenly with Firefox, no containerisation or anything. Here component tests started running successfully but then hit the VK_ERROR_INCOMPATIBLE_DRIVER error halfway through: https://github.com/cylc/cylc-ui/actions/runs/8363704833/job/22932503711?pr=1696

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress:        13.7.0                                                                         │
  │ Browser:        Firefox 124 (headless)                                                         │
  │ Node Version:   v20.8.1 (/home/runner/runners/2.314.1/externals/node20/bin/node)               │
  │ Specs:          5 found (boxPlot.cy.js, cylc-graph-node.cy.js, cylc-icons.cy.js, ganttchart.cy │
  │                 .js, viewToolbar.cy.js)                                                        │
  │ Searched:       **/*.cy.{js,jsx,ts,tsx}                                                        │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
yktoo commented 6 months ago

I had set the Cypress --headless flag and the --disable-gpu flag for Chrome to run in a headless environment.

@l00sed Does it also apply to headless Electron?

MikeMcC399 commented 6 months ago

@MetRonnie

l00sed commented 6 months ago

I had set the Cypress --headless flag and the --disable-gpu flag for Chrome to run in a headless environment.

@l00sed Does it also apply to headless Electron?

Yes, I saw the same thing with Electron, thanks for asking.

l00sed commented 6 months ago

Usually the error/warning from vk happens for us after a few minutes delay. So while debugging timeouts, it seemed like it might be happening at the beginning. Once the tests were running normally (quickly), we still see the vk warnings— but usually they'll show up somewhere randomly in the logs after about 5 minutes of tests have already completed.

yktoo commented 6 months ago

I had set the Cypress --headless flag and the --disable-gpu flag for Chrome to run in a headless environment.

@l00sed Does it also apply to headless Electron?

Yes, I saw the same thing with Electron, thanks for asking.

@l00sed I don't see --disable-gpu among known Electron switches, do you happen to remember what exactly you passed to Electron?

l00sed commented 6 months ago

I'd used --headless as a Cypress flag and set a conditional launch argument for Chrome to add --disable-gpu (only for Chrome). Electron is still using the --headless Cypress flag. I was still seeing the Vulkan error output regardless.

yktoo commented 6 months ago

I'd used --headless as a Cypress flag and set a conditional launch argument for Chrome to add --disable-gpu (only for Chrome). Electron is still using the --headless Cypress flag. I was still seeing the Vulkan error output regardless.

Well, according to the docs, --headless is the default for cypress run, so I'm afraid it won't make any difference.

Which is a bad news, for us it's a complete blocker now. Our tests simply get stuck forever.

l00sed commented 6 months ago

My only suggestion might be to try with DEBUG=cypress:* cypress run. For me this revealed that the tests were actually stuck because it was on a server that couldn't fetch resources from the public Internet. It was timing out trying to get fonts, scripts, GTM, etc.

CryZo commented 6 months ago

@l00sed thanks for pointing me in the right direction!

First I've followed your advice passing --disable-gpu to chrome. cypress.config.ts:

import { defineConfig } from 'cypress';

export default defineConfig({
  e2e: {
    ...
    setupNodeEvents(on, config) {
      on('before:browser:launch', (browser, launchOptions) => {
        if (browser.family === 'chromium') {
          launchOptions.args.push('--disable-gpu');
        }
        return launchOptions;
      })
    },
  }
});

Then I changed my cypress command from yarn run cypress run to yarn run cypress run --browser chrome. However after that Cypress started to argue, that chrome wasn't installed - which was (shame on me) absolutely true! So I added it to my dockerfile:

RUN apt install -y fonts-liberation libu2f-udev libvulkan1 wget xdg-utils
RUN curl -o /usr/src/google-chrome-stable_current_amd64.deb "http://dl.google.com/linux/chrome/deb/pool/main/g/google-chrome-stable/google-chrome-stable_120.0.6099.224-1_amd64.deb" && \
  dpkg -i /usr/src/google-chrome-stable_current_amd64.deb ; \
  apt-get install -f -y && \
  rm -f /usr/src/google-chrome-stable_current_amd64.deb

The result is not only, that my screenshots fitted better, the Vulkan error is gone as well. I don't know which of these steps fixed it, but maybe it helps somebody.

TestIsTest001 commented 6 months ago

I am facing the same issue with 13.7.0 and i cannot downgrade because i was facing this issue: https://github.com/cypress-io/cypress/issues/5016

Can some one please advise if this is a cypress issue or it is an issue of configs.. If there is some checks that needs to be done on my side to verify that it is not coming from our CI config pls do let me know what are the check that i need to do..

FYI i am using chrome 121 thanks

TestIsTest001 commented 6 months ago

Those who are using 13.7.0, is the warning only appearing or is also timing out the CI please?..I got the warning but so far it just appeared but did not block the CI so i need to know if it is interminent or it is really not blocking the pipeline