search

cypress-io / cypress

Fast, easy and reliable testing for anything that runs in a browser.
https://cypress.io
MIT License
47.1k stars 3.19k forks source link

Warning: terminator_CreateInstance: Failed to CreateInstance in ICD2. Skipping ICD #30401

Open weiners4 opened 1 month ago

weiners4 commented 1 month ago

Current behavior

I observed the following warning/error when running tests using Cypress v13.13.1 installed locally on RHEL v8.10 VM and also when using the Cypress:Included image v13.13.1 running Debian GNU/Linux 12

Warning: terminator_CreateInstance: Failed to CreateInstance in ICD2. Skipping ICD
Error: Loader Message: setup_loader_term_phys_devs: Failed to detect any vaild GPUs in the current config
Warning: vkEnumeratePhysicalDevices
 at GatherPhysicalDevices (../../vulkanInfo.cpp:131)
 at Initialize (../../BackendVk.cpp:353)
 at Create (../../BackendVk.cpp:266)
 at operator() (../../BackendVk.cpp:521)

As far as I can tell, the warning/error does not impact test execution or degrade performance as the tests still run to completion and the time duration of test execution is not increasing. I observed the warning/error being displayed from 1-3 times when running tests from the CLI in headless mode. The only pattern seems to be that the warning/error is displayed to stdout when running multiple tests in a group (eg. all E2E or API). If the tests are run one at a time independently then I never see this warning/error.

Desired behavior

The desired behavior is, if possible, to suppress these warnings/errors because they lead to confusion about whether there was a problem during test execution.

Test code to reproduce

I don't have the ability to publish a repo or my test specs or Cypress configuration.

Cypress Version

v13.13.1

Node version

v18.18.2 and v20.5.1

Operating System

RHEL v8.10 and Debian GNU/Linux 12

Debug Logs

I turned on the debugging in Cypress, but the debug didn't lend any clues as to what was the root cause of the warning/error

Other

I did a number of things to try and troubleshoot the root cause of the warning/error and was able to resolve the issue as per the explanation below. After making the changes described below, I do not see the error/warning anymore. The root cause appears to be how the Vulkan drivers are installed. -The Vulkan Loader provides an Installable Client Driver (ICD), which supports systems that have multiple GPUs and enables control over how the GPUs work. The Vulkan driver can be installed as Explicit, Implicit or ICD Manifest type. On my VM, the driver appears to have been installed as the Implicit Layer Manifest type because the driver file is located in /usr/share/vulkan/implicit_layer.d/. However when running Cypress tests, Cypress appears to be looking for what is known as the ICD Manifest type driver file and that file would be located in /usr/share/vulkan/icd.d/. There was no file in that directory on my VM, so I created a driver file and placed the file in that location as explained below.

-Several driver files were located in /usr/share/vulkan/implicit_layer.d/. Based on doing a search for RPMs with vulkan in the name (rpm -qa | grep -i vulkan) on my system, I determined that the mesa-vulkan-drivers were installed. The driver file installed in the implicit_layer.d/ directory was VkLayer_MESA_device_select.json

--I created a new file with the same name in the icd.d/ directory --The file should have root as the owner:group and permissions of 644, just like the other driver files in icd.d/ --The content of the new driver file created in icd.d/ was similar to the other driver files in icd.d/. In the new driver file the api_version and library_path were updated to match what was found in the driver file in implicit_layer.d/VkLayer_MESA_device_select.json

--The location and name of the new driver file is /usr/share/vulkan/icd.d/VkLayer_MESA_device_select.json --The content of the json file is shown below (normal json indentation not shown) --Once this file is created with the required content and placed in the icd.d/ directory the warning/error should no longer be displayed to stdout during test execution

Note that for library_path the fullpath to the driver shared object file is used { "ICD": { "api_version": "1.3.211" "library_path": "/usr/lib64/libVkLayer_MESA_device_select.so" }, "file_format_version": "1.0.0" }

weiners4 commented 1 month ago

Should these be linked to both: https://github.com/cypress-io/cypress/pull/29278 https://github.com/cypress-io/cypress/issues/29085

MikeMcC399 commented 1 month ago

@weiners4

I have also seen the message occurring from time to time, without the warning interrupting test execution:

Warning: terminator_CreateInstance: Failed to CreateInstance in ICD2. Skipping ICD

I would assume that it is another "garbage warning" like the following may also be:

Typically, these warnings either need to be suppressed in cli/lib/exec/spawn.js or they go away when Electron is updated (which sometimes introduces new garbage warnings!).

The issues you quoted are related, and useful as background reading, however they are already closed:

MikeMcC399 commented 1 month ago

Confirmed with Cypress 13.15.0 in Ubuntu 22.04.1 LTS with Node.js v20.12.0 under VMware Workstation 17.6.1

For example using https://github.com/cypress-io/cypress-realworld-app:

  Running:  ui/notifications.spec.ts                                                      (14 of 21)

  Notifications
Warning: terminator_CreateInstance: Failed to CreateInstance in ICD 4.  Skipping ICD.
    ✓ renders an empty notifications state (2641ms)
    notifications from user interactions
      ✓ User A likes a transaction of User B; User B gets notification that User A liked transaction  (4802ms)
      ✓ User C likes a transaction between User A and User B; User A and User B get notifications that User C liked transaction (4824ms)
      ✓ User A comments on a transaction of User B; User B gets notification that User A commented on their transaction (3308ms)
      ✓ User C comments on a transaction between User A and User B; User A and B get notifications that User C commented on their transaction (5195ms)
      ✓ User A sends a payment to User B (3070ms)
      ✓ User A sends a payment request to User C (2380ms)

In this environment:

$ sudo apt install vulkan-tools
$ vulkaninfo --summary
WARNING: [Loader Message] Code 0 : terminator_CreateInstance: Received return code -3 from call to vkCreateInstance in ICD /usr/lib/x86_64-linux-gnu/libvulkan_virtio.so. Skipping this driver.

Also seeing this warning running Ubuntu directly on laptop x86 hardware (with GPU).

jordi72 commented 2 weeks ago

I'm suffering this issue in Cypress 13.15.1 in Ubuntu 22.04.1 LTS with NodeJS 22.10.0 inside Azure Devops

Warning: terminator_CreateInstance: Failed to CreateInstance in ICD 4. Skipping ICD.

After this message, almost everything starts to crash.

I'm running Cypress in Chrome 130.x.x headless With this export in the pipeline prior to the execution of cypress, trying to minimize this issue: export XDG_RUNTIME_DIR=/tmp/runtime

MikeMcC399 commented 2 weeks ago

@jordi72

Based on previous similar GPU-related error messages and the fact that https://github.com/cypress-io/cypress-realworld-app runs successfully to completion, even though it emits the warning message, there seems to be a high likelihood that this is a garbage error and that test failures aren't actually caused by the warning.

I ran the same test mentioned above in https://github.com/cypress-io/cypress/issues/30401#issuecomment-2411098992 using a beta version of Cypress 14.0.0 from https://github.com/cypress-io/cypress/commit/bd89ca710f14e3a62c4a02ab3cbe6d83fb0fb4aa and the error message was no longer output. The tests also ran successfully to completion.

Cypress 14 beta updates the version of Electron, and I assume that this has caused the warning message to disappear. Again, just based on gut-feel and previous experience. This hasn't been announced for testing yet, but anybody can follow the instructions on https://docs.cypress.io/app/references/advanced-installation#Install-pre-release-version to install a pre-release version for testing purposes. The branch is release/14.0.0, not the develop branch.