cypress-io / cypress

Fast, easy and reliable testing for anything that runs in a browser.
https://cypress.io
MIT License
46.58k stars 3.15k forks source link

Video functionality is not working on Firefox 93 and above #18415

Open pranavj1001 opened 2 years ago

pranavj1001 commented 2 years ago

Current behavior

Cypress sessions are failing to record videos on Firefox 93 and 94. I checked this behavior is happening on both Windows 10 and macOS Big Sur.

Affected Cypress Versions: Looks like all (confirmed on 6.8.0, 8.1.0, 8.4.1, 8.5.0) Affected OS: Looks like all (confirmed on macOS Big Sur and Windows 10)

Error Details Screenshot 2021-10-08 at 9 25 51 PM Screenshot of a spec (actions.spec.js - kitchen-sink) run on Firefox 93.

Desired behavior

Cypress sessions should be able to record videos on Firefox 93 and 94. It should work the way it's working on older Firefox versions and other browsers.

Test code to reproduce

Steps to reproduce this error:

  1. Make sure you have Firefox 93 or 94 as this error is specific to Firefox 93 and above browsers only.
  2. Make sure you have the cypress installed.
  3. Run any spec of your choice. You may use actions.spec.js from the kitchen-sink examples
  4. Observe that all the test cases pass but video processing was failed. Refer to the attached screenshot above.

Cypress Version

all

Other

No response

Edits

  1. 8th November 2021 - Updated the issue a bit to point that the error is happening on Firefox 94 too.
kpturner commented 2 years ago

This is the same for version 8.7.0 running on an AWS EC2 Ubuntu instance using Firefox 93 and above. Chrome and Electron just crash completely and are unusable (on Ubuntu at least) so Firefox is the only current option.

  1 passing (12s)

Warning: We failed processing this video.

This error will not alter the exit code.

TimeoutError: operation timed out
    at afterTimeout (/home/ubuntu/.cache/Cypress/8.7.0/Cypress/resources/app/packages/server/node_modules/bluebird/js/release/timers.js:46:19)
    at Timeout.timeoutTimeout [as _onTimeout] (/home/ubuntu/.cache/Cypress/8.7.0/Cypress/resources/app/packages/server/node_modules/bluebird/js/release/timers.js:76:13)
    at listOnTimeout (internal/timers.js:555:17)
    at processTimers (internal/timers.js:498:7)
laerteneto commented 2 years ago

The same happing over here on Windows 10 and Firefox 93. Cypress version: 9.2.1

image

HamboneWilson commented 2 years ago

Experiencing the same issue on Windows 10 and Firefox 93 Cypress version: 8.7.0

BlueWinds commented 2 years ago

Firefox 93 introduced a change where it now requires a recent user gesture in order to activate screen recording - https://bugzilla.mozilla.org/show_bug.cgi?id=1729889.

https://github.com/cypress-io/cypress/blob/develop/packages/driver/src/cy/video-recorder.ts#L11 is currently throwing an error, DOMException: Display capture requires transient activation from a user gesture. because of this change. We'll need to update Cypress to trigger some sort of user gesture before we attempt to record video.

magarwal19 commented 2 years ago

facing this issue with cypress 9.5 also No videos are getting recorded for Firefox execution OS: macOS Monterey 12.2.1 Firefox version: 97.0.1

shivankgoel commented 2 years ago

Facing similar issue, +1 to resolve this Cypress: 7.4.0 Browser: Firefox 97.0

magarwal19-wellsky commented 2 years ago

@jennifer-shehane can you please help with this? i am using firefox for my tests and not capturing videos is major drawback for us

laerteneto commented 2 years ago

The issue is still observed on Firefox 98, Windows 10, running Cypress 9.6.0

image

jpaquit commented 2 years ago

In some cases, could be a Windows maximum file path limitation. https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation

Experienced same "bug" as ffmpeg intends to create a file with a path length of 265 characters. To confirm behaviour, try to create path "by hand" in Windows Explorer: you should not be able to write the full filename, meaning the maximum path limitation has been reached.

After win32 long path name feature activation (described in the link above), I am able to create by hand the expected file in windows terminal (cmd), but not in explorer, neither ffmpeg.

A quick search about this ffmpeg behaviour leads me here: https://trac.ffmpeg.org/ticket/8885

As a quick workaround, I created a virtual hard drive "J:" pointing to my repository root in order to save some characters:

C:>subst J: "C:\Users\Julien PAQUIT\Documents\git\softwaretests" C:>j: J:>npx cypress --spec whatever

Cypress : v9.5.1 Windows: v10.0.19044 Node.js: v14.17.3 Chrome: v101

kaiyoma commented 2 years ago

Also running into this (Cypress 9.6.1, Firefox 100, Windows 10 and Linux). How do we dynamically alter our Cypress config at runtime to disable videos for only Firefox?

todd-m-kemp commented 2 years ago

The lack of video recording for Firefox is a big challenge for my organization. Until recently we were using only Chrome for our test runs but we decided that we needed to diversify the browser we use for our tests. When we experience failing or flaky results in tests that run in Firefox, not having a video recording show up in the Cypress Dashboard makes it very difficult (and in some cases impossible) to understand the underlying cause of the failure/flake. Instead of being able to fix the problem with the test, we just have to leave it and wait for the issue to inevitably occur again. Having video functionality restored for test runs in Firefox is critically important for our ability to have a flake-free test experience using Firefox as the test browser.

salvatore-piccione commented 2 years ago

Hello, I was able to reproduce this issue with the following configuration: Cypress: 9.7 Firefox: 101 OS: Debian 9 (actually running on Docker image based on node:14).

Boojapho commented 2 years ago

Reproduced with the following configuration: Cypress: 10.3.1 Firefox: 91 OS: RedHat 8 (docker image)

Chrome video capture works fine.

Wilhop commented 2 years ago

Same issue, video is created on Chrome but not with Firefox. Cypress: 10.3.1 Firefox: 103.0.1 Warning: We failed processing this video. This error will not alter the exit code. TimeoutError: operation timed out at afterTimeout (C:\Users\villevi\AppData\Local\Cypress\Cache\10.3.1\Cypress\resources\app\packages\server\node_modules\bluebird\js\release\timers.js:46:19) at Timeout.timeoutTimeout [as _onTimeout] (C:\Users\villevi\AppData\Local\Cypress\Cache\10.3.1\Cypress\resources\app\packages\server\node_modules\bluebird\js\release\timers.js:76:13) at listOnTimeout (node:internal/timers:557:17) at processTimers (node:internal/timers:500:7)

Abcmsaj commented 2 years ago

Just want to add my 2c to this - we're getting it now as well for Cypress 10.4.0 and 10.5.0. We have recently added a nested directory, and a spec within that dir is warning about being unable to record a video, and then cypress dashboard hangs. The CLI shows that the video could not be found. This is actually more relevant to discussions in https://github.com/cypress-io/cypress/issues/9128 but that was closed off and merged into this... we're also running Chrome!

The "fix" we found was to add trashAssetsBeforeRuns: false but that doesnt feel like much of a fix as it could potentially be looking for old videos, seeing they exist, and uploading them to the dashboard. Doesnt throw the warning for recording the video anymore tho

bensonhi commented 1 year ago

Same here. Running on Cypress: 9.7.0 Firefox 104 (headless) v14.19.3 (/usr/bin/node)
Screen Shot 2022-09-05 at 12 08 16 PM

magarwal19 commented 1 year ago

Do we have any update on this fix? it have been open since long and it is impacting automation for my application

preetibellad commented 1 year ago

I am facing the same issue with Cypress version 10.9.0 / 10.10.0 and Firefox 105. Chrome videos are getting generated but not for Firefox run. Can we have the fix ASAP please?

SkyZeroZx commented 1 year ago

I am facing the same issue with Cypress version 10.9.0 / 10.10.0 and Chrome . Image of error image

tballast commented 1 year ago

Having the same issue.

Cypress: 11.2.0 Firefox 107

image

kingsonlo commented 1 year ago

Also getting the same issue, Cypress: 9.7.0, Firefox 99 (headless). Any updates on a fix?

ghost commented 1 year ago

I am facing the same issue with the following information:

KevinPusateri commented 1 year ago

Have you news on this topic? Any workaround?

MikeMcC399 commented 1 year ago

@KevinPusateri

Any workaround?

A workaround to avoid the error message is to disable videos (see https://docs.cypress.io/guides/references/configuration#Videos).

Hopefully the label https://github.com/cypress-io/cypress/labels/stage%3A%20backlog on this issue means that it will soon be picked up for resolution.

mmonfared commented 1 year ago

I'm facing this issue as well. I've created a simple Reproduction here:

https://github.com/mmonfared/cypress-issue-18415/actions/runs/4115105978

Run locally: npx cypress run --browser firefox doesn't create video. Run in CI: There are 2 stages for chrome and firefox, I got the artifacts from Chrome but not from Firefox.

I've tested with Mac/Ubuntu/Windows | Firefox version 109

mirobo commented 1 year ago

Same issue here, but no a blocker for me (still get videos for Chrome and Electron and if an issue occurred I have to reproduce it anyway locally to find the cause)

Cypress: 10.7.0 Browser: Firefox 110 (headless) Node Version: v18.14.0 (/usr/bin/node) OS: Debian in Docker

Warning: We failed processing this video.
This error will not alter the exit code.
TimeoutError: operation timed out
    at afterTimeout (/xyz/cypress-cache/10.7.0/Cypress/resources/app/packages/server/node_modules/bluebird/js/release/timers.js:46:19)
    at Timeout.timeoutTimeout [as _onTimeout] (/xyz/cypress-cache/10.7.0/Cypress/resources/app/packages/server/node_modules/bluebird/js/release/timers.js:76:13)
    at listOnTimeout (node:internal/timers:559:17)
    at processTimers (node:internal/timers:502:7)
kriegaex commented 1 year ago

Same problem in my customer's Jenkins CI environment, Cypress 10.11.0, FF 103.0.2, NodeJS 14. In the same Docker image, running the build against Chrome works nicely.

Warning: We failed processing this video.

This error will not alter the exit code.

TimeoutError: operation timed out
    at afterTimeout (/opt/app/cypress/.cache/Cypress/10.11.0/Cypress/resources/app/packages/server/node_modules/bluebird/js/release/timers.js:46:19)
    at Timeout.timeoutTimeout [as _onTimeout] (/opt/app/cypress/.cache/Cypress/10.11.0/Cypress/resources/app/packages/server/node_modules/bluebird/js/release/timers.js:76:13)
    at listOnTimeout (node:internal/timers:559:17)
    at processTimers (node:internal/timers:502:7)
Narretz commented 1 year ago

How do other e2e test runners do video recording? Playwright seems to do it, no?

jpaquit commented 1 year ago

I was having an issue with one test ; I made a dirty replacement of onboarded ffmpeg.exe binary (ffmpeg version N-92722-gf22fcd4483 Copyright (c) 2000-2018 the FFmpeg developers built with gcc 8.2.1 (GCC) 20181201) with one provided by a link on ffmpeg.org (ffmpeg version N-110014-ga6e9d01f88-20230315 Copyright (c) 2000-2023 the FFmpeg developers built with gcc 12.2.0 (crosstool-NG 1.25.0.90_cf9beb1)) and it solved my issue :)

Thanks to set DEBUG=cypress:server:video I was able to retrieve path of ffmpeg.exe binary image

May this help?

kriegaex commented 1 year ago

@jpaquit, you seem to work on Windows. Can anyone confirm that the same approach (replacing the npm-installed ffmpeg by a pre-built static version) solves the problem on Linux?

I was having an issue with one test

Does that mean, before the workaround videos were recorded for all but one test, or do you only have one test? I would like to know if you really were experiencing the same general problem described here.

kriegaex commented 1 year ago

Replacing the embedded ffmpeg with a statically compiled 6.0 version from the download site does not change the behaviour for us by one iota. I let the tests run with DEBUG=cypress:server*, but nothing helpful jumped at me with regard to video processing. There is lots of logging, but only about other parts of Cypress. I can however provide the logs, if one of the Cypress committers/maintainers would deign to respond and actually engage in trying to resolve this issue. Just tell me what information you need, and you shall have it ASAP.

jpaquit commented 1 year ago

@kriegaex, yep working under w10. I have a huge amount of test cases, and lately, trying to test for regressions, one particular test case failed because of ffmpeg. As previously stated in this thread (https://github.com/cypress-io/cypress/issues/18415#issuecomment-1125942005), I fixed some issues by allowing long path names under W10, but ffmpeg seemed to suffer some sort of very similar limitation. That's why I tried to replace ffmpeg binary, with success.

My point here is that Cypress is a framework with lots of dependencies that live their own live far away from Cypress, and maybe some non-breaking dependency updates could fix your issue as they are not strictly related to Cypress.

I first tried to dumbly "npm-upgrade" ffmpeg-installer from within the Cypress cache, but it turns that too many dependencies have to be updated, making Cypress unable to start. That's the reason why I walked around npm and squashed the binary with a newer (4 years !) one. But an older one may solve the issue, too !

Cypress 9.7.0 embeds bluebird 3.5.3 ; last (3 years old) bluebird version on NPM is 3.7.2 Edit : my bad, server embeds 3.7.2, still seeking for something that works...

kriegaex commented 1 year ago

The thing is, I just have the Cypress log to work with, and it does not tell me where to look for the problem's root cause in ffmpeg, Firefox or wherever else. It just says there is a timeout, which is not particularly helpful. Maybe there are missing codecs, native or JS libraries, whatever - I simply have no clue. I would go and bother another project's maintainers with issues, if I knew what causes the problem. But I do not. I just know that despite ffmpeg being called, there is no video in the output folder.

jpaquit commented 1 year ago

Seems like there is nothing in the pipe when using Firefox: no frame equals no video. I can read this in the debug log: image Is this what we are looking for?

jpaquit commented 1 year ago

Trying to fix various issues shown in cypress:launcher : image

Could be fixed by adding the appropriate properties inside plugin index file:

module.exports = (on) => {
  on('before:browser:launch', (browser = {}, launchOptions) => {
    if (browser.family === 'firefox') {
      // launchOptions.preferences is a map of preference names to values
      launchOptions.preferences['fission.bfcacheInParent'] = false;
      launchOptions.preferences['fission.webContentIsolationStrategy'] = 0;
      return launchOptions;
    }
  });
}

Removes the warning but does not fix the current issue.

jpaquit commented 1 year ago

Running with --headed and double/triple-clicking the white background (iframe that displays application under test) while cypress loads spec makes video recording work. image

As previously stated, the issue stands probably here: image

kkcheng0816 commented 1 year ago

Same issue here, but get video for Chrome Cypress: 12.10.0 Browser: Firefox 113 (headed) Node Version: v19.9.0 OS: MacOS 13.3.1

Warning: We failed processing this video.

This error will not affect or change the exit code.

TimeoutError: operation timed out at afterTimeout (/Users/.../Library/Caches/Cypress/12.10.0/Cypress.app/Contents/Resources/app/node_modules/bluebird/js/release/timers.js:46:19) at Timeout.timeoutTimeout [as _onTimeout] (/Users/.../Library/Caches/Cypress/12.10.0/Cypress.app/Contents/Resources/app/node_modules/bluebird/js/release/timers.js:76:13) at listOnTimeout (node:internal/timers:559:17) at process.processTimers (node:internal/timers:502:7)

kriegaex commented 1 year ago

Sorry for the ping, but is anyone from the Cypress team ever going to fix this after two years? Or would, for starters, someone at least deign to give this some attention? How can you let down users like this?

jennifer-shehane commented 10 months ago

@kriegaex The last we investigated, this was a problem introduced by Firefox. This work is unfortunately not being prioritized at the moment.

kriegaex commented 10 months ago

@jennifer-shehane, thanks for replying after 2 years. Please describe, how and what you investigated and how you determined the problem to be in FF. Please also share the link to the corresponding FF bug, which surely back then you must have created or commented on. Maybe you even have a regression test for it in your code base, to find out whether the upstream bug has been fixed. Thank you.

P.S.: I am kind of surprised about how this problem can have such low priority, given the fact that FF is not a particularly exotic browser and recording videos is one of the main Cypress features. Especially in case of test failures, a video is usually much more helpful than a screenshot.

jennifer-shehane commented 9 months ago

This was one of the issues with Firefox video: https://github.com/cypress-io/cypress/issues/6337

kriegaex commented 9 months ago

@jennifer-shehane, thanks for the link. At first glance, it does not seem to be related.

adnanerlansyah403 commented 9 months ago

Any updates guys about this issue ?, I just got the problem here. I know there're must be better way to fix this issue.

todd-m-kemp commented 7 months ago

Is there an intention to fix this at some point? Or is video recording being phased out in favour of Test Replay? I ask because we're in the process of upgrading to Cypress 13 and we're wondering if we should just do video: false since we have Test Replay for Chromium and videos don't work in Firefox anyway. 🤔

Kualdir commented 7 months ago

Just came across this same issue. Recording works fine for Chrome and Edge but not for Firefox.

jennifer-shehane commented 7 months ago

@todd-m-kemp video: false is set by default in v13. We don't currently have this in our backlog to fix Firefox video. We are actively developing and improving Test Replay capturing. It's a fair assumption that more of our efforts will go towards Test Replay than video support, but we haven't made a decision to no longer support video in the future.

Kualdir commented 5 months ago

@jennifer-shehane If more dev time is going towards Test Replay. Would it be possible to have a feature to compress these Test Replays in a file we can send to our CI/CD pipeline artifacts (Bamboo) so we are able to use it in that flow? Currently how I understand Test Replays in cypress run are only accessible via the paid Cypress Cloud.

jennifer-shehane commented 5 months ago

@Kualdir Thanks for the feedback, I'll note this with our product team. Test Replay is available with any Cypress Cloud account. These types of features are the drivers behind keeping our company in business so that we can support the open source product.