shaka-project / shaka-player

JavaScript player library / DASH & HLS client / MSE-EME player
Apache License 2.0
7.17k stars 1.34k forks source link

FirefoxWindows timeout running DrmEngine.probeSupport #7449

Open joeyparrish opened 1 week ago

joeyparrish commented 1 week ago

Have you read the FAQ and checked for duplicate open issues? Yes

If the problem is related to FairPlay, have you read the tutorial?

N/A

What version of Shaka Player are you using?

main

Can you reproduce the issue with our latest release version? Yes

Can you reproduce the issue with the latest code from main? Yes

Are you using the demo app or your own custom app? Integration tests

If custom app, can you reproduce the issue using our demo app? N/A

What browser and OS are you using? Firefox 130 on Windows 10.

For embedded devices (smart TVs, etc.), what model and firmware version are you using? N/A

What are the manifest and license server URIs?

N/A: Integration tests

What configuration are you using? What is the output of player.getNonDefaultConfiguration()?

N/A: Integration tests

What did you do?

Run tests in the lab with --browsers FirefoxWindows --filter 'unloads properly after DRM error'

What did you expect to happen? Tests pass

What actually happened?

The one test in the filter times out

Are you planning send a PR to fix it? Yes

joeyparrish commented 1 week ago

The root cause seems to be the test's call to DrmEngine.probeSupport(), which doesn't complete for more than 120 seconds on FirefoxWindows, causing the test to time out.

Furthermore, the test gets skipped on FirefoxWindows because Widevine support is not detected.

The platform support check runs in less than 2 seconds on both FirefoxLinux and FirefoxMac, and both correctly detect Widevine support.

joeyparrish commented 1 week ago

On our Firefox Windows instance, one test for Widevine, no robustness string, no encryption scheme, through EME, takes 60 seconds.

The same through MediaCapabilities takes 120 seconds.

Running many of these in parallel with various schemes and robustness strings also takes 120 seconds, which is the only good news in my analysis of this so far. 😐

Checking ClearKey takes less than 1 second, so the issue is specific to Widevine.

joeyparrish commented 1 week ago

The error from the EME query is "Timed out while waiting for a CDM update".

After going back through our nightly test history, this issue started after we upgraded our lab from Firefox 126 to 130. According to https://bugzilla.mozilla.org/show_bug.cgi?id=1908144, they switched to Chrome's component update service to get the Widevine CDM in Firefox 129.

joeyparrish commented 1 week ago

I can confirm that outside of test automation and geckodriver, on startup at https://shaka-player-demo.appspot.com/support.html, Firefox seems to need to reload the page after downloading the Widevine CDM. This all happens really fast in a live session.

I tried using a fixed profile folder, so that Widevine could be pre-installed there, but it hangs without starting tests. In experiments running the Selenium node for Firefox outside of the service account, I can see Firefox start up, but it never receives a command to go to the test URL.

If I run Selenium outside of the headless service account without a custom profile folder, everything just works, including downloading, installing, and running the Widevine CDM without issue.

So the root trigger is headless mode in a service account. The profile folder, which attempts to work around that issue, causes a different, unknown issue, and is not a good workaround.

joeyparrish commented 1 week ago

Running outside the service account, in a UI-enabled environment, but with the "-headless" argument to Firefox, the CDM does work.

NB: Running the service scripts outside the service account, as an admin, might have broken some file permissions and forced me to wipe and reinstall shaka-lab-node. Oops.

Running inside the service account, without a UI, but with the "-headless" argument to Firefox, the CDM does not work.

So the service account environment is the key to triggering this, regardless of whether Firefox is run "headless".