[e2e]: unable to run full e2e test suite locally on either chrome or firefox

[plasmacorral]

Describe the bug

I have a mac M1 running Ventura 13.4 and I am not able to run the full test suite locally on either chrome or firefox. I can run individual tests with no issue, but even when all tests are passing in CircleCI I cannot seem to get through the entire e2e suite locally.

At this moment I have issues running chrome e2e until we have the chrome driver update merged. When I try to run e2e on chrome against commit 2645768e6eb6a99fddc214ed0cd4090585c7593f I get the error below. When I try to run e2e on develop commit 2958c2230b1d335fe9c6bef4e2eb3dad29a7b60e I generate the same error.

Right now may not be the best time, since it looks like we also have some e2e failures presenting in CI. However, even when CI e2e 100% passes (like it did with https://github.com/MetaMask/metamask-extension/pull/19283) I was still producing these errors and was not able to complete the full e2e suite locally.

Steps to reproduce

1. Checkout a branch that passes e2e in CI
2. then yarn
3. then yarn build:test
4. then yarn test:e2e:chrome or yarn test:e2e:firefox
5. Note that all tests are not completed

Error messages or log output

Error: Exited with code '1'
at runInShell (/Users/~/metamask-extension/deve lopment/lib/run-command. js: 134:29)
at processTicksAndRejections (node: internal/process/task_queues:96:5)
at async/Users/~/metamask-extension/test/e2e/run-e2e-test.js:101:5 at async retry (/Users/~/metamask-extension/development/lib/retry.js:30:7)
async main (/Users/~/metamask-extension/test/e2e/run-e2e-test.js:100:3)
Error: Retry limit reached
retry (/Users/~/metamask-extension/development/lib/retry.js:39:9)
at processTicksAndRejections (node: internal/process/task_queues:96:5)
at async main (/Users/~/metamask-extension/test/ee/run-e2e-test.is: 100:3)
Error: Exited with code '1'
at runInShell(/Users/~/metamask-extension/development/lib/run-command.js:134:29)
at processTicksAndRejections (node: internal/process/task_queues:96:5)
at async main (/Users/~/metamask-extension/test/e2e/run-all.js: 107:5)

Version

10.32

Build type

None

Browser

Chrome, Firefox

Operating system

MacOS

Hardware wallet

No response

Additional context

No response

[plasmacorral]

Just an update here to say that I still experience the inability to complete any tests locally on Chrome when I have v116 installed but our tests use Chromedriver 114.

Today this is on Apple silicon running Ventura 13.5.2 with our latest develop commit 4b23ea8c95bea9ea12336537bb6bda4568a99098

When I try to run the full test suite on Firefox here is my result:

➜  metamask-extension git:(develop) yarn test:e2e:firefox                                            
bigint: Failed to load bindings, pure JS will be used (try npm run rebuild?)
This version of µWS is not compatible with your Node.js build:

Error: Cannot find module './uws_darwin_arm64_108.node'
Falling back to a NodeJS implementation; performance may be degraded.

<testsuite name="Mocha Tests" tests="2" failures="0" errors="1" skipped="0" timestamp="Tue, 12 Sep 2023 20:37:31 GMT" time="42.728">
<testcase classname="4byte setting" name="makes a call to 4byte when the setting is on" time="23.241"><failure>waitUntilXWindowHandles timed out polling window handles
Error: waitUntilXWindowHandles timed out polling window handles
    at Driver.waitUntilXWindowHandles (test/e2e/webdriver/driver.js:379:15)
    at /Users/~/metamask-extension/test/e2e/tests/4byte-directory.spec.js:25:22
    at openDapp (test/e2e/helpers.js:109:5)
    at Context.&#x3C;anonymous&#x3E; (test/e2e/tests/4byte-directory.spec.js:5:3)</failure></testcase>
<testcase classname="4byte setting" name="does not try to get contract method name from 4byte when the setting is off" time="19.458"/>
</testsuite>
Error: Exited with code '1'
    at runInShell (/Users/~/development/lib/run-command.js:134:29)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /Users/~/test/e2e/run-e2e-test.js:124:5
    at async retry (/Users/~/development/lib/retry.js:36:7)
    at async main (/Users/~/test/e2e/run-e2e-test.js:123:3)
Error: Retry limit reached
    at retry (/Users/~/development/lib/retry.js:50:9)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async main (/Users/~/test/e2e/run-e2e-test.js:123:3)
Error: Exited with code '1'
    at runInShell (/Users/~/development/lib/run-command.js:134:29)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async main (/Users/~/test/e2e/run-all.js:161:5)

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity in the last 90 days. It will be closed in 45 days if there is no further activity. The MetaMask team intends on reviewing this issue before close, and removing the stale label if it is still a bug. We welcome new comments on this issue. We do not intend on closing issues if they report bugs that are still reproducible. Thank you for your contributions.

[github-actions[bot]]

This issue was closed because there has been no follow up activity in the last 45 days. If you feel this was closed in error, please reopen and provide evidence on the latest release of the extension. Thank you for your contributions.