testcafe times out waiting for browser.

[flackr]

I'm sure it's not a problem with VTT, but everytime I run testcafe I get a timeout waiting to establish a connection with the browser:

$ npm run testcafe-firefox-headless-all

> virtualtabletop@0.5.0 testcafe-firefox-headless-all
> testcafe firefox:headless tests/testcafe

Browserslist: caniuse-lite is outdated. Please run:
  npx browserslist@latest --update-db
  Why you should do it regularly: https://github.com/browserslist/browserslist#browsers-data-updating
ERROR Cannot establish one or more browser connections.
1 of 1 browser connections have not been established:
- firefox:headless

Hints:
- Increase the value of the "browserInitTimeout" option if it is too low (currently: 2 minutes for local browsers and 6 minutes for remote browsers). This option determines how long TestCafe waits for browsers to be ready.
- The error can also be caused by network issues or remote device failure. Make sure that your network connection is stable and you can reach the remote device.

Type "testcafe -h" for help.

Any thoughts what could be causing this?

[robartsd]

Responding from #1501

Thanks @robartsd for the test update! I filed a separate issue #1565 for issues around running testcafe. I'll try your instructions out on my system.

Is your system running the server in a container? Testcafe would also be running in the container, but would normally be unable to connect to a browser outside the container.

If your container is using a Debian based system, npm run testcafe-gitpod-all should work to run headless tests and the gitpod noVNC script should also work if you need to add graphical output to your container (may need to configure your container management to give you access to the noVNC port outside the container).

If your container is using a packaging system that is not compatible with Debian, use it's package management system to install a browser then running testcafe headless with that browser should work (you may need to do more research if you also need to add graphical support to your container).

[flackr]

Responding from #1501

Thanks @robartsd for the test update! I filed a separate issue #1565 for issues around running testcafe. I'll try your instructions out on my system.

The instructions seem very specific to gitpod. In particular, they set up a VNC server but I have a local DISPLAY I can use and see / interact with.

Is your system running the server in a container? Testcafe would also be running in the container, but would normally be unable to connect to a browser outside the container.

My system isn't running in a container, just straight on my Ubuntu machine (specifically 22.04.1 LTS ). This used to work just directly running npm run testcafe-firefox-headless-all or npm run testcafe-firefox-all. Is running outside of a container not supported anymore?

[robartsd]

The instructions seem very specific to gitpod. In particular, they set up a VNC server but I have a local DISPLAY I can use and see / interact with.

Yes, the instructions are specific to running in a container that does not have access to a local display (not actually specific to Gitpod).

Is your system running the server in a container? Testcafe would also be running in the container, but would normally be unable to connect to a browser outside the container.

My system isn't running in a container, just straight on my Ubuntu machine (specifically 22.04.1 LTS ). This used to work just directly running npm run testcafe-firefox-headless-all or npm run testcafe-firefox-all. Is running outside of a container not supported anymore?

I don't think that there should be any issues. I'm pretty sure @ArnoldSmith86 normarly develops on a local server and tests with firefox, so I would expect issues running testcafe in such environments would have been noticed.

[flackr]

Hm, so actually when I run npm run testcafe-firefox-all I get an error message saying "Your firefox profile cannot be loaded. It may be missing or inaccessible". I don't get any such error when just running firefox. Looking this up, this might be https://github.com/DevExpress/testcafe/issues/4974 to which the solution seems to be either don't use snap (however snap seems to be the thing that is used when you apt install firefox in ubuntu) or to pass a custom profile to testcafe per https://testcafe.io/documentation/402828/guides/intermediate-guides/browsers#user-profiles-enable-extensions-and-custom-settings. I tried adding :userProfile to the firefox target and it does indeed launch the browser but all of the tests fail with the following:

✖ Public library: Scoreboard (variant 2)

   1) - Error in fixture.beforeEach hook -
      FetchError: request to http://localhost:8272/state/testcafe-testing failed, reason: connect ECONNREFUSED 127.0.0.1:8272

      Browser: Firefox 109.0 / Linux 0.0

         1486 |                         }, request.timeout);
         1487 |                 });
         1488 |         }
         1489 |
         1490 |         req.on('error', function (err) {
       > 1491 |                 reject(new FetchError(`request to ${request.url} failed, reason: ${err.message}`, 'system', err));
         1492 |                 finalize();
         1493 |         });
         1494 |
         1495 |         req.on('response', function (res) {
         1496 |                 clearTimeout(reqTimeout);

         at ClientRequest.<anonymous> (/home/flackr/projects/virtualtabletop/node_modules/node-fetch/lib/index.js:1491:11)

   2) - Error in fixture.after hook -
      FetchError: request to http://localhost:8272/state/testcafe-testing failed, reason: connect ECONNREFUSED 127.0.0.1:8272

      Browser: Firefox 109.0 / Linux 0.0

         1486 |                         }, request.timeout);
         1487 |                 });
         1488 |         }
         1489 |
         1490 |         req.on('error', function (err) {
       > 1491 |                 reject(new FetchError(`request to ${request.url} failed, reason: ${err.message}`, 'system', err));
         1492 |                 finalize();
         1493 |         });
         1494 |
         1495 |         req.on('response', function (res) {
         1496 |                 clearTimeout(reqTimeout);

         at ClientRequest.<anonymous> (/home/flackr/projects/virtualtabletop/node_modules/node-fetch/lib/index.js:1491:11)

   3) Failed to load the page at "http://localhost:8272/testcafe-testing".
      Increase the value of the "pageRequestTimeout" variable, enable the "retryTestPages" option, or use quarantine mode to perform additional attempts to execute this test.
      You can find troubleshooting information for this issue at "https://go.devexpress.com/TestCafe_FAQ_ARequestHasFailed.aspx".

      Error details:
      Failed to find a DNS-record for the resource at "http://localhost:8272/testcafe-testing".

      Browser: Firefox 109.0 / Linux 0.0