Firefox doesn't run on GitLab (as root) #1220

Closed pkly closed 1 month ago

Hello, I've build an image adding docker stuff on top of cypress/included:cypress-13.15.0-node-20.17.0-chrome-129.0.6668.70-1-ff-130.0.1-edge-129.0.2792.52-1, but when I run it in CI it complains that firefox is not installed?

You can also use a custom browser: https://on.cypress.io/customize-browsers
Available browsers found on your system are:
 - chrome
 - edge
 - electron

I'm not sure what happened along the way to make firefox no longer available in CI.

Sorry if this is my error, but I'm confused how could I remove an entire browser by simply installing docker and running npm install -g typescript ts-node in said image.

@pkly

Possibly you are running into the problem Firefox not found.

Which CI provider are you using?
How are you running your Docker image?

BTW typescript is already globally installed (although that is not documented)

$ docker run -it --rm --entrypoint bash cypress/included:13.15.0 -c "npm ls -g"
/usr/local/lib
+-- corepack@0.29.3
+-- cypress@13.15.0
+-- npm@10.8.2
`-- typescript@5.6.2

@MikeMcC399 gitlab ci's on a custom runner, it's just a docker image built on top of the original one with some stuff added, I didn't explicitly specify a user for the runner though, so that could be it maybe

@pkly

If you add

firefox --version

into your GitLab workflow, you should get a usable error message if you are running the container under the (default) root user.

This is the message for GitHub. Check which uid is mentioned by GitLab and run your container under that uid

Run firefox --version
  firefox --version
  shell: sh -e {0}
Running Firefox as root in a regular user's session is not supported.  ($HOME is /github/home which is owned by uid 1001.)
Error: Process completed with exit code 1.

https://github.com/cypress-io/cypress/issues/27121 is an open issue to improve the Cypress error message in this situation.

@MikeMcC399

Thanks for the suggestion, after running the command in gitlab-ci and changing the user to the one spewed out it seems like the browser wants to launch, but gets confused as to where it's HOME is, since now I get Fontconfig error: No writable cache directories, which did not happen before setting user, and chrome launched fine, now it doesn't.

For the time being I'll probably drop firefox testing and maybe look into fixing it by just bruteforcing firefox to be able to be ran as a root user, since that creates less issues overall, and it's only in an image so it basically doesn't matter.

Additionally I kind of have to run cypress as root in that scenario because of some cursed things we have to do to prepare our full test suite. At this time I need docker to run commands, though I might look into allowing things to run through a http api instead, which would eliminate the need to even modify the image in the first place.

@pkly

Thanks for your progress report and I'm sorry to hear you weren't able to get this to work so far.

Could you please change the title of your post, for example "Firefox doesn't run on GitLab"? The current title is misleading for anybody with the same issue.

@pkly

There is a simple working example of running Firefox in GitLab CI on the Cypress documentation site using a Cypress Docker image cypress/browsers:

Testing with Cypress Docker Images as .gitlab-ci.yml

stages:
  - test

test:
  image: cypress/browsers:node-20.14.0-chrome-126.0.6478.114-1-ff-127.0.1-edge-126.0.2592.61-1
  stage: test
  script:
    # install dependencies
    - npm ci
    # start the server in the background
    - npm start &
    # run Cypress tests
    - npx cypress run --browser firefox

Thanks, but as mentioned already my setup requires me to start about 20 different services and then have access to docker to run some commands on them (from inside cypress), because of which cypress itself must contain docker (as a separate image), and it needs access to docker (so root), so that example does nothing for me.

@pkly

We would need a reproducible failing example in order to investigate this issue further. I successfully ran the following simple pipeline on GitLab, testing against Firefox. This runs by default as root (uid=0(root) gid=0(root) groups=0(root)):

stages:
  - test

test:
  image: 
    name: cypress/included:13.15.0
    entrypoint: [""]
  stage: test
  script:
    - npx cypress verify
    - npx cypress cache list
    - id
    - firefox --version
    - npx cypress run --browser firefox

$ id
uid=0(root) gid=0(root) groups=0(root)
$ firefox --version
Mozilla Firefox 130.0.1
$ npx cypress run --browser firefox
DevTools listening on ws://127.0.0.1:33287/devtools/browser/cd3af526-128f-4ca5-96b9-69eee9429ba2
====================================================================================================
  (Run Starting)
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress:        13.15.0                                                                        │
  │ Browser:        Firefox 130 (headless)                                                         │
  │ Node Version:   v20.17.0 (/usr/local/bin/node)                                                 │
  │ Specs:          1 found (spec.cy.js)                                                           │
  │ Searched:       cypress/e2e/**/*.cy.{js,jsx,ts,tsx}                                            │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘

I'll drop in the full docker file tomorrow

FROM cypress/included:cypress-13.15.0-node-20.17.0-chrome-129.0.6668.70-1-ff-130.0.1-edge-129.0.2792.52-1

RUN apt-get update
RUN apt-get install -y ca-certificates curl gnupg lsb-release
RUN curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

RUN  echo \
      "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian \
      $(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null

RUN apt-get update
RUN apt-get install -y docker-ce docker-ce-cli containerd.io

RUN npm install -g typescript@5.6 ts-node@10.9.2

That's it really. And yes, I need docker access inside of cypress (for exec), I know it's not great but for the time being it's required.

Additionally my entrypoint in gitlab is /usr/bin/env (unsure why exactly tbh, it just works?)

@pkly

Additionally my entrypoint in gitlab is /usr/bin/env (unsure why exactly tbh, it just works?)

Would you like to try using entrypoint: [""] instead?

I have opened a separate issue https://github.com/cypress-io/cypress-docker-images/issues/1222 to cover the missing documentation.

I built a Docker image using your Docker file and successfully ran it against Firefox locally using the following:

docker build . -t firefox-test
docker run -it --rm -v .:/app -w /app --entrypoint cypress firefox-test run --browser firefox

So this did not point to any issue with the image itself.

Note that cypress/base and cypress/browsers are the preferred images for running in CI. The intention for cypress/included was for it mainly to be used as a convenient locally used image. See Cypress Docker variants.

I could use cypress/browsers, since I'd like to just have them installed but from my understanding it seems I'd need to install cypress in it? I'm not testing a node app, it's all php, so I'd need to install it in the script or have a previous step which kinda defeats the point.

I updated the entrypoint to [""], and, well, it seems something changed? Here's output with entrypoint set to ""

npx cypress run --browser $BROWSER
DevTools listening on ws://127.0.0.1:39015/devtools/browser/0945ba54-c771-4cd2-bb5b-5f7f20488e99
[230:1002/113816.609299:ERROR:egl_util.cc(44)] Failed to load GLES library: /root/.cache/Cypress/13.15.0/Cypress/libGLESv2.so: /root/.cache/Cypress/13.15.0/Cypress/libGLESv2.so: cannot open shared object file: Permission denied
[230:1002/113816.631309:ERROR:viz_main_impl.cc(196)] Exiting GPU process due to errors during initialization

I'm so confused. It seems like now it works but can't find something, but I have an idea why that might be the case. To allow some caching I also set some volumes in the runner itself, along with /root. Welp. I'll try to debug it later on my side and return with information.

I wish I could read, I would spare both of us some time.

@pkly

It appears that there is something wrong in the way that you are running the image if you are getting "Permission denied"

[230:1002/113816.609299:ERROR:egl_util.cc(44)] Failed to load GLES library: /root/.cache/Cypress/13.15.0/Cypress/libGLESv2.so: /root/.cache/Cypress/13.15.0/Cypress/libGLESv2.so: cannot open shared object file: Permission denied

The /root directory in the image is set up with lax permissions.

root@86b95038a0ca:/# ls -al /root/.cache/Cypress/13.15.0/Cypress
total 344748
drwxrwxrwx 4 root root      4096 Sep 25 18:09 .
drwxrwxrwx 3 root root      4096 Sep 25 20:12 ..
-rwxrwxrwx 1 root root 173434608 Sep 25 18:09 Cypress
-rwxrwxrwx 1 root root      1096 Sep 25 18:04 LICENSE.electron.txt
-rwxrwxrwx 1 root root   9242930 Sep 25 18:04 LICENSES.chromium.html
-rwxrwxrwx 1 root root 129957239 Sep 25 18:10 browser_v8_context_snapshot.bin
-rwxrwxrwx 1 root root     54096 Sep 25 18:04 chrome-sandbox
-rwxrwxrwx 1 root root    136037 Sep 25 18:04 chrome_100_percent.pak
-rwxrwxrwx 1 root root    196924 Sep 25 18:04 chrome_200_percent.pak
-rwxrwxrwx 1 root root   1322280 Sep 25 18:04 chrome_crashpad_handler
-rwxrwxrwx 1 root root  10717392 Sep 25 18:04 icudtl.dat
-rwxrwxrwx 1 root root    252120 Sep 25 18:04 libEGL.so
-rwxrwxrwx 1 root root   6615784 Sep 25 18:04 libGLESv2.so
-rwxrwxrwx 1 root root   2882824 Sep 25 18:04 libffmpeg.so
-rwxrwxrwx 1 root root   4292368 Sep 25 18:04 libvk_swiftshader.so
-rwxrwxrwx 1 root root   7469008 Sep 25 18:04 libvulkan.so.1
drwxrwxrwx 2 root root      4096 Sep 25 18:04 locales
drwxrwxrwx 3 root root      4096 Sep 25 18:04 resources
-rwxrwxrwx 1 root root   5499605 Sep 25 18:04 resources.pak
-rwxrwxrwx 1 root root    267462 Sep 25 18:04 snapshot_blob.bin
-rwxrwxrwx 1 root root    626313 Sep 25 18:04 v8_context_snapshot.bin
-rwxrwxrwx 1 root root       107 Sep 25 18:04 vk_swiftshader_icd.json

@pkly

I could use cypress/browsers, since I'd like to just have them installed but from my understanding it seems I'd need to install cypress in it? I'm not testing a node app, it's all php, so I'd need to install it in the script or have a previous step which kinda defeats the point.

I don't know quite what you mean here. https://github.com/cypress-io/cypress-docker-images/issues/1220#issuecomment-2385551990 gave an example of how to use cypress/browsers. You would generally have a repo with cypress included in package.json and package-lock.json (for npm). When you execute npm ci it would install dependencies including cypress from the lock file.

@MikeMcC399

Not quite. The reason was that for caching purposes (composer/npm) I long ago set up a volume in the runner we use to point to the home directory of the CI server instead of the image. So /root in the image would be my /home/gitlab-runner. So any files in the images would be lost.

I've updated that mount point to no longer point to /root, instead mount only .ssh and other such folders. Firefox now starts.

Regardless of the image I'd use I'd need to add docker on top of it. This way I can install the dependencies we use in the image itself instead of installing them every time the job runs. And yeah, sure, you can cache them, but it's still slower. It doesn't really matter that much though.

In hindsight it was silly to not look more in the configuration of the runner itself, thank you for the help.

What I find confusing is that chromium based browsers don't even care that /root is mounted over. Like nothing happened.

Maybe add a note to the firefox section suggesting that the /root path is mounted over? Anyway, again thanks a lot.

@pkly

What I find confusing is that chromium based browsers don't even care that /root is mounted over. Like nothing happened.

Maybe add a note to the firefox section suggesting that the /root path is mounted over? Anyway, again thanks a lot.

You're welcome!

I don't believe there is any specific warning in the documentation about the /root directory. That could be added, however that would be a general warning, not one specific to Firefox.

cypress-io / cypress-docker-images

Firefox doesn't run on GitLab (as root) #1220