[🐛 Bug]: standalone-chromium fails to start on Mac M1

[craigmcnamara]

What happened?

Container fails to start when novnc exits during boot.

Command used to start Selenium Grid with Docker

version: '3.7'
services:
  app:
    image: monami-app
    build:
      context: .
      args:
        - GRAPHQL_PRO_KEY=${GRAPHQL_PRO_KEY}
    command: bundle exec rails server -p 3000 -b '0.0.0.0'
    volumes:
      - .:/app
      - gems:/gems
      - node_modules:/app/node_modules
    ports:
      - '80:3000'
      - '43447:43447'
    depends_on:
      - postgres
      - redis
    entrypoint: /app/dev-entrypoint
    tty: true
    stdin_open: true
    environment:
      - WEBPACKER_DEV_SERVER_HOST=webpacker
      - SELENIUM_REMOTE_HOST=selenium
  webpacker:
    build:
      context: .
      args:
        - GRAPHQL_PRO_KEY=${GRAPHQL_PRO_KEY}
        - DISABLE_SPRING=true
    command: ./bin/webpack-dev-server
    volumes:
      - .:/app
      - gems:/gems
      - node_modules:/app/node_modules
    ports:
      - '3035:3035'
    entrypoint: /app/docker/webpack-entrypoint
  postgres:
    image: postgres:12-alpine
    volumes:
      - postgres-data:/var/lib/postgresql/data
    environment:
      - POSTGRES_HOST_AUTH_METHOD=trust
  redis:
    image: redis:6-alpine
    command: redis-server
    volumes:
      - redis:/data
  sidekiq:
    build:
      context: .
      args:
        - GRAPHQL_PRO_KEY=${GRAPHQL_PRO_KEY}
    command: bundle exec sidekiq
    depends_on:
      - postgres
      - redis
    volumes:
      - .:/app
      - gems:/gems
      - node_modules:/app/node_modules
    entrypoint: docker/sidekiq-entrypoint.sh
    tty: true
    stdin_open: true
    environment:
      - WEBPACKER_DEV_SERVER_HOST=webpacker
      - DISABLE_SPRING=true
  selenium:
    image: seleniarm/standalone-chromium:latest
    ports:
      - '4444:4444'
    shm_size: '256mb'
  mailcatcher:
    build:
      context: ./docker/mailcatcher
    command: mailcatcher -f --http-ip 0.0.0.0 --smtp-ip 0.0.0.0
    ports:
      - "1080:1080"

volumes:
  postgres-data:
  gems:
  redis:
  node_modules:

Relevant log output

monami-selenium-1  | 2021-12-07 23:50:53,540 INFO Included extra file "/etc/supervisor/conf.d/selenium.conf" during parsing
monami-selenium-1  | 2021-12-07 23:50:53,541 INFO supervisord started with pid 9
monami-selenium-1  | 2021-12-07 23:50:54,555 INFO spawned: 'xvfb' with pid 10
monami-selenium-1  | 2021-12-07 23:50:54,565 INFO spawned: 'vnc' with pid 11
monami-selenium-1  | 2021-12-07 23:50:54,569 INFO spawned: 'novnc' with pid 12
monami-selenium-1  | 2021-12-07 23:50:54,572 INFO spawned: 'selenium-standalone' with pid 13
monami-selenium-1  | Setting up SE_NODE_GRID_URL...
monami-selenium-1  | 2021-12-07 23:50:54,584 INFO success: xvfb entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
monami-selenium-1  | 2021-12-07 23:50:54,584 INFO success: vnc entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
monami-selenium-1  | 2021-12-07 23:50:54,584 INFO success: novnc entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
monami-selenium-1  | 2021-12-07 23:50:54,584 INFO success: selenium-standalone entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
monami-selenium-1  | /opt/bin/generate_config: line 28: google-chrome: command not found
monami-selenium-1  | Selenium Grid Standalone configuration:
monami-selenium-1  | [network]
monami-selenium-1  | relax-checks = true
monami-selenium-1  |
monami-selenium-1  | [node]
monami-selenium-1  | session-timeout = "300"
monami-selenium-1  | override-max-sessions = false
monami-selenium-1  | detect-drivers = false
monami-selenium-1  | max-sessions = 1
monami-selenium-1  |
monami-selenium-1  | [[node.driver-configuration]]
monami-selenium-1  | display-name = "chrome"
monami-selenium-1  | stereotype = '{"browserName": "chrome", "browserVersion": ".", "platformName": "Linux"}'
monami-selenium-1  | max-sessions = 1
monami-selenium-1  |
monami-selenium-1  | Starting Selenium Grid Standalone...
monami-selenium-1  | 23:50:54.829 INFO [LoggingOptions.configureLogEncoding] - Using the system default encoding
monami-selenium-1  | 23:50:54.831 INFO [OpenTelemetryTracer.createTracer] - Using OpenTelemetry for tracing
monami-selenium-1  | 23:50:55.112 INFO [NodeOptions.getSessionFactories] - Detected 7 available processors
monami-selenium-1  | 23:50:55.131 INFO [NodeOptions.report] - Adding chrome for {"browserVersion": ".","browserName": "chrome","platformName": "Linux","se:vncEnabled": true} 1 times
monami-selenium-1  | 23:50:55.137 INFO [Node.<init>] - Binding additional locator mechanisms: id, name, relative
monami-selenium-1  | 23:50:55.147 INFO [LocalDistributor.add] - Added node c0d38425-03b1-44d7-90dd-28861d254b45 at http://172.23.0.2:4444. Health check every 120s
monami-selenium-1  | 23:50:55.147 INFO [GridModel.setAvailability] - Switching node c0d38425-03b1-44d7-90dd-28861d254b45 (uri: http://172.23.0.2:4444) from DOWN to UP
monami-selenium-1  | 23:50:55.211 INFO [Standalone.execute] - Started Selenium Standalone 4.0.0 (revision 3a21814679): http://172.23.0.2:4444
monami-selenium-1  | 2021-12-07 23:50:55,959 INFO exited: novnc (exit status 1; not expected)

Operating System

macOS Monterey

Docker Selenium version (tag)

4.0.0-20211111

[craigmcnamara]

I think this is failing because of this line https://github.com/seleniarm/docker-selenium/blob/dab951c64b97c6c023826f0f2596da4b8ff09eaf/NodeChromium/Dockerfile#L32

generate_config seems to use browser_name to decide how to get a version and fails because the Chromium container doesn't have Chrome binaries.

[viktorianer]

I have the log entry /opt/bin/generate_config: line 47: google-chrome: command not found in my logs with seleniarm/node-chromium:4.0.0-20211111, but do not have this exit status 1; not expected.

Instead of this, I see:

...
[1639062301.953][SEVERE]: bind() failed: Cannot assign requested address (99)
16:05:02.113 INFO [ProtocolHandshake.createSession] - Detected dialect: W3C
16:05:43.255 WARN [SeleniumSpanExporter$1.lambda$export$0] - {
  "traceId": "a4880b1106bd5d7c8efd3f579d63fddf","eventTime": 1639062343254771209,
  "eventName": "HTTP request execution complete","attributes": {
    "http.flavor": 1,"http.handler_class": "org.openqa.selenium.remote.http.Route$PredicatedRoute",
    "http.host": "selenium_hub:4444","http.method": "POST","http.request_content_length": "2",
    "http.scheme": "HTTP","http.status_code": 404,
    "http.target": "\u002fsession\u002fa4303ca4fdcfdebbdd656cf51f742276\u002felement\u002f33c56cab-01bc-4580-8de8-7c52a2deed61\u002fclick",
    "http.user_agent": "selenium\u002f4.1.0 (ruby linux)"
  }
}
...

And some of my tests (out of ~2k) will fail with (It is probably not related, I am not sure.):

Selenium::WebDriver::Error::WebDriverError: You are trying to work with something that isn't a file.

And I have no idea how to fix it. Because the test and views have not even a file upload -__-.

The failed tests are doing something like:

visit("/resources/new")
fill_in("name", with: "Test")

We use Angular and the screenshots showing us correct rendered view, with cursor inside this name text field. The error only appears on Macs with M1 chip and with any seleniarm/node-chromium:4.x image.

With older seleniarm/node-chromium:4.0.0-beta-1-20210215 images, I get the same error, but do not see log entry like /opt/bin/generate_config: line 47: google-chrome: command not found

[jamesmortensen]

@craigmcnamara if you try to run seleniarm/standalone-chromium without docker-compose, are you able to interact with the container?

$ docker run --rm -it -p 4444:4444 -p 5900:5900 -p 7900:7900 --shm-size 3g seleniarm/standalone-chromium:20211110

You can view it via noVNC here http://localhost:7900 and interact with the Selenium Console at http://localhost:4444/ui/index.html.

Also, try increasing the shared memory from 256mb to 3gb for the selenium container to avoid starving the resources:

selenium:
    image: seleniarm/standalone-chromium:latest
    ports:
      - '4444:4444'
    shm_size: '3gb'

The google-chrome error may be largely cosmetic, as that line is just attempting to get the browser version. I'll fix it anyway so the logs look cleaner.

/opt/bin/generate_config: line 47: google-chrome: command not found

Additionally, the novnc: exit status: 1 is just for the novnc component. It's functioning though, since we can see the desktop via http://localhost:7900. I'll see if I can fix whatever is causing that too, since it's again distracting.

[jamesmortensen]

I got rid of the google-chrome binary issue: https://github.com/seleniarm/docker-selenium/commit/6629e6f2e734385cfbd8d618c01f231a5e97e608.

[viktorianer]

I got rid of the google-chrome binary issue: 6629e6f.

Hey @jamesmortensen , could you please release an arm46 version too? For now, we can not use this image in our teams, because of this. Thank you in advance.

[jamesmortensen]

@viktorianer what kind of hardware are you running? I assume you meant ARM64. If so, the Chromium and Firefox images are ARM64 images. Are you pulling from here? https://hub.docker.com/r/seleniarm/standalone-chromium

[viktorianer]

I am running on M1, as mentioned above earlier. But we would like to use it on Linux/Windows too.

I can not use it in a project with docker-compose.yml, because I can only refer to one source of image. And this new image (4.0.0-20211213) is only available for arm64, not for amd64. The older image (4.0.0-beta-1-20210215) was available for both arch types.

It would be great to have the new image for both arch types too. 🥇 Would it be possible?

[jamesmortensen]

I get what you're saying @viktorianer it should be possible since debian:latest is also multi-arch. There are a couple places I can think of that have a dependency on architecture, and these would need to be addressed, possibly by using uname -m to derive the architecture type at build time.

Dependencies on ARCH for doing a find/replace in a Java conf file in Base Docker container:

https://github.com/seleniarm/docker-selenium/blob/dab951c64b97c6c023826f0f2596da4b8ff09eaf/Base/Dockerfile#L16 https://github.com/seleniarm/docker-selenium/blob/dab951c64b97c6c023826f0f2596da4b8ff09eaf/Base/Dockerfile#L36

Dependencies on ARCH for building geckodriver from source in NodeFirefox:

https://github.com/seleniarm/docker-selenium/blob/dab951c64b97c6c023826f0f2596da4b8ff09eaf/NodeFirefox/build-step-1.sh#L8 https://github.com/seleniarm/docker-selenium/blob/dab951c64b97c6c023826f0f2596da4b8ff09eaf/NodeFirefox/build-step-2.sh#L32 https://github.com/seleniarm/docker-selenium/blob/dab951c64b97c6c023826f0f2596da4b8ff09eaf/NodeFirefox/build-step-2.sh#L40

Chromium may be easier than Firefox

Firefox has more dependencies on ARCH. If you wanted to get started sooner with Selenium/Standalone-Chromium, it has less dependencies on ARCH, so as long as the Java 11 dependency can be made multi-arch, you should be able to build a multi-arch Chromium image:

docker buildx build --platform linux/amd64,linux/arm64 -t local-seleniarm/standalone-chromium:latest .

I'll be able to look at this more over the weekend. In the meantime, hope this is helpful!

[jamesmortensen]

@viktorianer I also had success using this script to allow collaboration between team members:

#!/usr/bin/env node
const { spawn } = require('child_process');

function getWdioPortArgument(argsArr) {
    return argsValue(argsArr, '-p', '--port', '4444');
}

function getVncPortArgument(argsArr) {
    return argsValue(argsArr, '-n', '--vnc', '5900');
}

function getNoVncPortArgument(argsArr) {
    return argsValue(argsArr, '-b', '--novnc', '7900');
}

function getDockerImageArgument(argsArr) {
    const os = require('os');
    return argsValue(argsArr, '-i', '--image',
        !os.cpus()[0].model.includes('Apple M1')
            ? 'selenium/standalone-chrome-debug'
            : 'local-seleniarm/standalone-chromium:latest');
}

function getHelpArgument(argsArr) {
    return argsArr.includes('-h') || argsArr.includes('--help');
}

function argsValue(argsArr, shortForm, longForm, defaultValue) {
    return argsArr.reduce((acc, elem, index, array) => {
        if ((array[index - 1] === shortForm || array[index - 1] === longForm))
            acc = elem;
        return acc;
    }, defaultValue);
}

if (getHelpArgument(process.argv)) {
    console.log(`
Usage: 
    ./start-selenium-docker.js [[-h|--help] | [-i|--image IMAGE_REF] | [-p|--port WD_PORT] | [-n|--vnc VNC_PORT] | [-b|--novnc NOVNC_PORT]]

    -h -> Help (this output)
    -i -> Docker Image and Tag (default selenium/standalone-chrome-debug or local-seleniarm/standalone-chromium depending on architecture/CPU type)
    -p -> Webdriver/Selenium port (default 4444)
    -n -> VNC Port (default 5900)
    -b -> NoVNC Port (default 7900)

    `);
    process.exit(0);
}

const SELENIUM_PORT = getWdioPortArgument(process.argv);
const VNC_PORT = getVncPortArgument(process.argv);
const NOVNC_PORT = getNoVncPortArgument(process.argv);
const IMAGE = getDockerImageArgument(process.argv);

const cmd = `docker run --rm -v ${process.cwd()}:${process.cwd()}:ro -p ${SELENIUM_PORT}:4444 -p ${VNC_PORT}:5900 -p ${NOVNC_PORT}:7900 --shm-size 3g ${IMAGE}`;

console.log(cmd);

module.exports = function () {
    return new Promise((resolve, reject) => {
        const process = spawn(cmd.split(' ')[0], cmd.split(' ').filter((arg, index) => { if (index != 0) return arg; }), { stdio: 'inherit' });

        process.on('close', (code) => {
            if (code !== 0)
                reject(code);
            else
                resolve(code);
        });
    });
}

if (process.argv[1] === __filename)
    module.exports(cmd);

node start-selenium-docker will look at your computer's CPU type to determine whether to use the seleniarm images or the official selenium images. You might just need to change the two image names in the getDockerImageArgument methods to reflect the new names. It's different than a multi-arch setup and in some ways has advantages because all of the Intel/AMD64 users still get to use the official images while the aaarch64/M1 people use the experimental stuff. Hope this helps.

[craigmcnamara]

@jamesmortensen I just pulled the latest tag and everything is working great. Thanks for the hard work!

[viktorianer]

@craigmcnamara @jamesmortensen Thanks for your help. But I do not think just about my project(s) in this case.

In general, I would love to have this changes at https://github.com/SeleniumHQ/docker-selenium/issues/1076. But, I do not know how to help here.

I can see the images from https://github.com/sj26/docker-selenium, they are build for arm and amd. https://hub.docker.com/layers/seleniarm/node-chromium/4.0.0-beta-1-20210215/images/sha256-148f4749119f950d33229f0ec37eb772a55de97491b462830bffc46981d39869?context=explore

And these images we can use in our docker-compose.yml, without any hack. But they are not up-to-date. So, that why I am saying it would be better if these changes from here, would be included in https://github.com/SeleniumHQ/docker-selenium.

And thanks for this great work and effort! 👍🏼

[jamesmortensen]

@viktorianer I am sure it can be done, but this is a different issue than #1, the google-chrome issue. Can you open a new issue for the multi-arch build?

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.