search

NCIOCPL / cgov-digital-platform

The Cancer.gov Digital Communications Platform
GNU General Public License v2.0
11 stars 33 forks source link

Chrome is missing from Docker image #48

Closed bryanpizzillo closed 5 years ago

bryanpizzillo commented 5 years ago

This is required by beHat.

bkline commented 5 years ago

Possibly relevant to this issue:

bkline commented 5 years ago

Also:

bkline commented 5 years ago

It looks like the selenium/standalone-chrome docker image is installing X (even if most of it isn't used for headless mode). The way I determined this was to spin up the base image:

docker run -it selenium/node-base:3.5.3-boron /bin/bash

and then run the command in the selenium/standalone-chrome Dockerfile which installs chromium. I could see apt pulling in all the X-windows libraries. To confirm this I asked apt to install emacs and this what it said:

...
The following NEW packages will be installed:
  emacs emacs24 emacs24-bin-common emacs24-common emacs24-common-non-dfsg emacs24-el emacsen-common
  gconf-service gconf-service-backend gconf2-common ghostscript gsfonts imagemagick-common
  libasn1-8-heimdal libcupsfilters1 libcupsimage2 libfftw3-double3 libfribidi0 libgconf-2-4 libgif7
  libgomp1 libgpm2 libgs9 libgs9-common libgssapi3-heimdal libhcrypto4-heimdal libheimbase1-heimdal
  libheimntlm0-heimdal libhx509-5-heimdal libijs-0.35 libjbig2dec0 libkrb5-26-heimdal libldap-2.4-2
  liblockfile-bin liblockfile1 liblqr-1-0 libm17n-0 libmagickcore-6.q16-2 libmagickwand-6.q16-2 libotf0
  libpaper-utils libpaper1 libroken18-heimdal libsasl2-2 libsasl2-modules libsasl2-modules-db
  libwind0-heimdal m17n-db poppler-data
The following packages will be upgraded:
  libcups2
1 upgraded, 49 newly installed, 0 to remove and 65 not upgraded.
Need to get 54.2 MB of archives.
After this operation, 173 MB of additional disk space will be used.
Do you want to continue? [Y/n]

By comparison, here's what apt said when I did the same thing in our own container:

...
The following NEW packages will be installed:
  adwaita-icon-theme at-spi2-core dbus dconf-gsettings-backend dconf-service emacs emacs25
  emacs25-bin-common emacs25-common emacs25-el emacsen-common fontconfig fonts-droid-fallback
  fonts-noto-mono ghostscript glib-networking glib-networking-common glib-networking-services
  gsettings-desktop-schemas gsfonts gtk-update-icon-cache hicolor-icon-theme humanity-icon-theme
  imagemagick-6-common install-info libasound2 libasound2-data libatk-bridge2.0-0 libatk1.0-0
  libatk1.0-data libatspi2.0-0 libavahi-client3 libavahi-common-data libavahi-common3 libcairo-gobject2
  libcairo2 libcolord2 libcroco3 libcups2 libcupsfilters1 libcupsimage2 libdatrie1 libdbus-1-3 libdconf1
  libdrm-common libdrm2 libegl-mesa0 libegl1 libepoxy0 libfftw3-double3 libfribidi0 libgbm1
  libgdk-pixbuf2.0-0 libgdk-pixbuf2.0-bin libgdk-pixbuf2.0-common libgif7 libglapi-mesa libglib2.0-0
  libglib2.0-data libglvnd0 libgomp1 libgraphite2-3 libgs9 libgs9-common libgtk-3-0 libgtk-3-bin
  libgtk-3-common libharfbuzz0b libice6 libidn11 libijs-0.35 libjbig2dec0 libjson-glib-1.0-0
  libjson-glib-1.0-common liblcms2-2 liblockfile-bin liblockfile1 liblqr-1-0 libltdl7 libm17n-0
  libmagickcore-6.q16-3 libmagickwand-6.q16-3 libotf0 libpango-1.0-0 libpangocairo-1.0-0
  libpangoft2-1.0-0 libpaper-utils libpaper1 libpixman-1-0 libproxy1v5 librest-0.7-0 librsvg2-2
  librsvg2-common libsm6 libsoup-gnome2.4-1 libsoup2.4-1 libthai-data libthai0 libwayland-client0
  libwayland-cursor0 libwayland-egl1-mesa libwayland-server0 libx11-xcb1 libxcb-dri2-0 libxcb-dri3-0
  libxcb-present0 libxcb-render0 libxcb-shm0 libxcb-sync1 libxcb-xfixes0 libxcomposite1 libxcursor1
  libxdamage1 libxfixes3 libxft2 libxi6 libxinerama1 libxkbcommon0 libxrandr2 libxrender1 libxshmfence1
  libxt6 libxtst6 m17n-db poppler-data shared-mime-info ubuntu-mono x11-common xdg-user-dirs xkb-data
0 upgraded, 130 newly installed, 0 to remove and 5 not upgraded.
Need to get 65.7 MB of archives.
After this operation, 247 MB of additional disk space will be used.
Do you want to continue? [Y/n]

As you can see, a lot more stuff (for X) is missing from our container. Not the end of the world, but the idea that we can just drop the binary for chromium into the container and run it doesn't look like it's going to work. Even if we run chrome with the headless option, it's still going to require that the runtime libraries against which it's built be present. So I think we'll just have to bite the bullet and accept the additional 75MB needed for the X libraries.

bkline commented 5 years ago

I have been able to install google-chrome-stable and chromedriver. In order for the behat tests to run we will need to patch the code for tests:behat:run, which fails without adding --no-sandbox to the google-chrome command. Google has some instructions which say

Note: --no-sandbox is not needed if you properly setup a user in the container.

... linking to a Dockerfile illustrating what needs to be done:

# Add a chrome user and setup home dir.
RUN groupadd --system chrome && \
    useradd --system --create-home --gid chrome --groups audio,video chrome && \
    mkdir --parents /home/chrome/reports && \
    chown --recursive chrome:chrome /home/chrome

However, I tried this and even after su - chrome the tests still timed out. The following patch to vendor/acquia/blt/src/Robo/Commands/Tests/BehatCommand.php is necessary in order to make the tests work.

--- BehatCommand.php-original   2018-11-25 12:19:47.000000000 -0500
+++ BehatCommand.php    2018-11-25 12:18:56.000000000 -0500
@@ -175,7 +175,7 @@
     $this->logger->info("Launching headless chrome...");
     $this->getContainer()
       ->get('executor')
-      ->execute("'$chrome_bin' --headless --disable-web-security --remote-debugging-port={$this->chromePort} {$this->chromeArgs} $chrome_host")
+      ->execute("'$chrome_bin' --no-sandbox --headless --disable-web-security --remote-debugging-port={$this->chromePort} {$this->chromeArgs} $chrome_host")
       ->background(TRUE)
       ->printOutput(TRUE)
       ->printMetadata(TRUE)

Let's talk about this patch, and if we agree that we need to apply it, discuss the best ways of doing that (probably a sed command installed as a post-install-command in composer.json), along the lines of

sed -i -e "s/'\$chrome_bin' --headless/'\$chrome_bin' --no-sandbox --headless/" vendor/acquia/blt/src/Robo/Commands/Tests/BehatCommand.php

See also https://blt.readthedocs.io/en/latest/alternative-environment-tips/lando/

bryanpizzillo commented 5 years ago

Let's wait until we get access to our TAM and see about the implications of this. Most of those issues were from BLT 8.x and we are on 9.2.1, so I don't know if thier environment still has the issues. Our Docker may still have it.

bryanpizzillo commented 5 years ago

@bkline Knows this, but to be official, Acquia TAMs stated that there would be no issue in patching BLT. I think the next steps are:

bkline commented 5 years ago

@bryanpizzillo: I'm working on the next steps, as outlined in the previous comment. I hadn't read that comment closely enough until today, so didn't realize last week that my simple approach of using sed wasn't going to fly. So many tools to learn! 😄

So I followed the link given in that comment and read up on how to set up things for Cameron Eagon's composer-patches plugin. I created the patch (wasn't able to find instructions about which directory patch would be run from, so assumed the root of our repo) and installed it in ./patch/blt:

--- vendor/acquia/blt/src/Robo/Commands/Tests/BehatCommand.php-dist 2018-12-28 14:59:44.000000000 -0500
+++ vendor/acquia/blt/src/Robo/Commands/Tests/BehatCommand.php  2018-12-28 15:00:37.000000000 -0500
@@ -175,7 +175,7 @@
     $this->logger->info("Launching headless chrome...");
     $this->getContainer()
       ->get('executor')
-      ->execute("'$chrome_bin' --headless --disable-web-security --remote-debugging-port={$this->chromePort} {$this->chromeArgs} $chrome_host")
+      ->execute("'$chrome_bin' --no-sandbox --headless --disable-web-security --remote-debugging-port={$this->chromePort} {$this->chromeArgs} $chrome_host")
       ->background(TRUE)
       ->printOutput(TRUE)
       ->printMetadata(TRUE)

Then I made the necessary addition to composer.json as instructed, so the patches section now looks like this:

        "patches": {
            "drupal/core": {
                "2599228 : Programmatically created translatable content type returns SQL error on content creation": "https://www.drupal.org/files/issues/2018-11-29/2599228-104.patch"
            },
            "acquia/blt": {
                "Fix behat debugging with chrome": "./patches/blt/issue-48-fix-behat-chrome-invocation.patch"
            }
        },

Then on to the composer update command:

$ composer update acquia/blt
Could not scan for classes inside "/Users/bkline/Projects/repos/cgov-digital-platform/vendor/drupal/core/lib/Drupal.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/Users/bkline/Projects/repos/cgov-digital-platform/vendor/drupal/core/lib/Drupal/Component/Utility/Timer.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/Users/bkline/Projects/repos/cgov-digital-platform/vendor/drupal/core/lib/Drupal/Component/Utility/Unicode.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/Users/bkline/Projects/repos/cgov-digital-platform/vendor/drupal/core/lib/Drupal/Core/Database/Database.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/Users/bkline/Projects/repos/cgov-digital-platform/vendor/drupal/core/lib/Drupal/Core/DrupalKernel.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/Users/bkline/Projects/repos/cgov-digital-platform/vendor/drupal/core/lib/Drupal/Core/DrupalKernelInterface.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/Users/bkline/Projects/repos/cgov-digital-platform/vendor/drupal/core/lib/Drupal/Core/Site/Settings.php" which does not appear to be a file nor a folder
Gathering patches for root package.
> DrupalProject\composer\ScriptHandler::checkComposerVersion
Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 0 installs, 1 update, 0 removals
Gathering patches for root package.
Gathering patches for dependencies. This might take a minute.
  - Updating acquia/blt (9.2.1 => 9.2.3): Downloading (100%)
Writing lock file
Generating autoload files
Updating BLT templated files...
 > blt blt:update --ansi -y
Copying files from BLT's template into your project...
Merging BLT's blt.yml template with your project's blt/blt.yml...
Removing deprecated files and directories...
> blt:init:shell-alias
[warning] Could not find your CLI configuration file.
[warning] Looked in ~/.zsh, ~/.bash_profile, ~/.bashrc, ~/.profile, and ~/.functions.
[Filesystem\FilesystemStack] touch ["/Users/bkline/.bash_profile"]
PHP CodeSniffer Config installed_paths set to ../../drupal/coder/coder_sniffer/
> DrupalProject\composer\ScriptHandler::createRequiredFiles

Well, that was unexpected. 😞 Not only was my patch ignored, but I had no idea it was going to try to do all that other stuff. Did I mis-read the instructions? Are you sure you don't want me to use the approach I proposed above (a sed command installed as a post-install-command in composer.json)?