Open KludgeKML opened 2 years ago
MySQL versions below 8 are not available in arm64 architectures
The good news here is that none of the GOV.UK apps are dependent on MySQL < 8.
There's a ticket on the database migration board to remove the MySQL 5.6 service from the docker compose configuration. Right now it's still defined but unused/redundant.
Thanks for opening the issue here and sorry that this issue has caught you at the time of you onboarding. I've put in a request with IT for an M1 machine myself to see if I can help (or at least experience the pain)
Additional:
inotify
does not run under QEMU and will thus struggle in docker engine when running in AMD64 compatibility mode. I am unaware if there is an ARM build for it however if there is, our docker installation does not grab the relevant gems for the platform. It appears that Docker might use QEMU to run ARM on M1 chips anyway currently, so we may need to find a replacement.govuk_test
gem fails to run correctly due to being unable to call chrome driver via the webdrivers
gem. We might be able to get ahead of this by updating the webdrivers gem value.rabbitmq
is best run on ARM64 mode rather than through the AMD64 QEMU due to known instability when emulated.publishing-api
. This is a regression from current behaviour.*.dev.gov.uk
endpoints do not route correctly and cannot find the containers. Curls are intermittent, which makes me think it's a pipeline issue?Seems a lot of this we can fix by upgrading to the latest version of the gems we rely on, however some of it may be bespoke for our environment.
Per @Tetrino 's suggestion I've made a branch that we can use as a fake main for M1 changes:
https://github.com/alphagov/govuk-docker/tree/m1-investigation
If we make M1-related changes, we can branch from there and PR into there and we'll have a handy way of 1) getting the fixes we need and 2) comparing the changes with main.
https://github.com/alphagov/govuk-docker/pull/564 - PR for fixing the dnsmasq problem.
Note for related apps
Apart from the things required to get govuk-docker working at all, any app you want to run needs to have:
config.file_watcher = ActiveSupport::EventedFileUpdateChecker
in /config/environments/development.rb(The last two items remove spring and inotify autorestart in development mode, so they're not entirely ideal. Possibly there should be some boilerplate to allow them on intel systems).
govuk_test gem fails to run correctly due to being unable to call chrome driver via the webdrivers gem. We might be able to get ahead of this by updating the webdrivers gem value.
This looks like it's linked to inotify. If you bash into a running app's container and try to run google-chrome-stable, it fails with a coredump about inotify:
root@35558c032be5:/govuk/static# /usr/bin/google-chrome-stable --no-sandbox
qemu: uncaught target signal 5 (Trace/breakpoint trap) - core dumped
qemu: uncaught target signal 5 (Trace/breakpoint trap) - core dumped
[370:370:0131/164025.878444:ERROR:nacl_fork_delegate_linux.cc(329)] Bad NaCl helper startup ack (0 bytes)
qemu: uncaught target signal 5 (Trace/breakpoint trap) - core dumped
[334:379:0131/164025.994483:ERROR:file_path_watcher_linux.cc(321)] inotify_init() failed: Function not implemented (38)
[0131/164026.063886:ERROR:scoped_ptrace_attach.cc(27)] ptrace: Function not implemented (38)
qemu: uncaught target signal 5 (Trace/breakpoint trap) - core dumped
Are these problems when trying to run the tool through a compatibility tool (rosetta or similar) and an x86 architecture or are you compiling the images with an arm64 arch?
These are when I force amd64 builds for the app container (not necessarily on the other containers) by
# Install packages for building ruby
FROM --platform=linux/amd64 buildpack-deps
(So they're running through rosetta, I believe)
Without the --platform, google-chrome-stable won't install at all, with it it installs, but then won't run (because of inotify I think). The other option is remove the --platform tag and include chromium rather than google-chrome-stable, but the webdrivers used in gov_uk test seem not to be able to find nor use chromium.
I found that I am able to build using the following changes: https://github.com/alphagov/govuk-docker/pull/569
There's still errors for me to work through but build seems to "mostly" finish
@DilwoarH I think you can simplify that slightly by just editing Dockerfile.govuk-base and changing the line
# Install packages for building ruby
FROM buildpack-deps
...to....
# Install packages for building ruby
FROM --platform=linux/amd64 buildpack-deps
# Set M1 Workaround variables
ENV DISABLE_SPRING true
This forces all the app images to be built in amd64 (rather than specifying each one in its compose file), and turns off spring in any app that uses it (because spring relies on inotify, which doesn't work under qemu).
For any individual app you want to run you'll then need to bump it to ruby 2.7.5 (uncontroversial, because lots of them are being bumped to that anyway at the moment), and disable the listen gem (which uses inotify). I haven't come up with a nice way to disable listen at the base the way spring is disabled, sadly.
Another option is to use amd64/buildpack-deps
instead of calling FROM
at that point. We can also force docker to run in AMD64 mode as default platform using DOCKER_DEFAULT_PLATFORM=linux/amd64
however I agree the dockerfile is the better location.
PR to fix the chrome issues: https://github.com/alphagov/govuk-docker/pull/576
I found I didn't need to set anything up to build things for amd64 - it just worked like that by default?
Current state of affairs (mostly good):
Thanks to everyone's hard work, we're now at a stage where mostly everything works on M1 laptops. If you have one, you can proceed just as everyone else does, with the following exceptions:
# Tell the govuk_test gem not to try to load webdrivers
ENV GOVUK_TEST_USE_SYSTEM_CHROMEDRIVER yes
The default docker MySQL-8 app does not have an ARM64/v8 image. You can use an official MySQL image by replacing the mysql-8 image with mysql/mysql-server:8.0.28-aarch64
in /docker-compose.yml
The default docker ES-6 app does not have an ARM64/v8 image. You can replace it with ES-7 with no ill effects (that I've spotted yet) by replacing elasticsearch-6 dependencies in (eg) /projects/licence_finder/docker-compose.yml (affects: licence-finder, search-api)
Mongo 2.6 is not supported. You can replace it with Mongo 3.6 with no ill effects by replacing mongo2.6 with mongo3.6 in (eg) /projects/router/docker-compose.yml (affects: router, router_api)
govuk-puppet can't run, because it requires Ruby 1.9.3 at the moment (not supported)
brew tap hashicorp/tap
brew install hashicorp/tap/terraform
@KludgeKML
Until govuk_tests is updated, you will need to tell it not to try to find drivers
We released govuk_test v3.0.0 today 🚀
The GOVUK_TEST_USE_SYSTEM_CHROMEDRIVER
workaround won't be needed once apps have been updated. And despite it being a major release, dependabot should pick it up because our repos don't specify a version constraint.
I've put together a branch that should contain all the known fixes (thanks @KludgeKML, they're mostly yours) that collates what we know so far: https://github.com/alphagov/govuk-docker/pull/581
An issue to track findings on running govuk-docker on M1 Laptops.
Current issues: