search

instructure / canvas-lms

The open LMS by Instructure, Inc.
https://github.com/instructure/canvas-lms/wiki
GNU Affero General Public License v3.0
5.69k stars 2.51k forks source link

docker-compose setup fails on Linux #1649

Closed grahamb closed 4 years ago

grahamb commented 4 years ago

Summary:

Attempting to start the docker-compse setup (manually, not using the script, but it shouldn't matter) fails with a yarn error in client_apps/canvas_quizzes

Steps to reproduce:

  1. Clone the repo
  2. Run docker-compose run web bundle install
  3. Wait

Expected behavior:

The command should complete successfully and exit without error.

Actual behavior:

The Dockerfile calls the canvas:compile_assets rake task. During that task, it attempts to build the apps in client_apps. That task fails with an error (full output attached to this issue):

Building client app 'canvas_quizzes'
warning package.json: No license field
warning " > graphiql-explorer@0.4.3" has incorrect peer dependency "graphql@^0.6.0 || ^0.7.0 || ^0.8.0-b || ^0.9.0 || ^0.10.0 || ^0.11.0 || ^0.12.0 || ^0.13.0".
warning " > karma-webpack@3.0.5" has incorrect peer dependency "webpack@^2.0.0 || ^3.0.0".
warning " > moment-timezone-data-webpack-plugin@1.1.0" has unmet peer dependency "moment-timezone@>= 0.1.0".
warning "workspace-aggregator-64029318-0a00-4560-a03d-fdaaec5985da > canvas-rce > axe-testcafe@3.0.0" has incorrect peer dependency "axe-core@>=2.2.3 <4".
warning "workspace-aggregator-64029318-0a00-4560-a03d-fdaaec5985da > canvas_quizzes > jasmine_react > react@0.10.0" has unmet peer dependency "envify@~1.2.0".
error An unexpected error occurred: "ENOENT: no such file or directory, lstat '/usr/src/app/node_modules/.yarn-integrity'".
rake aborted!
Package installation failure for client app canvas_quizzes

Additional notes:

OS: RHEL7 Docker version 19.03.5, build 633a0ea docker-compose version 1.25.1, build a82fef07 Docker is running as a regular user; that user is in the docker group and has no problems performing docker operations.

canvas_docker_output.txt

grahamb commented 4 years ago

This looks like it's a Red Hat issue; I've been able to work around it. However, I'm running into a new issue that appears permission-related. Running dcr bundle install errors out because it can't write Gemfile.lock

There was an error while trying to write to/usr/src/app/Gemfile.lock. It is likely that you need to grant write permissions for that path.

If I run dcr ls -la, I get:

grahamb@lcp-grahamb-dev  ~/code/canvas  ef20c9  sfu-release/2020-06-18 ●  dcr ls -la .
Starting canvas_redis_1    ... done
Starting canvas_postgres_1 ... done
total 1620
drwxr-xr-x.   30  11659   8088   4096 Jun 16 17:26 .
drwxr-xr-x.    4 root   root       48 Jun 16 03:39 ..
drwxr-xr-x.   15  11659   8088   4096 Jan 20 23:14 app
-rw-r--r--.    1  11659   8088   2016 Jan 20 23:14 babel.config.js
drwxr-xr-x.    2  11659   8088   4096 Jun  8 23:09 bin
-rw-r--r--.    1  11659   8088    244 Jan 20 23:14 bower.json
-rw-r--r--.    1  11659   8088     46 Jan 20 23:14 .bowerrc
drwxr-xr-x.    6  11659   8088   4096 Apr 15 22:43 build
drwxr-xr-x.    3  11659   8088     40 Jan 20 23:14 client_apps
-rw-r--r--.    1  11659   8088     38 Jan 20 23:14 .codeclimate.yml
-rw-r--r--.    1  11659   8088   1676 Jan 20 23:14 code_of_conduct.md
drwxr-xr-x.    7  11659   8088   4096 Jun 15 21:30 config
-rw-r--r--.    1  11659   8088    236 Jan 20 23:14 config.ru
-rw-r--r--.    1  11659   8088   6248 Jan 20 23:14 CONTRIBUTING.md
-rw-r--r--.    1  11659   8088    621 Jan 20 23:14 COPYRIGHT
drwxr-xr-x.    3  11659   8088     20 Jan 20 23:14 db
-rw-r--r--.    1  11659   8088    549 Apr 27 16:26 .dive-ci
drwxr-xr-x.    9  11659   8088   4096 Apr 15 17:09 doc
drwxr-xr-x.    9  11659   8088   4096 May 25 22:45 docker-compose
-rw-r--r--.    1  11659   8088   2181 Jan 20 23:14 docker-compose.jenkins.yml
-rw-r--r--.    1  11659   8088   4023 Jun 16 04:55 docker-compose.local.grahamb.yml
-rw-r--r--.    1  11659   8088    233 May 25 22:45 docker-compose.new-jenkins.canvas.yml
-rw-r--r--.    1  11659   8088    545 Jun  8 23:09 docker-compose.new-jenkins.consumer.yml
-rw-r--r--.    1  11659   8088    174 May 25 22:45 docker-compose.new-jenkins-flakey-spec-catcher.yml
-rw-r--r--.    1  11659   8088    203 May 25 22:45 docker-compose.new-jenkins-karma.yml
-rw-r--r--.    1  11659   8088    922 May 25 22:45 docker-compose.new-jenkins.multiple-processes.yml
-rw-r--r--.    1  11659   8088    734 Jan 20 23:14 docker-compose.new-jenkins-package-translations.yml
-rw-r--r--.    1  11659   8088    993 Jun  8 23:09 docker-compose.new-jenkins-selenium.yml
-rw-r--r--.    1  11659   8088    229 May 25 22:45 docker-compose.new-jenkins.vendored-gems.yml
-rw-r--r--.    1  11659   8088   1232 May 25 22:45 docker-compose.new-jenkins.yml
-rw-r--r--.    1  11659   8088   1953 May 25 22:45 docker-compose.spring.yml
-rw-r--r--.    1  11659   8088    452 May 25 22:45 docker-compose.yml
-rw-r--r--.    1  11659   8088   4174 Jun 16 07:03 Dockerfile
-rw-r--r--.    1  11659   8088     65 Jan 20 23:14 Dockerfile.githook
-rw-r--r--.    1  11659   8088    820 Jan 20 23:14 Dockerfile.package-translations
-rw-r--r--.    1  11659   8088   2852 Apr 15 17:09 Dockerfile-production
-rw-r--r--.    1  11659   8088    736 Apr 15 22:43 .dockerignore
-rw-r--r--.    1  11659   8088    263 Jan 20 23:14 .editorconfig
-rw-r--r--.    1  11659   8088     92 Jun 16 15:41 .env
-rw-r--r--.    1  11659   8088    360 Jan 20 23:14 .eslintignore
-rw-r--r--.    1  11659   8088   7397 May 25 22:45 .eslintrc.js
drwxr-xr-x.    3  11659   8088   4096 May 25 22:45 frontend_build
-rw-r--r--.    1  11659   8088   1260 Jan 20 23:14 Gemfile
drwxr-xr-x.    2  11659   8088   4096 Jun 15 21:30 Gemfile.d
drwxr-xr-x.   53  11659   8088   4096 Jun 16 17:02 gems
drwxr-xr-x.    8  11659   8088   4096 Jun 16 17:28 .git
drwxr-xr-x.    2  11659   8088     44 Jan 20 23:14 .github
-rw-r--r--.    1  11659   8088   1623 Apr 15 22:43 .gitignore
-rw-r--r--.    1  11659   8088    393 Jan 20 23:14 .gitmessage
-rw-r--r--.    1  11659   8088   4783 Jan 20 23:14 gulpfile.js
drwxr-xr-x.    2  11659   8088     23 Jan 20 23:14 hooks
-rw-r--r--.    1  11659   8088     65 Jan 20 23:14 .i18nignore
-rw-r--r--.    1  11659   8088     49 Jan 20 23:14 .i18nrc
-rw-r--r--.    1  11659   8088    378 Jan 20 23:14 issue_template.md
-rw-r--r--.    1  11659   8088  18506 Jun  8 23:09 Jenkinsfile
-rw-r--r--.    1  11659   8088   3958 Jun  8 23:09 Jenkinsfile.contract-tests
-rw-r--r--.    1  11659   8088   1434 May 25 22:45 Jenkinsfile.dive
-rw-r--r--.    1  11659   8088   5395 May 25 22:45 Jenkinsfile.js
-rw-r--r--.    1  11659   8088   2398 May 25 22:45 Jenkinsfile.main-for-coverage
-rw-r--r--.    1  11659   8088   1907 May 25 22:45 Jenkinsfile.master-bouncer-check-all
-rw-r--r--.    1  11659   8088   1850 May 25 22:45 Jenkinsfile.package-translations
-rw-r--r--.    1  11659   8088   7520 May 25 22:45 Jenkinsfile.selenium.flakey_spec_catcher
-rw-r--r--.    1  11659   8088   4689 May 25 22:45 Jenkinsfile.selenium.performance.chrome
-rw-r--r--.    1  11659   8088   1156 May 25 22:45 Jenkinsfile.test-subbuild
-rw-r--r--.    1  11659   8088   2532 May 25 22:45 Jenkinsfile.vendored-gems
-rw-r--r--.    1  11659   8088   2394 May 25 22:45 Jenkinsfile.xbrowser
drwxr-xr-x.    2  11659   8088   4096 Jun  8 23:09 jest
-rw-r--r--.    1  11659   8088   2311 May 25 22:45 jest.config.js
-rw-r--r--.    1  11659   8088   5246 Mar 11 17:05 karma.conf.js
drwxr-xr-x.   31  11659   8088   8192 Jun  8 23:09 lib
-rw-r--r--.    1  11659   8088  34520 Jan 20 23:14 LICENSE
-rw-r--r--.    1  11659   8088     59 Jan 20 23:14 .lintstagedrc.js
drwxr-xr-x.    2 docker docker     27 Jun 16 17:26 log
drwxr-xr-x. 1870 docker docker  53248 Jun 16 17:25 node_modules
-rw-r--r--.    1  11659   8088     18 Jan 20 23:14 .npmrc
-rw-r--r--.    1  11659   8088      9 Apr 27 16:26 .nvmrc
-rw-r--r--.    1  11659   8088  13681 May 25 22:45 package.json
drwxr-xr-x.   14  11659   8088   4096 Jun 16 17:02 packages
drwxr-xr-x.    3  11659   8088     59 Jan 20 23:14 pact
drwxr-xr-x.    2 docker docker      6 Jun 16 17:17 pacts
-rw-r--r--.    1  11659   8088 170532 May 25 22:45 parallel_runtime_rspec.log
-rw-r--r--.    1  11659   8088     91 Jan 20 23:14 .prettierrc
drwxr-xr-x.    9  11659   8088   4096 Mar 30 01:10 provision
drwxr-xr-x.   13  11659   8088   4096 Jun 16 17:26 public
-rw-r--r--.    1  11659   8088    325 Apr 15 22:43 Rakefile
-rw-r--r--.    1  11659   8088    702 Jan 20 23:14 README.md
drwxr-xr-x.    2 docker docker      6 Jun 16 17:17 reports
-rw-r--r--.    1  11659   8088   6078 Mar 11 17:05 .rubocop.yml
drwxr-xr-x.    3  11659   8088   4096 Jun 15 21:30 script
drwxr-xr-x.   31  11659   8088   4096 Jun  8 23:09 spec
-rw-r--r--.    1  11659   8088      0 May 25 22:45 stats-prod.json
-rw-r--r--.    1  11659   8088    433 Jan 20 23:14 .stylelintrc
drwxr-xr-x.    2 docker docker      6 Jun 16 17:17 tmp
-rw-r--r--.    1  11659   8088   1242 Jan 20 23:14 .travis.yml
-rw-r--r--.    1  11659   8088   2824 Mar 30 01:10 Vagrantfile
-rw-r--r--.    1  11659   8088   8600 Mar 30 01:10 Vagrant.md
-rw-r--r--.    1  11659   8088    443 Mar 30 01:10 vagrantrc.yml.example
-rw-r--r--.    1  11659   8088   1035 Mar 30 01:10 webpack.config.js
-rw-r--r--.    1  11659   8088   2473 Jan 20 23:14 webpack.test.config.js
drwxr-xr-x.    3 docker docker     88 Jun 16 17:26 .yardoc
-rw-r--r--.    1  11659   8088 927043 Jun 11 21:35 yarn.lock

11659 is my UID on the host; 8088 is my primary GID on the host. In the container, the docker user's UID and GID are 9999.

I can't change my UID on the host; we use sssd to authenticate against Active Directory and 11659 is my AD UID.

Running bundle install as root (docker-compose run -u root --rm web bundle install) does work, but is not ideal.

tierra commented 4 years ago

With the overrides applied to mount host bind-mounted volumes into the containers (with your code checkout), this is a pretty common thing we also run into as well. I've been using setfacl to just grant the docker containers write access to my host checkout:

setfacl -Rm u:9999:rwX,g:9999:rwX .
setfacl -dRm u:9999:rwX,g:9999:rwX .

I wouldn't say it's ideal, but it does solve a lot of headaches, not just with Gemfile.lock, but also other paths that the Canvas build system writes to.

grahamb commented 4 years ago

Hi @tierra! I just was reading #1221 where you mentioned pre-creating Gemfile.lock, and that did seem to work. I'll give setfacl a try as well. Thanks!

grahamb commented 4 years ago

Boom, that did it. Thanks again!