search

instructure / canvas-lms

The open LMS by Instructure, Inc.
https://github.com/instructure/canvas-lms/wiki
GNU Affero General Public License v3.0
5.71k stars 2.52k forks source link

Parallel::UndumpableException: RuntimeError: error running dress_code #1836

Open argctl opened 3 years ago

argctl commented 3 years ago

Summary:

Running instructions in the production start guide causes error in ruby build process

Steps to reproduce:

  1. Follow setup up to RAILS_ENV=production bundle exec rake canvas:compile_assets
  2. Run RAILS_ENV=production bundle exec rake canvas:compile_assets
  3. Wait

Expected behavior:

Finishes build without errors

Actual behavior:

Gives:

rake aborted!
Parallel::UndumpableException: RuntimeError: error running dress_code
/var/canvas/lib/tasks/css.rake:26:in `block (2 levels) in <top (required)>'
/var/canvas/lib/tasks/canvas.rake:76:in `block (6 levels) in <top (required)>'
/var/canvas/lib/tasks/canvas.rake:10:in `log_time'
/var/canvas/lib/tasks/canvas.rake:79:in `block (5 levels) in <top (required)>'
/var/canvas/vendor/bundle/ruby/2.6.0/gems/parallel-1.20.0/lib/parallel.rb:507:in `call_with_index'
/var/canvas/vendor/bundle/ruby/2.6.0/gems/parallel-1.20.0/lib/parallel.rb:477:in `process_incoming_jobs'
/var/canvas/vendor/bundle/ruby/2.6.0/gems/parallel-1.20.0/lib/parallel.rb:459:in `block in worker'
/var/canvas/vendor/bundle/ruby/2.6.0/gems/parallel-1.20.0/lib/parallel.rb:450:in `fork'
/var/canvas/vendor/bundle/ruby/2.6.0/gems/parallel-1.20.0/lib/parallel.rb:450:in `worker'
/var/canvas/vendor/bundle/ruby/2.6.0/gems/parallel-1.20.0/lib/parallel.rb:441:in `block in create_workers'
/var/canvas/vendor/bundle/ruby/2.6.0/gems/parallel-1.20.0/lib/parallel.rb:440:in `each'
/var/canvas/vendor/bundle/ruby/2.6.0/gems/parallel-1.20.0/lib/parallel.rb:440:in `each_with_index'
/var/canvas/vendor/bundle/ruby/2.6.0/gems/parallel-1.20.0/lib/parallel.rb:440:in `create_workers'
/var/canvas/vendor/bundle/ruby/2.6.0/gems/parallel-1.20.0/lib/parallel.rb:380:in `work_in_processes'
/var/canvas/vendor/bundle/ruby/2.6.0/gems/parallel-1.20.0/lib/parallel.rb:283:in `map'
/var/canvas/lib/tasks/canvas.rake:72:in `block (4 levels) in <top (required)>'
/var/canvas/lib/tasks/canvas.rake:71:in `each'
/var/canvas/lib/tasks/canvas.rake:71:in `block (3 levels) in <top (required)>'
/var/canvas/lib/tasks/canvas.rake:70:in `block (2 levels) in <top (required)>'
/var/canvas/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/exe/rake:27:in `<top (required)>'
Tasks: TOP => canvas:compile_assets
(See full trace by running task with --trace)

Additional notes:

Not the first time I've had issues installing canvas. Possibly a Ruby versioning issue still left in the guides?

argctl commented 3 years ago

Changed to ruby version 2.4 and same issue

argctl commented 3 years ago

Still an issue

argctl commented 3 years ago 
#17 19.56 /var/canvas/vendor/bundle/ruby/2.6.0/gems/dress_code-1.2.0/lib/dress_code/extractor.rb:31:in `scan': invalid byte sequence in US-ASCII (ArgumentError)
#17 19.56       from /var/canvas/vendor/bundle/ruby/2.6.0/gems/dress_code-1.2.0/lib/dress_code/extractor.rb:31:in `scan'
#17 19.56       from /var/canvas/vendor/bundle/ruby/2.6.0/gems/dress_code-1.2.0/lib/dress_code/extractor.rb:25:in `parse_file'
#17 19.56       from /var/canvas/vendor/bundle/ruby/2.6.0/gems/dress_code-1.2.0/lib/dress_code/extractor.rb:20:in `block in parse_files'
#17 19.56       from /var/canvas/vendor/bundle/ruby/2.6.0/gems/dress_code-1.2.0/lib/dress_code/extractor.rb:20:in `each'
#17 19.56       from /var/canvas/vendor/bundle/ruby/2.6.0/gems/dress_code-1.2.0/lib/dress_code/extractor.rb:20:in `flat_map'
#17 19.56       from /var/canvas/vendor/bundle/ruby/2.6.0/gems/dress_code-1.2.0/lib/dress_code/extractor.rb:20:in `parse_files'
#17 19.56       from /var/canvas/vendor/bundle/ruby/2.6.0/gems/dress_code-1.2.0/lib/dress_code/extractor.rb:15:in `extract'
#17 19.56       from /var/canvas/vendor/bundle/ruby/2.6.0/gems/dress_code-1.2.0/bin/dress_code:88:in `<top (required)>'
#17 19.56       from bin/dress_code:16:in `load'
#17 19.56       from bin/dress_code:16:in `<main>'

this appears to be an issue that happens prior to the actual error. I found something from a long time ago that told me to do this:

ENV LANG en_US.UTF-8
ENV LANGUAGE en_US.UTF-8
ENV LC_ALL en_US.UTF-8

checking further on this.

argctl commented 3 years ago

https://stackoverflow.com/questions/17031651/rails-invalid-byte-sequence-in-us-ascii-argument-error-when-i-run-rake-dbsee

amireh commented 3 years ago

Can you paste the contents of config/styleguide.yml or run it through a linter like http://www.yamllint.com to verify its contents are valid YAML?

And on a separate front, can you run RAILS_ENV=production COMPILE_ASSETS_STYLEGUIDE=0 bundle exec rake canvas:compile_assets and see if it goes through?

argctl commented 3 years ago

styleguide.yml should be the one from the repo, here it is:

# required - the file to generate
out_file: app/views/info/styleguide.html.erb

# required - the files to extract docs from
glob: app/stylesheets/**/*.scss

# optional - components can display where they are defined, this will be
# stripped from that definition
base_dir: app/stylesheets

# optional - mustache template rendered for the `out_file`
template: doc/styleguide/template.mustache

# optional - Inline DressCode CSS styles, defaults to true
dress_code_css: true

# optional - Inline DressCode JS, defaults to true
dress_code_js: true

Running RAILS_ENV=production COMPILE_ASSETS_STYLEGUIDE=0 bundle exec rake canvas:compile_assets now and will update when it's done

argctl commented 3 years ago

same error when I run RAILS_ENV=production COMPILE_ASSETS_STYLEGUIDE=0 bundle exec rake canvas:compile_assets. I am running with sudo, but I chowned the entire directory with the sysadmin user and got the same results yesterday.

argctl commented 3 years ago

doing some line by line debugging by overriding the dress_code/extractor.rb file, I get this for the path under my debug message:

#18 20.59 /var/canvas/vendor/bundle/ruby/2.6.0/gems/dress_code-1.2.0/lib/dress_code/extractor.rb:33:in `scan': invalid byte sequence in US-ASCII (ArgumentError)
#18 20.59       from /var/canvas/vendor/bundle/ruby/2.6.0/gems/dress_code-1.2.0/lib/dress_code/extractor.rb:33:in `scan'
#18 20.59       from /var/canvas/vendor/bundle/ruby/2.6.0/gems/dress_code-1.2.0/lib/dress_code/extractor.rb:27:in `parse_file'
#18 20.59       from /var/canvas/vendor/bundle/ruby/2.6.0/gems/dress_code-1.2.0/lib/dress_code/extractor.rb:20:in `block in parse_files'
#18 20.59       from /var/canvas/vendor/bundle/ruby/2.6.0/gems/dress_code-1.2.0/lib/dress_code/extractor.rb:20:in `each'
#18 20.59       from /var/canvas/vendor/bundle/ruby/2.6.0/gems/dress_code-1.2.0/lib/dress_code/extractor.rb:20:in `flat_map'
#18 20.59       from /var/canvas/vendor/bundle/ruby/2.6.0/gems/dress_code-1.2.0/lib/dress_code/extractor.rb:20:in `parse_files'
#18 20.59       from /var/canvas/vendor/bundle/ruby/2.6.0/gems/dress_code-1.2.0/lib/dress_code/extractor.rb:15:in `extract'
#18 20.59       from /var/canvas/vendor/bundle/ruby/2.6.0/gems/dress_code-1.2.0/bin/dress_code:88:in `<top (required)>'
#18 20.59       from bin/dress_code:16:in `load'
#18 20.59       from bin/dress_code:16:in `<main>'
#18 20.59 PAAAAAAAAAAAAATTTTTTTTTTTTTHHHHHHHHHHHHHH HHHHHHHHEEEEEEEEEERRRRRRRRRREEEEEEEEEE
#18 20.59 app/stylesheets/base/_layout.scss
#18 20.59 PAAAAAAAAAAAAATTTTTTTTTTTTTHHHHHHHHHHHHHH HHHHHHHHEEEEEEEEEERRRRRRRRRREEEEEEEEEE
#18 20.59 app/stylesheets/base/_#right-side.scss
#18 20.59 PAAAAAAAAAAAAATTTTTTTTTTTTTHHHHHHHHHHHHHH HHHHHHHHEEEEEEEEEERRRRRRRRRREEEEEEEEEE
#18 20.59 app/stylesheets/base/_#left-side.scss
#18 20.59 PAAAAAAAAAAAAATTTTTTTTTTTTTHHHHHHHHHHHHHH HHHHHHHHEEEEEEEEEERRRRRRRRRREEEEEEEEEE
#18 20.59 app/stylesheets/base/_print.scss
amireh commented 3 years ago

The flag should've made it skip that particular task, I'm surprised you still got the same error. Can you provide more information about your environment: OS, Ruby, Bundler, Node, and Canvas versions (git rev-parse HEAD)?

argctl commented 3 years ago

Sure, I'm installing from scratch in docker for ubuntu 16.04. Here is the base image I created:

https://hub.docker.com/layers/143495118/dkamer/canvas-lms-base/stable/images/sha256-d55d85b8d84a9c1e0d57d865e49c562dcfb0b6be4eae10946722436386330ecb?context=explore

I only made one change to the production guide. I used 2.2.0 for bundler because I received an error when using the suggested version. The error told me it required 2.2.0 to 2.2.11

It's node 14, ruby 2.6, and canvas on stable branch from yesterday (6db5ad0517d6260512653a315eeb169a6eb5ff9a).

argctl commented 3 years ago

Here's the dockerfile for the above image:

FROM ubuntu:16.04

RUN apt update

ENV DEBIAN_FRONTEND noninteractive

RUN apt-get install -y git-core sudo
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y software-properties-common
RUN add-apt-repository ppa:brightbox/ruby-ng
RUN apt update
RUN apt-get install -y ruby2.6 ruby2.6-dev zlib1g-dev libxml2-dev \
                       libsqlite3-dev postgresql libpq-dev \
                       libxmlsec1-dev curl make g++

RUN curl -sL https://deb.nodesource.com/setup_14.x | bash -
RUN apt-get install -y nodejs
RUN apt-get install -y python

RUN adduser --disabled-password --gecos '' sysadmin
RUN adduser sysadmin sudo
RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
USER sysadmin

WORKDIR /home/sysadmin

RUN git clone https://github.com/instructure/canvas-lms.git canvas
WORKDIR canvas
RUN git checkout stable

RUN sudo mkdir -p /var/canvas
RUN sudo chown -R sysadmin /var/canvas
WORKDIR /home/sysadmin/canvas
RUN sudo cp -av . /var/canvas
WORKDIR /var/canvas

RUN sudo gem install bundler --version 2.2.0
RUN bundle _2.2.0_ install --path vendor/bundle

RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
RUN echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
RUN sudo apt-get update && sudo apt-get install -y yarn=1.19.1-1
RUN yarn install --verbose --network-timeout 100000000
argctl commented 3 years ago

Here's the one throwing the error:

FROM dkamer/canvas-lms-base:stable

RUN sudo adduser --disabled-password --gecos '' canvasuser
# RUN adduser canvasuser sudo
# RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoer

USER sysadmin
WORKDIR /var/canvas
COPY assets/config/database.yml config/database.yml
COPY assets/config/security.yml config/security.yml
COPY assets/config/domain.yml config/domain.yml
# COPY assets/config/dynamic_settings.yml config/dynamic_settings.yml
# COPY assets/config/outgoing_mail.yml config/outgoing_mail.yml

RUN mkdir -p log tmp/pids public/assets app/stylesheets/brandable_css_brands
RUN touch app/stylesheets/_brandable_variables_defaults_autogenerated.scss
RUN touch Gemfile.lock
RUN touch log/production.log
RUN sudo chown -R canvasuser config/environment.rb log tmp public/assets \
      app/stylesheets/_brandable_variables_defaults_autogenerated.scss \
      app/stylesheets/brandable_css_brands Gemfile.lock config.ru
RUN yarn install
# RUN sudo chown -R sysadmin /var/canvas
# RUN chmod +x /var/canvas/bin/dress_code
# ENV LANG en_US.UTF-8
# ENV LANG US-ASCII
# ENV LANGUAGE US-ASCII
# ENV LC_ALL US-ASCII
# ENV LANGUAGE en_US.UTF-8
# ENV LC_ALL en_US.UTF-8
# RUN yarn upgrade
# RUN export LANG=en_US.UTF-8
# RUN export LANGUAGE=en_US.UTF-8
# RUN export LC_ALL=en_US.UTF-8
COPY assets/override/extractor.rb /var/canvas/vendor/bundle/ruby/2.6.0/gems/dress_code-1.2.0/lib/dress_code/extractor.rb
# RUN RAILS_ENV=production sudo bundle exec rake canvas:compile_assets --trace
RUN RAILS_ENV=production COMPILE_ASSETS_STYLEGUIDE=0 sudo bundle exec rake canvas:compile_assets
RUN sudo chown -R canvasuser public/dist/brandable_css
amireh commented 3 years ago

Thanks, I'm using your image and was able to reproduce. I'll push a fix shortly.

argctl commented 3 years ago

Thanks, I'll just manually override the files with your changes for now so we can get moving.

huangdengpan commented 3 years ago

change dress_code version, solved

brandonfranzke commented 3 years ago

@huangdengpan can you elaborate how you changed the dress_code version. I tried by:

  1. clone https://github.com/instructure/dress_code
  2. gem install dress_code
  3. cp ~/.gem//bin/dress_code bin/dress_code

but I receive the same error. I am unfamiliar with ruby so maybe there are extra steps to run? I appreciate any help

rake aborted!
Parallel::UndumpableException: RuntimeError: error running dress_code
/usr/src/app/lib/tasks/css.rake:28:in `block (2 levels) in <top (required)>'
/usr/src/app/lib/tasks/canvas.rake:78:in `block (6 levels) in <top (required)>'
/usr/src/app/lib/tasks/canvas.rake:12:in `log_time'
/usr/src/app/lib/tasks/canvas.rake:81:in `block (5 levels) in <top (required)>'
/home/docker/.gem/gems/parallel-1.20.0/lib/parallel.rb:507:in `call_with_index'
/home/docker/.gem/gems/parallel-1.20.0/lib/parallel.rb:477:in `process_incoming_jobs'
/home/docker/.gem/gems/parallel-1.20.0/lib/parallel.rb:459:in `block in worker'
/home/docker/.gem/gems/parallel-1.20.0/lib/parallel.rb:450:in `fork'
/home/docker/.gem/gems/parallel-1.20.0/lib/parallel.rb:450:in `worker'
/home/docker/.gem/gems/parallel-1.20.0/lib/parallel.rb:441:in `block in create_workers'
/home/docker/.gem/gems/parallel-1.20.0/lib/parallel.rb:440:in `each'
/home/docker/.gem/gems/parallel-1.20.0/lib/parallel.rb:440:in `each_with_index'
/home/docker/.gem/gems/parallel-1.20.0/lib/parallel.rb:440:in `create_workers'
/home/docker/.gem/gems/parallel-1.20.0/lib/parallel.rb:380:in `work_in_processes'
/home/docker/.gem/gems/parallel-1.20.0/lib/parallel.rb:283:in `map'
/usr/src/app/lib/tasks/canvas.rake:74:in `block (4 levels) in <top (required)>'
/usr/src/app/lib/tasks/canvas.rake:73:in `each'
/usr/src/app/lib/tasks/canvas.rake:73:in `block (3 levels) in <top (required)>'
/usr/src/app/lib/tasks/canvas.rake:72:in `block (2 levels) in <top (required)>'
/home/docker/.gem/gems/rake-13.0.3/exe/rake:27:in `<top (required)>'
Tasks: TOP => canvas:compile_assets
(See full trace by running task with --trace)

UPDATE:

I do notice that running the system command in lib/tasks/css.rake manually:

$ bin/dress_code config/styleguide.yml
# writing file: app/views/info/styleguide.html.erb
$ echo $?
0
hkpanchani commented 3 years ago

any luck solving this bug?? I am stucked at installation.

amireh commented 3 years ago

@hkpanchani while the problem was fixed in the gem, I forgot to update canvas-lms to use the newer version. In the meantime, you can manually update dress_code to version 1.2.1 inside Gemfile.d/assets.rb.

amireh commented 3 years ago

OK the problem should go away if you pull from latest canvas-lms master, 00d076d. Thanks all.

gregawoods commented 3 years ago

Hi there,

I get the same error on stable branch. I checked and I am using dress_code 1.2.1. My ruby version is 2.6.8.

Adding the COMPILE_ASSETS_STYLEGUIDE=0 flag does allow it to run without error. Am I missing anything vital by running it this way?