search

instructure / canvas-lms

The open LMS by Instructure, Inc.
https://github.com/instructure/canvas-lms/wiki
GNU Affero General Public License v3.0
5.7k stars 2.51k forks source link

Undumpable Exception with canvas:compile_assets rake task #559

Closed redconfetti closed 9 years ago

redconfetti commented 9 years ago

Just updated my local development instance of Canvas on master branch, receiving this error when compiling assets. This seems similar to issues reported in #552, #498, and #547, though different errors arose in those reports.

$ bundle exec rake canvas:compile_assets
DEPRECATION WARNING: You have Rails 2.3-style plugins in vendor/plugins! Support for these plugins will be removed in Rails 4.0. Move them out and bundle them in your Gemfile, or fold them in to your app as lib/myplugin/* and config/initializers/myplugin.rb. See the release notes for more on this: http://weblog.rubyonrails.org/2012/1/4/rails-3-2-0-rc2-has-been-released. (called from <top (required)> at /Users/jasonmiller/Projects/canvas-lms/Rakefile:11)
DEPRECATION WARNING: You have Rails 2.3-style plugins in vendor/plugins! Support for these plugins will be removed in Rails 4.0. Move them out and bundle them in your Gemfile, or fold them in to your app as lib/myplugin/* and config/initializers/myplugin.rb. See the release notes for more on this: http://weblog.rubyonrails.org/2012/1/4/rails-3-2-0-rc2-has-been-released. (called from <top (required)> at /Users/jasonmiller/Projects/canvas-lms/Rakefile:11)
working in 8 processes
--> Starting: 'Compile sass and make jammit css bundles'
--> Starting: 'npm run compile-sass'
--> Starting: 'css:styleguide'
--> Starting: 'compile coffee, js 18n, and run r.js optimizer'
--> Starting: 'Generate documentation [yardoc]'
--> Starting: 'js:generate'
--> creating styleguide
Building client app 'canvas_quiz_statistics'
    Running 'npm run build'...
/Users/jasonmiller/.rvm/gems/ruby-2.1.3@canvas-lms/bin/rake: No such file or directory - ./script/build
            -------------------------------------------------------------------
            BUILD FAILURE:

            -------------------------------------------------------------------

> canvas-lms@0.0.0 compile-sass /Users/jasonmiller/Projects/canvas-lms
> node script/compile-sass.js

# writing file: app/views/info/styleguide.html.erb
--> Finished: 'css:styleguide' in 6.485412
Files:          91
Modules:         0 (    0 undocumented)
Classes:        90 (   26 undocumented)
Constants:       0 (    0 undocumented)
Methods:       384 (    7 undocumented)
 93.04% documented
API Documentation successfully generated in public/doc/api
See public/doc/api/index.html
--> Finished: 'Generate documentation [yardoc]' in 26.480803
--> Finished: 'npm run compile-sass' in 28.910356
--> Starting: 'Jammit'
--> Finished: 'Jammit' in 0.717718
--> Finished: 'Compile sass and make jammit css bundles' in 29.628696
rake aborted!
Undumpable Exception -- #<RuntimeError: Build script failed for client app canvas_quiz_statistics>
/Users/jasonmiller/.rvm/gems/ruby-2.1.3@canvas-lms/gems/parallel-0.5.16/lib/parallel.rb:173:in `work_in_processes'
/Users/jasonmiller/.rvm/gems/ruby-2.1.3@canvas-lms/gems/parallel-0.5.16/lib/parallel.rb:55:in `map'
/Users/jasonmiller/Projects/canvas-lms/lib/tasks/canvas.rake:167:in `block (3 levels) in <top (required)>'
/Users/jasonmiller/Projects/canvas-lms/lib/tasks/canvas.rake:166:in `block (2 levels) in <top (required)>'
/Users/jasonmiller/.rvm/gems/ruby-2.1.3@canvas-lms/bin/ruby_executable_hooks:15:in `eval'
/Users/jasonmiller/.rvm/gems/ruby-2.1.3@canvas-lms/bin/ruby_executable_hooks:15:in `<main>'
Tasks: TOP => canvas:compile_assets
(See full trace by running task with --trace)

Similar error occurring after I switch to stable branch.

I see that the 'BUILD FAILURE' output is originating from the js:build_client_apps rake task.

I used Homebrew on my machine to update NodeJS, and also used npm cache clean to clear the cache.

redconfetti commented 9 years ago

It appears that this has something to do with building client_apps that are under /public/javascripts/client_apps, which are symlinked to their actual location under /client_apps.

$ pwd
/Users/jasonmiller/Projects/canvas-lms/public/javascripts/client_apps
$ ls -la
total 40
drwxr-xr-x    7 jasonmiller  staff   238 Dec 16 12:32 .
drwxr-xr-x  137 jasonmiller  staff  4658 Dec 16 12:00 ..
-rw-r--r--    1 jasonmiller  staff   162 Dec 16 10:52 README
lrwxr-xr-x    1 jasonmiller  staff    71 Dec 16 12:32 canvas_quiz_statistics -> ../../../client_apps/canvas_quiz_statistics/dist/canvas_quiz_statistics
lrwxr-xr-x    1 jasonmiller  staff    74 Sep  8 16:31 canvas_quiz_statistics.js -> ../../../client_apps/canvas_quiz_statistics/dist/canvas_quiz_statistics.js
lrwxr-xr-x    1 jasonmiller  staff    55 Dec 16 12:32 canvas_quizzes -> ../../../client_apps/canvas_quizzes/dist/canvas_quizzes
lrwxr-xr-x    1 jasonmiller  staff    58 Dec 16 12:32 canvas_quizzes.js -> ../../../client_apps/canvas_quizzes/dist/canvas_quizzes.js

It looks like work on these is very recent, and as noted they're "TOTALLY EXPERIMENTAL".

I commented out lib/tasks/js.rake:206 and this stopped the build failure error, but still didn't resolve the primary issue.

...
API Documentation successfully generated in public/doc/api
See public/doc/api/index.html
--> Finished: 'Generate documentation [yardoc]' in 40.460169
--> Finished: 'npm run compile-sass' in 41.68782
--> Starting: 'Jammit'
--> Finished: 'Jammit' in 0.773902
--> Finished: 'Compile sass and make jammit css bundles' in 42.462354
rake aborted!
Undumpable Exception -- #<SystemExit: exit>
/Users/jasonmiller/.rvm/gems/ruby-2.1.3@canvas-lms/gems/parallel-0.5.16/lib/parallel.rb:173:in `work_in_processes'
/Users/jasonmiller/.rvm/gems/ruby-2.1.3@canvas-lms/gems/parallel-0.5.16/lib/parallel.rb:55:in `map'
/Users/jasonmiller/Projects/canvas-lms/lib/tasks/canvas.rake:167:in `block (3 levels) in <top (required)>'
/Users/jasonmiller/Projects/canvas-lms/lib/tasks/canvas.rake:166:in `block (2 levels) in <top (required)>'
/Users/jasonmiller/.rvm/gems/ruby-2.1.3@canvas-lms/bin/ruby_executable_hooks:15:in `eval'
/Users/jasonmiller/.rvm/gems/ruby-2.1.3@canvas-lms/bin/ruby_executable_hooks:15:in `<main>'
Tasks: TOP => canvas:compile_assets
(See full trace by running task with --trace)
redconfetti commented 9 years ago

I tried to see which exact tasks were failing. It appears that both js:generate and js:build are failing.

$ $GEM_HOME/bin/bundle exec rake js:generate
Building client app 'canvas_quiz_statistics'
    Running 'npm run build'...
/Users/jasonmiller/.rvm/gems/ruby-2.1.3@canvas-lms/bin/rake: No such file or directory - ./script/build
            -------------------------------------------------------------------
            BUILD FAILURE:

            -------------------------------------------------------------------
rake aborted!
Build script failed for client app canvas_quiz_statistics
/Users/jasonmiller/Projects/canvas-lms/lib/tasks/js.rake:177:in `block (4 levels) in <top (required)>'
/Users/jasonmiller/Projects/canvas-lms/lib/tasks/js.rake:148:in `chdir'
/Users/jasonmiller/Projects/canvas-lms/lib/tasks/js.rake:148:in `block (3 levels) in <top (required)>'
/Users/jasonmiller/Projects/canvas-lms/lib/tasks/js.rake:145:in `each'
/Users/jasonmiller/Projects/canvas-lms/lib/tasks/js.rake:145:in `block (2 levels) in <top (required)>'
/Users/jasonmiller/Projects/canvas-lms/lib/tasks/js.rake:206:in `block (2 levels) in <top (required)>'
/Users/jasonmiller/.rvm/gems/ruby-2.1.3@canvas-lms/bin/ruby_executable_hooks:15:in `eval'
/Users/jasonmiller/.rvm/gems/ruby-2.1.3@canvas-lms/bin/ruby_executable_hooks:15:in `<main>'
Tasks: TOP => js:build_client_apps
(See full trace by running task with --trace)
$ $GEM_HOME/bin/bundle exec rake js:build
--> Concatenating JavaScript bundles with r.js
rake aborted!
Error running js:build:
Error: Error: ENOENT, no such file or directory '/Users/jasonmiller/Projects/canvas-lms/public/javascripts/client_apps/canvas_quiz_statistics'
    at Object.fs.statSync (fs.js:695:18)

ABORTING/Users/jasonmiller/Projects/canvas-lms/lib/tasks/js.rake:280:in `block (3 levels) in <top (required)>'
/Users/jasonmiller/Projects/canvas-lms/lib/tasks/js.rake:278:in `block (2 levels) in <top (required)>'
/Users/jasonmiller/.rvm/gems/ruby-2.1.3@canvas-lms/bin/ruby_executable_hooks:15:in `eval'
/Users/jasonmiller/.rvm/gems/ruby-2.1.3@canvas-lms/bin/ruby_executable_hooks:15:in `<main>'
Tasks: TOP => js:build
(See full trace by running task with --trace)

I'm rolling back to an older version just so that I can continue doing my LTI integration work now.

redconfetti commented 9 years ago

Updated to c3c0a29c on master branch, getting a different error now.

bundle exec rake canvas:compile_assets
DEPRECATION WARNING: You have Rails 2.3-style plugins in vendor/plugins! Support for these plugins will be removed in Rails 4.0. Move them out and bundle them in your Gemfile, or fold them in to your app as lib/myplugin/* and config/initializers/myplugin.rb. See the release notes for more on this: http://weblog.rubyonrails.org/2012/1/4/rails-3-2-0-rc2-has-been-released. (called from <top (required)> at /Users/jasonmiller/Projects/canvas-lms/Rakefile:11)
DEPRECATION WARNING: You have Rails 2.3-style plugins in vendor/plugins! Support for these plugins will be removed in Rails 4.0. Move them out and bundle them in your Gemfile, or fold them in to your app as lib/myplugin/* and config/initializers/myplugin.rb. See the release notes for more on this: http://weblog.rubyonrails.org/2012/1/4/rails-3-2-0-rc2-has-been-released. (called from <top (required)> at /Users/jasonmiller/Projects/canvas-lms/Rakefile:11)
working in 8 processes
--> Starting: 'Compile sass and make jammit css bundles'
--> Starting: 'css:styleguide'
--> Starting: 'npm run compile-sass'
--> Starting: 'compile coffee, js 18n, and run r.js optimizer'
--> Starting: 'Generate documentation [yardoc]'
--> Starting: 'js:generate'
--> creating styleguide
Building client app 'canvas_quizzes'
    Running 'npm install'...

> canvas-lms@0.0.0 compile-sass /Users/jasonmiller/Projects/canvas-lms
> node script/compile-sass.js

npm WARN package.json canvas_quizzes@1.0.0 No repository field.
    Running 'npm run build'...
Client app 'canvas_quizzes' was built successfully.
--> Creating ember app bundles
--> Generating plugin extensions
--> Compiling React JSX
--> Creating ember app bundles finished in 0.019937
--> Pre-compiling handlebars templates
--> Pre-compiling ember handlebars templates
--> Generating plugin extensions finished in 0.041704
--> Compiling CoffeeScript with 'coffee' binary
# writing file: app/views/info/styleguide.html.erb
--> Finished: 'css:styleguide' in 7.979734
--> Compiling React JSX finished in 4.015904
--> Pre-compiling ember handlebars templates finished in 4.272083
(libuv) Failed to create kqueue (24)
(libuv) Failed to create kqueue (24)

events.js:72
        throw er; // Unhandled 'error' event
              ^
Error: write EBADF
    at errnoException (net.js:904:11)
    at WriteStream.Socket._write (net.js:645:26)
    at doWrite (_stream_writable.js:225:10)
    at writeOrBuffer (_stream_writable.js:215:5)
    at WriteStream.Writable.write (_stream_writable.js:182:11)
    at WriteStream.Socket.write (net.js:615:40)
    at printLine (/Users/jasonmiller/Projects/canvas-lms/node_modules/coffee-script/lib/coffee-script/command.js:24:27)
    at /Users/jasonmiller/Projects/canvas-lms/node_modules/coffee-script/lib/coffee-script/command.js:436:20
    at fs.js:955:21
    at Object.oncomplete (fs.js:107:15)
--> Pre-compiling handlebars templates finished in 8.341474
Files:          91
Modules:         0 (    0 undocumented)
Classes:        90 (   26 undocumented)
Constants:       0 (    0 undocumented)
Methods:       384 (    7 undocumented)
 93.04% documented
API Documentation successfully generated in public/doc/api
See public/doc/api/index.html
--> Finished: 'Generate documentation [yardoc]' in 32.991009
--> Finished: 'npm run compile-sass' in 34.689515
--> Starting: 'Jammit'
--> Finished: 'Jammit' in 0.69281
--> Finished: 'Compile sass and make jammit css bundles' in 35.382921
rake aborted!
Undumpable Exception -- #<RuntimeError: Unable to compile coffeescripts in /Users/jasonmiller/Projects/canvas-lms/app/coffeescripts/bundles>
/Users/jasonmiller/.rvm/gems/ruby-2.1.3@canvas-lms/gems/parallel-0.5.16/lib/parallel.rb:173:in `work_in_processes'
/Users/jasonmiller/.rvm/gems/ruby-2.1.3@canvas-lms/gems/parallel-0.5.16/lib/parallel.rb:55:in `map'
/Users/jasonmiller/Projects/canvas-lms/lib/tasks/canvas.rake:167:in `block (3 levels) in <top (required)>'
/Users/jasonmiller/Projects/canvas-lms/lib/tasks/canvas.rake:166:in `block (2 levels) in <top (required)>'
/Users/jasonmiller/.rvm/gems/ruby-2.1.3@canvas-lms/bin/ruby_executable_hooks:15:in `eval'
/Users/jasonmiller/.rvm/gems/ruby-2.1.3@canvas-lms/bin/ruby_executable_hooks:15:in `<main>'
Tasks: TOP => canvas:compile_assets
(See full trace by running task with --trace)
redconfetti commented 9 years ago

Well, I don't know if I was getting dinged by a bug this morning regarding the canvas_quiz_statistics client app build process, and then rebasing to a fresher version of master resolved that... but left me with this coffeescript error.

I'm on a Mac running 10.9.5. I searched online regarding the '(libuv) Failed to create kqueue (24)' error above and found Randomly getting (libuv) Failed to create kqueue (24) Exception #5658.

This issue was resolved when I ran ulimit -n 8192 as suggested in that issue.

See also Increase the maximum number of open file descriptors and Wikipedia - File_descriptor

javawizard commented 9 years ago

A fresh clone or a rm -r client_apps/canvas_quiz_statistics should also help.

rsaray commented 9 years ago

What is the reason for this warning? npm WARN package.json canvas_quizzes@1.0.0 No repository field.

claydiffrient commented 9 years ago

@rsaray That simply means that in the package.json file for canvas_quizzes there isn't a repository specified. It can be completely ignored, as it won't cause any adverse effects.

suyogtaksande commented 9 years ago

i am getting this warning. need suggestions

npm WARN package.json canvas_quizzes@1.0.0 No repository field

Files: 97 Modules: 0 ( 0 undocumented) Classes: 96 ( 30 undocumented) Constants: 0 ( 0 undocumented) Methods: 424 ( 9 undocumented) 92.50% documented API Documentation successfully generated in public/doc/api See public/doc/api/index.html --> Finished: 'Generate documentation [yardoc]' in 45.1148107179979 rake aborted! Undumpable Exception -- # /home/suyog/.rvm/gems/ruby-2.2.1/gems/parallel-1.4.1/lib/parallel.rb:63:in work' /home/suyog/.rvm/gems/ruby-2.2.1/gems/parallel-1.4.1/lib/parallel.rb:280:inblock (4 levels) in work_in_processes' /home/suyog/.rvm/gems/ruby-2.2.1/gems/parallel-1.4.1/lib/parallel.rb:447:in with_instrumentation' /home/suyog/.rvm/gems/ruby-2.2.1/gems/parallel-1.4.1/lib/parallel.rb:279:inblock (3 levels) in work_in_processes' /home/suyog/.rvm/gems/ruby-2.2.1/gems/parallel-1.4.1/lib/parallel.rb:273:in loop' /home/suyog/.rvm/gems/ruby-2.2.1/gems/parallel-1.4.1/lib/parallel.rb:273:inblock (2 levels) in work_in_processes' /home/suyog/.rvm/gems/ruby-2.2.1/gems/parallel-1.4.1/lib/parallel.rb:138:in `block (2 levels) in in_threads' Tasks: TOP => canvas:compile_assets (See full trace by running task with --trace)

simonista commented 9 years ago

Looks like something about running your rake tasks in parallel is causing problems. I would try temporarily modifying lines 163 - 169 of lib/tasks/canvas.rake from

times = nil
real_time = Benchmark.realtime do
  times = Parallel.map(tasks, :in_processes => processes.to_i) do |name, lamduh|
    log_time(name) { lamduh.call }
  end
end
combined_time = times.reduce(:+)

to

tasks.map do |name, fn|
  puts "running #{name}"
  fn.call
end

That will be slower but hopefully will help us narrow down the problem.

simonista commented 8 years ago

That's actually good, hopefully that's the real problem. Do you have the full stacktrace?