Receiving directory already being watched error for ember compiled assets

dchersey commented 8 years ago

Every time I start puma I receive a large number of warnings to the tune of

      ** ERROR: directory is already being watched! **

        Directory: /Users/david/projects/ia/frontend/tmp/template_compiler-output_path-lTPkTaqW.tmp/modules/ember-cli-foundation-6-sass/templates

        is already being watched through: /Users/david/projects/ia/frontend/tmp/template_compiler-output_path-lTPkTaqW.tmp/modules/ember-cli-foundation-6-sass/templates

        MORE INFO: https://github.com/guard/listen/wiki/Duplicate-directory-errors

I am using Rails 5 beta 3. But I used to see these with my other rails 4 app too.

I am not using guard or any type of reloading strategy other than what is provided by ember. This is a pretty new app with only a few routes and no rails code yet.

The link in the error message is not very helpful. Have you seen this before and can you suggest any way to suppress or remove?

seanpdoyle commented 8 years ago

@dchersey I have not seen this particular error before, but I have a hunch that it has to do with serving from a multi-process server.

Could you try configuring Puma to use a single worker and a single thread in development?

dchersey commented 8 years ago

I was configured that way:

[53785] Puma starting in cluster mode...
[53785] * Version 3.1.0 (ruby 2.3.0-p0), codename: El Niño Winter Wonderland
[53785] * Min threads: 1, max threads: 1
[53785] * Environment: development
[53785] * Process workers: 1
[53785] * Phased restart available
[53785] * Listening on tcp://localhost:3000
[53785] Use Ctrl-C to stop
[53785] - Worker 0 (pid: 53814) booted, phase: 0

however I found that when I deleted the frontend/tmp directory the majority of the errors went away. They have not come back yet, even though the frontend build ran again.

I am getting only one of them, from somewhere in the rails 5 gem:

        Directory: /Users/david/projects/ia/vendor/ruby/2.3.0/bundler/gems/rails-dbfa8fdfc29e/activerecord/test/fixtures/all/admin

        is already being watched through: /Users/david/projects/ia/vendor/ruby/2.3.0/bundler/gems/rails-dbfa8fdfc29e/activerecord/test/fixtures/to_be_linked

        MORE INFO: https://github.com/guard/listen/wiki/Duplicate-directory-errors

I'm going to keep watching this one and let you know if I can isolate it further.

seanpdoyle commented 8 years ago

Thanks!

Our Rails 5 support is still in the works, so please share anything you find!

seanpdoyle commented 8 years ago

Closing this issue for now.

@dchersey please feel free to share any of your findings, and let me know if you think this issue should be reopened.

phortx commented 7 years ago

This error still occurs. I can't reproduce it reliable but it seems if the server runs for some minutes while you're changing code and then run another rails process like rake or rspec, the new process will print these errors.

seanpdoyle commented 7 years ago

Thanks @phortx, I'll reopen this.

anhari commented 7 years ago

Just wanted to chime in and say that I get flooded with similar errors when running rails generators or migrations, i.e.:

rails g model Comment body:text

** ERROR: directory is already being watched! **

Directory: /Users/anhari/code/ember.js/bob_loblaws_law_blog/frontend/tmp/broccoli_merge_trees-input_base_path-3Zhg6yH9.tmp

is already being watched through: /Users/anhari/code/ember.js/bob_loblaws_law_blog/frontend/tmp/babel-output_path-gYS6oRw7.tmp

MORE INFO: https://github.com/guard/listen/wiki/Duplicate-directory-errors
** ERROR: directory is already being watched! **

Directory: /Users/anhari/code/ember.js/bob_loblaws_law_blog/frontend/tmp/broccoli_merge_trees-input_base_path-95VMVFHk.tmp

is already being watched through: /Users/anhari/code/ember.js/bob_loblaws_law_blog/frontend/tmp/babel-output_path-7WeWFjtt.tmp

MORE INFO: https://github.com/guard/listen/wiki/Duplicate-directory-errors

...

phortx commented 7 years ago

This is really annoying - Do you think, you could take a look at it soon?

Thank you very much!

phortx commented 7 years ago

Some interesting information: If I stop the server and restart again, these errors pop up too.

If I do a ps aux | grep -i rails I see a lot of fsevent_watch processes, despite the server isn't running.

seanpdoyle commented 7 years ago

Could you try specifying listen ~> 2.7 in your application's Gemfile?

phortx commented 7 years ago

I've tried that and run bundle update listen. If I start the server (unicorn), I'll get following error:

I, [2017-01-13T20:31:44.708426 #78716]  INFO -- : Celluloid 0.17.3 is running in BACKPORTED mode. [ http://git.io/vJf3J ]
E, [2017-01-13T20:31:46.616495 #78716] ERROR -- : wrong number of arguments (2 for 1) (ArgumentError)
/Users/benny/.rvm/gems/ruby-2.3.3/gems/celluloid-0.17.3/lib/celluloid/calls.rb:48:in `check'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/celluloid-0.17.3/lib/celluloid/calls.rb:26:in `dispatch'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/celluloid-0.17.3/lib/celluloid/call/sync.rb:16:in `dispatch'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/celluloid-0.17.3/lib/celluloid/cell.rb:50:in `block in dispatch'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/celluloid-0.17.3/lib/celluloid/cell.rb:76:in `block in task'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/celluloid-0.17.3/lib/celluloid/actor.rb:339:in `block in task'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/celluloid-0.17.3/lib/celluloid/task.rb:44:in `block in initialize'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/celluloid-0.17.3/lib/celluloid/task/fibered.rb:14:in `block in create'
(celluloid):0:in `remote procedure call'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/celluloid-0.17.3/lib/celluloid/call/sync.rb:45:in `value'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/celluloid-0.17.3/lib/celluloid/proxy/sync.rb:22:in `method_missing'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/listen-2.7.12/lib/listen/listener.rb:206:in `_init_actors'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/listen-2.7.12/lib/listen/listener.rb:87:in `block in <class:Listener>'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/celluloid-fsm-0.20.5/lib/celluloid/fsm.rb:176:in `instance_eval'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/celluloid-fsm-0.20.5/lib/celluloid/fsm.rb:176:in `call'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/celluloid-fsm-0.20.5/lib/celluloid/fsm.rb:129:in `transition_with_callbacks!'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/celluloid-fsm-0.20.5/lib/celluloid/fsm.rb:97:in `transition'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/listen-2.7.12/lib/listen/listener.rb:100:in `start'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/evented_file_update_checker.rb:90:in `boot!'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/evented_file_update_checker.rb:61:in `initialize'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/i18n_railtie.rb:59:in `new'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/i18n_railtie.rb:59:in `initialize_i18n'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/i18n_railtie.rb:15:in `block in <class:Railtie>'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/lazy_load_hooks.rb:41:in `execute_hook'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/lazy_load_hooks.rb:50:in `block in run_load_hooks'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/lazy_load_hooks.rb:49:in `each'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/lazy_load_hooks.rb:49:in `run_load_hooks'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/railties-5.0.1/lib/rails/application/finisher.rb:65:in `block in <module:Finisher>'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/railties-5.0.1/lib/rails/initializable.rb:30:in `instance_exec'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/railties-5.0.1/lib/rails/initializable.rb:30:in `run'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/railties-5.0.1/lib/rails/initializable.rb:55:in `block in run_initializers'
/Users/benny/.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/tsort.rb:228:in `block in tsort_each'
/Users/benny/.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/tsort.rb:350:in `block (2 levels) in each_strongly_connected_component'
/Users/benny/.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/tsort.rb:431:in `each_strongly_connected_component_from'
/Users/benny/.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/tsort.rb:349:in `block in each_strongly_connected_component'
/Users/benny/.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/tsort.rb:347:in `each'
/Users/benny/.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/tsort.rb:347:in `call'
/Users/benny/.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/tsort.rb:347:in `each_strongly_connected_component'
/Users/benny/.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/tsort.rb:226:in `tsort_each'
/Users/benny/.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/tsort.rb:205:in `tsort_each'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/railties-5.0.1/lib/rails/initializable.rb:54:in `run_initializers'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/railties-5.0.1/lib/rails/application.rb:352:in `initialize!'
/Users/benny/workspaces/eoceo/smiledoctor/core/config/environment.rb:5:in `<top (required)>'
config.ru:4:in `require_relative'
config.ru:4:in `block in <main>'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/rack-2.0.1/lib/rack/builder.rb:55:in `instance_eval'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/rack-2.0.1/lib/rack/builder.rb:55:in `initialize'
config.ru:1:in `new'
config.ru:1:in `<main>'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/unicorn-5.2.0/lib/unicorn.rb:56:in `eval'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/unicorn-5.2.0/lib/unicorn.rb:56:in `block in builder'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/unicorn-5.2.0/bin/unicorn_rails:139:in `block in rails_builder'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/unicorn-5.2.0/lib/unicorn/http_server.rb:752:in `build_app!'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/unicorn-5.2.0/lib/unicorn/http_server.rb:617:in `init_worker_process'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/unicorn-5.2.0/lib/unicorn/http_server.rb:639:in `worker_loop'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/unicorn-5.2.0/lib/unicorn/http_server.rb:508:in `spawn_missing_workers'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/unicorn-5.2.0/lib/unicorn/http_server.rb:132:in `start'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/unicorn-5.2.0/bin/unicorn_rails:209:in `<top (required)>'
/Users/benny/.rvm/gems/ruby-2.3.3/bin/unicorn_rails:22:in `load'
/Users/benny/.rvm/gems/ruby-2.3.3/bin/unicorn_rails:22:in `<top (required)>'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/ruby-debug-ide-0.6.1.beta3/lib/ruby-debug-ide.rb:91:in `debug_load'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/ruby-debug-ide-0.6.1.beta3/lib/ruby-debug-ide.rb:91:in `debug_program'
/Users/benny/.rvm/gems/ruby-2.3.3/gems/ruby-debug-ide-0.6.1.beta3/bin/rdebug-ide:157:in `<top (required)>'
-e:1:in `load'
-e:1:in `<main>'

Looks like an incompatibility issue.

seanpdoyle commented 7 years ago

Could you share the version of the listen gem you're running?

$ cat Gemfile.lock | grep listen

Would you be interested in creating a GitHub repo for a new Rails project with the minimal set of changes to exhibit this behavior?

phortx commented 7 years ago

Sure! Sorry for didn't that in the first round ...

cat Gemfile.lock | grep listen:

      listen (>= 2.7, < 4.0)
    listen (2.7.12)
    spring-watcher-listen (2.0.1)
      listen (>= 2.7, < 4.0)
  listen (~> 2.7.12)
  spring-watcher-listen (~> 2.0.0)

bundle show listen:

/Users/benny/.rvm/gems/ruby-2.3.3/gems/listen-2.7.12

phortx commented 7 years ago

I tried out some things and found out the following:

The easiest way to reproduce this error, is to start the webserver and then start a rails console (by running rails c). That will flood the console with those errors
That doesn't happen with webrick (rails s)

seanpdoyle commented 7 years ago

@phortx it looks like your Gemfile.lock declares listen (~> 2.7.12).

Is that what you're running in production, or was that changed because of my feedback?

According to https://github.com/thoughtbot/ember-cli-rails/issues/429#issuecomment-272526880, this caused issues.

seanpdoyle commented 7 years ago

That doesn't happen with webrick (rails s)

Interesting. What server command does this happen with?

phortx commented 7 years ago

I've added that based on your request.

Oh sorry, "that error" isn't very clear, if we have two errors ;)

My comment referred to the original error (directory already being watched)
Setting the listen gem version to ~> 2.7.12 and starting the webserver (unicorn or webrick) will cause the wrong number of arguments error described above.
I'm running the server with bundle exec unicorn (and doing a request, to boot up the worker)

Sorry for the confusion.

seanpdoyle commented 7 years ago

Setting the listen gem version to ~> 2.7.12

What version of listen are you using that raises the original (not this one) issue?

phortx commented 7 years ago

bundle show listen: /Users/benny/.rvm/gems/ruby-2.3.3/gems/listen-3.1.5

seanpdoyle commented 7 years ago

I'm having issues reproducing this in the example application.

I suspected that the listen gem was a transitive dependency of the application server.

Unfortunately, even after including both puma and unicorn in the Gemfile, the project still doesn't depend on the listen gem.

@phortx and @dchersey are you both using spring?

As a temporary fix, could you try setting DISABLE_SPRING=1 in your shell environment?

phortx commented 7 years ago

bundle show spring: Could not find gem 'spring'.

However, I've tried to reproduce with DISABLE_SPRING=1, same error (directory already being watched).

I'll try to send a PR for the example app to enable us to reproduce the issue.

seanpdoyle commented 7 years ago

@phortx according to https://github.com/thoughtbot/ember-cli-rails/issues/429#issuecomment-272540877, listen is included as a dependency because spring-watcher-listen depends on it.

Are you including listen directly?

phortx commented 7 years ago

I had spring in my Gemfile, but I've removed it, so cat Gemfile.lock | grep listen meanwhile produces:

      listen (>= 2.7, < 4.0)
    listen (3.1.5)
  listen

That doesn't fix the already watched error.

We have the listen gem in the Gemfile, without any specified version. I can add a version (gem 'listen', '~> 2.7.12'), but causes the wrong number of arguments.

I can remove the listen gem and run bundle update. But that doesn't fix the already watched error.

seanpdoyle commented 7 years ago

I can remove the listen gem and run bundle update. But that doesn't fix the already watched error.

That's very surprising, since the original issue includes a call out to guard/listen:

MORE INFO: https://github.com/guard/listen/wiki/Duplicate-directory-errors

phortx commented 7 years ago

guard requires listen.

If I remove guard and start unicorn or webrick, I get:

/Users/benny/.rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/dependencies.rb:293:in `require': Could not load the 'listen' gem. Add `gem 'listen'` to the development group of your Gemfile (LoadError)

Which seems like rails needs the listen gem? I'm so confused :D

dchersey commented 7 years ago

Hi @seanpdoyle, I'm not sure it's relevant but I am also using Spring. This issue is sporadic for me; if I clean the build dirs it goes away, only to resurface after several incremental builds.

I am in the midst of updating all the gems and ember after a 6 month hiatus so I am not able to retest right now but I appreciate the help in isolating this!

phortx commented 7 years ago

I'm even more confused. I've changed the example app more and more until it nearly looked like our app, I've even copied the whole frontend/ dir (the only differences are some entities in app/, the db/ dir and the spec/ dir) ... but I can't reproduce the error in the example application ...

phortx commented 7 years ago

I got it! Finally ... We have the following setting in our config/environments/development.rb:

config.file_watcher = ActiveSupport::EventedFileUpdateChecker

This is default when you create a new rails 5 application, you can check that file after running rails new asdfasdf.

Removing that line, solve the errors.

tricknotes / ember-cli-rails

Receiving directory already being watched error for ember compiled assets #429