search

Netflix-Skunkworks / Scumblr

Web framework that allows performing periodic syncs of data sources and performing analysis on the identified results
Apache License 2.0
2.64k stars 318 forks source link

Tasks seem stuck in loop #236

Closed tDorosheff closed 6 years ago

tDorosheff commented 6 years ago

When i attempted to run "bundle exec rake run_tasks" i could see each task running, but it never actually completed. I think it was because redis-server had hung. I restarted redis, and then also scumblr and tried to re-run rake, but the UI is constantly showing 25 tasks running, even after i've rebooted Scumblr. Is there a way to clear a cache of event tasks?

sbehrens commented 6 years ago

Hi,

Navigate to yourscumblr.com/sidekiq.

Click the queues button, delete each queue. You can also click Tasks, and select Stop All.

Let me know if that helps! -Scott

tDorosheff commented 6 years ago

I dont have a queues button in /sidekiq, just Application/framework/full traces. I ended up recreating scumblr from an earlier snapshot of my host, so for now i no longer have the issue of tasks hanging. Now however when i run my rake task, i am unable to connect to redis. In the past this was because redis-server was running as a service as the redis user. I would simply stop that service, and then initiate redis with.

nohup redis-server & nohup bundle exec sidekiq -l log/sidekiq.log & nohup bundle exec rails s -b 0.0.0.0 &

Here is the full trace of my rake task:

$ bundle exec rake run_tasks --trace /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rb-readline-0.5.1/lib/readline.rb:458: warning: already initialized constant Readline::HISTORY /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rb-readline-0.5.1/lib/readline.rb:486: warning: already initialized constant Readline::FILENAME_COMPLETION_PROC /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rb-readline-0.5.1/lib/readline.rb:517: warning: already initialized constant Readline::USERNAME_COMPLETION_PROC /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rb-readline-0.5.1/lib/readline.rb:523: warning: already initialized constant Readline::VERSION [DEPRECATION]last_commentis deprecated. Please uselast_descriptioninstead. ** Invoke run_tasks (first_time) ** Invoke environment (first_time) ** Execute environment ** Execute run_tasks Running group 1 rake aborted! Redis::CannotConnectError: Error connecting to Redis on 127.0.0.1:6379 (Errno::ECONNREFUSED) /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:345:inrescue in establish_connection' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:331:in establish_connection' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:101:inblock in connect' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:293:in with_reconnect' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:100:inconnect' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:364:in ensure_connected' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:221:inblock in process' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:306:in logging' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:220:inprocess' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:187:in call_pipelined' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:157:inblock in call_pipeline' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:293:in with_reconnect' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:155:incall_pipeline' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis.rb:2299:in block in multi' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis.rb:58:inblock in synchronize' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/2.3.0/monitor.rb:214:in mon_synchronize' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis.rb:58:insynchronize' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis.rb:2291:in multi' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-status-0.6.0/lib/sidekiq-status/storage.rb:16:inblock in store_for_id' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-status-0.6.0/lib/sidekiq-status/storage.rb:102:in block in redis_connection' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:64:inblock (2 levels) in with' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:63:in handle_interrupt' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:63:inblock in with' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:60:in handle_interrupt' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:60:inwith' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-status-0.6.0/lib/sidekiq-status/storage.rb:101:in redis_connection' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-status-0.6.0/lib/sidekiq-status/storage.rb:15:instore_for_id' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-status-0.6.0/lib/sidekiq-status/client_middleware.rb:26:in call' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-4.2.3/lib/sidekiq/middleware/chain.rb:130:inblock in invoke' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-4.2.3/lib/sidekiq/middleware/chain.rb:133:in invoke' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-4.2.3/lib/sidekiq/client.rb:206:inprocess_single' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-4.2.3/lib/sidekiq/client.rb:65:in push' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-4.2.3/lib/sidekiq/worker.rb:116:inclient_push' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-4.2.3/lib/sidekiq/worker.rb:60:in perform_async' /home/ubuntu/Scumblr/lib/tasks/sync_tasks.rake:30:inblock (2 levels) in <top (required)>' /home/ubuntu/Scumblr/lib/tasks/sync_tasks.rake:27:in each' /home/ubuntu/Scumblr/lib/tasks/sync_tasks.rake:27:inblock in <top (required)>' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:248:in block in execute' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:243:ineach' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:243:in execute' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:187:inblock in invoke_with_call_chain' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/2.3.0/monitor.rb:214:in mon_synchronize' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:180:ininvoke_with_call_chain' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:173:in invoke' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/application.rb:152:ininvoke_task' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/application.rb:108:in block (2 levels) in top_level' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/application.rb:108:ineach' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/application.rb:108:in block in top_level' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/application.rb:117:inrun_with_threads' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/application.rb:102:in top_level' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/application.rb:80:inblock in run' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/application.rb:178:in standard_exception_handling' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/application.rb:77:inrun' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/exe/rake:27:in <top (required)>' /home/ubuntu/.rbenv/versions/2.3.1/bin/rake:23:inload' /home/ubuntu/.rbenv/versions/2.3.1/bin/rake:23:in <top (required)>' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/cli/exec.rb:74:inload' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/cli/exec.rb:74:in kernel_load' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/cli/exec.rb:27:inrun' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/cli.rb:360:in exec' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/vendor/thor/lib/thor/command.rb:27:inrun' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in invoke_command' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/vendor/thor/lib/thor.rb:369:indispatch' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/cli.rb:20:in dispatch' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/vendor/thor/lib/thor/base.rb:444:instart' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/cli.rb:10:in start' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/exe/bundle:35:inblock in <top (required)>' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/friendly_errors.rb:121:in with_friendly_errors' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/exe/bundle:27:in<top (required)>' /home/ubuntu/.rbenv/versions/2.3.1/bin/bundle:23:in load' /home/ubuntu/.rbenv/versions/2.3.1/bin/bundle:23:in

' Errno::ECONNREFUSED: Connection refused - connect(2) for 127.0.0.1:6379 /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/2.3.0/socket.rb:1207:in __connect_nonblock' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/2.3.0/socket.rb:1207:inconnect_nonblock' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/connection/ruby.rb:189:in rescue in connect_addrinfo' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/connection/ruby.rb:181:inconnect_addrinfo' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/connection/ruby.rb:222:in block in connect' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/connection/ruby.rb:220:ineach' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/connection/ruby.rb:220:in each_with_index' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/connection/ruby.rb:220:inconnect' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/connection/ruby.rb:295:in connect' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:336:inestablish_connection' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:101:in block in connect' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:293:inwith_reconnect' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:100:in connect' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:364:inensure_connected' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:221:in block in process' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:306:inlogging' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:220:in process' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:187:incall_pipelined' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:157:in block in call_pipeline' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:293:inwith_reconnect' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:155:in call_pipeline' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis.rb:2299:inblock in multi' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis.rb:58:in block in synchronize' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/2.3.0/monitor.rb:214:inmon_synchronize' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis.rb:58:in synchronize' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis.rb:2291:inmulti' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-status-0.6.0/lib/sidekiq-status/storage.rb:16:in block in store_for_id' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-status-0.6.0/lib/sidekiq-status/storage.rb:102:inblock in redis_connection' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:64:in block (2 levels) in with' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:63:inhandle_interrupt' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:63:in block in with' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:60:inhandle_interrupt' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:60:in with' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-status-0.6.0/lib/sidekiq-status/storage.rb:101:inredis_connection' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-status-0.6.0/lib/sidekiq-status/storage.rb:15:in store_for_id' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-status-0.6.0/lib/sidekiq-status/client_middleware.rb:26:incall' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-4.2.3/lib/sidekiq/middleware/chain.rb:130:in block in invoke' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-4.2.3/lib/sidekiq/middleware/chain.rb:133:ininvoke' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-4.2.3/lib/sidekiq/client.rb:206:in process_single' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-4.2.3/lib/sidekiq/client.rb:65:inpush' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-4.2.3/lib/sidekiq/worker.rb:116:in client_push' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-4.2.3/lib/sidekiq/worker.rb:60:inperform_async' /home/ubuntu/Scumblr/lib/tasks/sync_tasks.rake:30:in block (2 levels) in <top (required)>' /home/ubuntu/Scumblr/lib/tasks/sync_tasks.rake:27:ineach' /home/ubuntu/Scumblr/lib/tasks/sync_tasks.rake:27:in block in <top (required)>' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:248:inblock in execute' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:243:in each' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:243:inexecute' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:187:in block in invoke_with_call_chain' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/2.3.0/monitor.rb:214:inmon_synchronize' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:180:in invoke_with_call_chain' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:173:ininvoke' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/application.rb:152:in invoke_task' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/application.rb:108:inblock (2 levels) in top_level' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/application.rb:108:in each' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/application.rb:108:inblock in top_level' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/application.rb:117:in run_with_threads' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/application.rb:102:intop_level' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/application.rb:80:in block in run' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/application.rb:178:instandard_exception_handling' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/application.rb:77:in run' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/exe/rake:27:in<top (required)>' /home/ubuntu/.rbenv/versions/2.3.1/bin/rake:23:in load' /home/ubuntu/.rbenv/versions/2.3.1/bin/rake:23:in<top (required)>' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/cli/exec.rb:74:in load' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/cli/exec.rb:74:inkernel_load' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/cli/exec.rb:27:in run' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/cli.rb:360:inexec' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/vendor/thor/lib/thor/command.rb:27:in run' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:ininvoke_command' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/vendor/thor/lib/thor.rb:369:in dispatch' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/cli.rb:20:indispatch' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/vendor/thor/lib/thor/base.rb:444:in start' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/cli.rb:10:instart' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/exe/bundle:35:in block in <top (required)>' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/friendly_errors.rb:121:inwith_friendly_errors' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/exe/bundle:27:in <top (required)>' /home/ubuntu/.rbenv/versions/2.3.1/bin/bundle:23:inload' /home/ubuntu/.rbenv/versions/2.3.1/bin/bundle:23:in <main>' IO::EINPROGRESSWaitWritable: Operation now in progress - connect(2) would block /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/2.3.0/socket.rb:1207:in__connect_nonblock' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/2.3.0/socket.rb:1207:in connect_nonblock' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/connection/ruby.rb:182:inconnect_addrinfo' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/connection/ruby.rb:222:in block in connect' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/connection/ruby.rb:220:ineach' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/connection/ruby.rb:220:in each_with_index' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/connection/ruby.rb:220:inconnect' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/connection/ruby.rb:295:in connect' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:336:inestablish_connection' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:101:in block in connect' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:293:inwith_reconnect' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:100:in connect' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:364:inensure_connected' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:221:in block in process' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:306:inlogging' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:220:in process' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:187:incall_pipelined' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:157:in block in call_pipeline' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:293:inwith_reconnect' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis/client.rb:155:in call_pipeline' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis.rb:2299:inblock in multi' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis.rb:58:in block in synchronize' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/2.3.0/monitor.rb:214:inmon_synchronize' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis.rb:58:in synchronize' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/redis-3.3.1/lib/redis.rb:2291:inmulti' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-status-0.6.0/lib/sidekiq-status/storage.rb:16:in block in store_for_id' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-status-0.6.0/lib/sidekiq-status/storage.rb:102:inblock in redis_connection' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:64:in block (2 levels) in with' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:63:inhandle_interrupt' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:63:in block in with' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:60:inhandle_interrupt' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:60:in with' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-status-0.6.0/lib/sidekiq-status/storage.rb:101:inredis_connection' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-status-0.6.0/lib/sidekiq-status/storage.rb:15:in store_for_id' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-status-0.6.0/lib/sidekiq-status/client_middleware.rb:26:incall' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-4.2.3/lib/sidekiq/middleware/chain.rb:130:in block in invoke' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-4.2.3/lib/sidekiq/middleware/chain.rb:133:ininvoke' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-4.2.3/lib/sidekiq/client.rb:206:in process_single' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-4.2.3/lib/sidekiq/client.rb:65:inpush' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-4.2.3/lib/sidekiq/worker.rb:116:in client_push' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/sidekiq-4.2.3/lib/sidekiq/worker.rb:60:inperform_async' /home/ubuntu/Scumblr/lib/tasks/sync_tasks.rake:30:in block (2 levels) in <top (required)>' /home/ubuntu/Scumblr/lib/tasks/sync_tasks.rake:27:ineach' /home/ubuntu/Scumblr/lib/tasks/sync_tasks.rake:27:in block in <top (required)>' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:248:inblock in execute' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:243:in each' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:243:inexecute' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:187:in block in invoke_with_call_chain' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/2.3.0/monitor.rb:214:inmon_synchronize' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:180:in invoke_with_call_chain' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:173:ininvoke' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/application.rb:152:in invoke_task' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/application.rb:108:inblock (2 levels) in top_level' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/application.rb:108:in each' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/application.rb:108:inblock in top_level' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/application.rb:117:in run_with_threads' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/application.rb:102:intop_level' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/application.rb:80:in block in run' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/application.rb:178:instandard_exception_handling' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/application.rb:77:in run' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rake-11.3.0/exe/rake:27:in<top (required)>' /home/ubuntu/.rbenv/versions/2.3.1/bin/rake:23:in load' /home/ubuntu/.rbenv/versions/2.3.1/bin/rake:23:in<top (required)>' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/cli/exec.rb:74:in load' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/cli/exec.rb:74:inkernel_load' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/cli/exec.rb:27:in run' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/cli.rb:360:inexec' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/vendor/thor/lib/thor/command.rb:27:in run' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:ininvoke_command' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/vendor/thor/lib/thor.rb:369:in dispatch' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/cli.rb:20:indispatch' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/vendor/thor/lib/thor/base.rb:444:in start' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/cli.rb:10:instart' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/exe/bundle:35:in block in <top (required)>' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/lib/bundler/friendly_errors.rb:121:inwith_friendly_errors' /home/ubuntu/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/bundler-1.15.1/exe/bundle:27:in <top (required)>' /home/ubuntu/.rbenv/versions/2.3.1/bin/bundle:23:inload' /home/ubuntu/.rbenv/versions/2.3.1/bin/bundle:23:in <main>' Tasks: TOP => run_tasks

Thanks Scott!

sbehrens commented 6 years ago

That error makes me think redis server isn't running or your scumblr config file is pointing to the wrong redis host. If you cant visit "your_scumblr_site.com/sidekiq" you must be on a very old version of the code. Can you share over what version you are using?

tDorosheff commented 6 years ago

Its been forever since i stood up Scumblr on this box. How can i go about getting a version for you?

tDorosheff commented 6 years ago

so i just tried going to /sidekiq again and it looks exactly how you described and completely different from the last time i checked the other day. i was able to see the queues and there were currently no tasks in queue. So i think the issue with that was like you said, perhaps redis was not currently running, though i could see it's PID. idk... PEBKAC

Anyway, as of now everything seems to be working correctly. Would still like to determine what version i am running though.

sbehrens commented 6 years ago

Upgrading is pretty easy if you are on the really old release. Check out the Wiki guide.