keylimetoolbox / resque-kubernetes

Run Resque (and ActiveJob) workers as Kubernetes Jobs and autoscale from 0!
MIT License
54 stars 15 forks source link

Fix `ContextFactory.context` function: it should not return an Array when all the contexts are not `applicable?` #23

Closed guizmaii closed 5 years ago

guizmaii commented 5 years ago

Hi @jeremywadsack ,

In my test suite, I have the following error:

undefined method `namespace' for #<Array:0x000000000a5f7648>
shared/bundle/ruby/2.3.0/gems/resque-kubernetes-1.3.0/lib/resque/kubernetes/jobs_manager.rb:73:in `build_client'
shared/bundle/ruby/2.3.0/gems/resque-kubernetes-1.3.0/lib/resque/kubernetes/jobs_manager.rb:66:in `client'
shared/bundle/ruby/2.3.0/gems/resque-kubernetes-1.3.0/lib/resque/kubernetes/jobs_manager.rb:57:in `jobs_client'
shared/bundle/ruby/2.3.0/gems/resque-kubernetes-1.3.0/lib/resque/kubernetes/jobs_manager.rb:79:in `finished_jobs'
shared/bundle/ruby/2.3.0/gems/resque-kubernetes-1.3.0/lib/resque/kubernetes/jobs_manager.rb:22:in `reap_finished_jobs'
bundle/ruby/2.3.0/gems/resque-kubernetes-1.3.0/lib/resque/kubernetes/job.rb:91:in `before_enqueue_kubernetes_job'
jeremywadsack commented 5 years ago

@guizmaii, I'm interested in knowing more about different contexts for authentication. Is this failing just because your test environment is not authenticated in anyway? (And if so, doesn't that lead to further failures because jobs_client is now nil?)

guizmaii commented 5 years ago

@jeremywadsack With this new version, now I have the following error: undefined method 'get_jobs' for nil:NilClass

guizmaii commented 5 years ago

Is this failing just because your test environment is not authenticated in anyway?

Yes it's not authenticated.

guizmaii commented 5 years ago

Ok I understand what's happening.

I configured this gem as follow:

Resque::Kubernetes.configuration do |config|
  config.environments = [ Rails.env ]
end

and it's not a good idea because if I only put ["production", "staging"], for example, the before_enqueue_kubernetes_job will return early enough in my test env to not generate these kind of problems because of this code:

# A before_enqueue hook that adds worker jobs to the cluster.
def before_enqueue_kubernetes_job(*_args)
  if defined? Rails
    return unless Resque::Kubernetes.environments.include?(Rails.env)
  end

   ...
end

Thanks for the quick answer and for your time. :)

jeremywadsack commented 5 years ago

Correct.

The reason for having config.environments was to have it explicitly exclude launching workers in test environment. You likely do not want to be changing any clusters when running tests (unless you have a specific staging environment that you would do that in.) Similarly, you probably don't want to launch workers in a cluster during development. I would suggest just production and staging as you have stated.

Resque::Kubernetes.configuration do |config|
  config.environments = %w[production staging]
end