Helm allow scaling sidekiq Deployment

[deepy]

Pitch

The current Helm setup creates one Deployment for sidekiq which handles all queues.

We could add something like this to values.yaml to allow scaling the queues separately.

  sidekiq:
    concurrency: 25
    workers:
    - name: pullpush
      count: 2
      queues: ["push,6", "pull"]
    - name: mailers
      count: 1
      queues: ["mailers"]

Each worker would correspond to one Deployment which would handle just the specified queues. The previous one worker that handles all the queues could be moved to a conditional defaulting to true.

The downside of this solution is that an administrator would have to know about all the queues to manage, but on the other hand this feature would likely only be used at the point where you know when you want to scale up. The benefit is that the chart itself doesn't have to know about the queues.

Another solution could be creating deployments scaled to 0 for each queue by default The downside here is that the chart would have to know about the queues, but on the other hand they don't change that often. Additionally an admin has no way of configuring a sidekiq that handles multiple queues. The benefit here is there's less magic and values.yaml would be a little smaller.

Motivation

The documentation is very adamant about: Make sure you only have one scheduler queue running!!

[SISheogorath]

Already implemented by #1