search

ruby-amqp / hutch

A system for processing messages from RabbitMQ.
https://gocardless.com/blog/hutch-inter-service-communication-with-rabbitmq/
MIT License
857 stars 137 forks source link

Run only some consumers #355

Closed andrykonchin closed 4 years ago

andrykonchin commented 4 years ago

I am wondering whether there is a way to run only some consumers in a Rails application.

The goal is to scale consumers independently and run certain consumers only on some certain instances.

There is a require option but it just registers a consumer again. autoload-rails isn't helpful as far as our consumers use Rails stuff.

michaelklishin commented 4 years ago

Hutch assumes that all consumers it can discover should run. Restricting discovery one way or another is what you want.

netrusov commented 4 years ago

@andrykonchin seems like --only-group option is what you are looking for. but before that you must configure consumer groups in hutch.yml:

consumer_groups:
  analytics:
    - AnalyticsConsumer
  # ... etc

I'm using this combined with systemd (docker will work great too) to run as many instances of specific consumers as I want

andrykonchin commented 4 years ago

Thank you @netrusov . Wondering why this option isn't documented and maintainers aren't aware of it.

michaelklishin commented 4 years ago

@andrykonchin Hutch is open source software, so if something isn't up to your liking, you are welcome to contribute to it, including documentation.

I maintain dozens of projects and cannot remember every single contribution to every single one of them. --only-group specifically was introduced in early 2018, so it's been a while.

Hutch could use some additional maintainers. If you'd like to become one and make it better for everyone, let us know.