search

samsondav / rihanna

Rihanna is a high performance postgres-backed job queue for Elixir
MIT License
439 stars 47 forks source link

Detail use with multiple dispatchers #39

Closed lpil closed 5 years ago

lpil commented 5 years ago

Rihanna requires 1 + N database connections per node, where 1 connection is used for the external API of enqueuing/retrying jobs and N is the number of dispatchers.

In the default configuration of one dispatcher per node, Rihanna will use 2 database connections per node.

This seems to imply that the number of dispatcher can be configured, but Rihanna is hard coded to only start 1 dispatcher and 1 Postgrex, so there is always 2.

Coupled with the fact that both processes have a hard-coded name it's not possible to have more instances of the tree as the name is already taken.

It seems that the job dispatcher name is never actually used and can be removed: https://github.com/samphilipd/rihanna/search?utf8=%E2%9C%93&q=Rihanna.JobDispatcher&type=

The Postgrex connection could be not started (see https://github.com/samphilipd/rihanna/issues/38), and then the user can add as many Rihanna subtrees to their application as needed :)

samsondav commented 5 years ago

I mean, you can run the dispatchers yourself... We do by spinning up Rihanna.JobDispatcher manually in our application supervision tree. So it's quite OK to have more than one.

The Rihanna.Supervisor is really just for convenience and suits what I imagine is the most common use case.

Admittedly this could probably be better documented.

lpil commented 5 years ago

Ah yes, because the name is passed in by the supervisor.

Documenting this sounds good. :)

lpil commented 5 years ago

I'll write something up for this later.

lpil commented 5 years ago

I've given this some thought and I think it might be a good idea not to encourage to design their own supervisor tree.

If users of Rihanna always use our supervisor then we have control over the structure of the tree, is required for features such as #18.