Research using a messaging server for RD integration

[hortongn]

Descriptive summary

Research the pros, cons, costs, and feasibility of using a messaging server for our integration with Research Directory.

Example solutions:

• RabbitMQ
• ActiveMQ

This also depends on IT@UC's ability to implement a server.

[hortongn]

Some resources:

https://medium.com/@igkuz/ruby-retry-scheduled-tasks-with-dead-letter-exchange-in-rabbitmq-9e38aa39089b

https://karolgalanciak.com/blog/2019/02/24/messages-on-rails-part-1-introduction-to-kafka-and-rabbitmq/

https://karolgalanciak.com/blog/2019/04/07/messages-on-rails-part-2-kafka/

https://karolgalanciak.com/blog/2019/06/23/messages-on-rails-part-3-rabbitmq/

https://blog.kontena.io/event-driven-microservices-with-rabbitmq-and-ruby/

https://github.com/ruby-amqp/bunny

[bsp3ars]

Research (5/16) - RabbitMQ (https://www.rabbitmq.com/):

• Open source (https://github.com/rabbitmq/rabbitmq-server)
• Smart broker/dumb consumer
• Good documentation (especially for rails)
• Well supported in Ruby
• Supports multiple protocols
• Free

ActiveMQ (https://activemq.apache.org/):

• Open source (https://github.com/apache/activemq) (issues @ http://issues.apache.org/jira/browse/AMQ)
• Smart broker/dumb consumer
• Seems to only allow STOMP (https://activemq.apache.org/cross-language-clients)
• Free

Research (5/17) - Apache Kafka (https://kafka.apache.org/):

• Open source (https://github.com/apache/kafka) (issues @ https://issues.apache.org/jira/browse/KAFKA)
• Dumb broker/smart consumer
• Different type of messaging server, potentially could hold messages indefinitely
• Potential of 1 million messages per second
• Meant for simple setups (no complex routing)
• Guide to use in Rails (https://karolgalanciak.com/blog/2019/04/07/messages-on-rails-part-2-kafka/)
• Free