search

mozilla / jira-bugzilla-integration

Jira Bugzilla Integration (JBI) - system to sync bugs and issues
Mozilla Public License 2.0
8 stars 22 forks source link

Add Dead Letter Queue class, integrate into heartbeat #918

Closed grahamalama closed 3 months ago

grahamalama commented 4 months ago

Built on the groundwork of #903 (cheers @leplatrem, when I squash this I'll make sure you're a coauthor), this PR adds the classes that allow us to manage our Dead Letter Queue.

We included a basic heartbeat check in this PR to assert that we are able to read / write from the queue. I removed the integration with the app (in the /bugzilla_webhook route) for now. The idea is that we'll deploy this change first to validate that we can connect to the directory we're mounting on the infra side. Then, we'll follow on with how we integrate it into the /bugzilla_webhook route.

grahamalama commented 4 months ago

One more big though I'm having about the architecture of this PR:

We've been calling this a (singular) queue. But almost feels like we're building a queue manager. Like to me, what we return from get_all() is not a queue, but a collection of queues. And each bug has its own queue of failing messages. Thoughts?

alexcottner commented 4 months ago

We've been calling this a (singular) queue. But almost feels like we're building a queue manager. Like to me, what we return from get_all() is not a queue, but a collection of queues. And each bug has its own queue of failing messages. Thoughts?

In my head, this is a very simple partitioned queue. We have a single queue where things are going, but blocking only occurs within the partition (per bug). These terms would be more well defined if we were using an actual queue service with partitions and compute nodes.