Publisher bottleneck - Githubissues

https://github.com/juntossomosmais/django-outbox-pattern/blob/ea419ef369e5b3fd27a5a18b55a20ec9654c4d85/django_outbox_pattern/management/commands/publish.py#L47

Such an implementation looks like a bottleneck

published = self.published_class.objects.filter(
    status=StatusChoice.SCHEDULE, expires_at__gte=timezone.now()
)
for message in published:
    try:
        attempts = self.producer.send(message)
    except ExceededSendAttemptsException as exc:
        logger.exception(exc)
        message.retry = exc.attempts
        message.status = StatusChoice.FAILED
        message.expires_at = timezone.now() + timedelta(15)
        self.stdout.write(f"Message no published with body:\n{message.body}")
    else:
        message.retry = attempts
        message.status = StatusChoice.SUCCEEDED
        self.stdout.write(f"Message published with body:\n{message.body}")
    finally:
        message.save()

because new messages can arrive faster than once a second

def _waiting():
    sleep(1)

while the solution does not allow itself to be scaled in any way; you cannot simply launch another Publisher, otherwise the message will be sent twice

juntossomosmais / django-outbox-pattern

Publisher bottleneck #44