DB's attempt to create a duplicate event times out the sqs poller rake task and re-queues the same message

[christinach]

Expected behavior

The enqueued message should be consumed by bibdata and get deleted.

Actual behavior

Bibdata database attempts to create a duplicate event. As a result and because of the constraints we have in Event and Dump it raises an error and aborts the sqs_poller rake task RAILS_ENV={{rails_app_env}} /usr/local/bin/bundle exec rake bibdata:indexing:poll_sqs. Then the polling times out and requeues the same message. This is happening until it reaches 1000 counts (MaxReceiveCount:1000) and then it's moved to the SQSDeadLetterQueue. 1000 counts take around 9-10 hours. The rest of the incremental messages pile up and we don't see events/dumps in Bibdata only after 9-10 hours.

Steps to replicate

Impact of this bug

Records delay to index. Bibdata raises too many errors.

Honeybadger link and code snippet, if applicable

https://app.honeybadger.io/projects/54497/faults/110646756

Validation failed: Message body has already been taken message_body: {"id":"39235082590006421"

https://app.honeybadger.io/projects/54497/faults/110272324

ActiveRecord::RecordInvalid: Validation failed: Message body has already been taken

[christinach]

Resolved by:

Deleted the message that was enqueued by the attempted duplicate event

Set a lower number of MaxReceiveCount