Ensure Redis buffer queues are signed correctly based on the new keys

[sastels]

Description

As a Notify dev, I need to be able to ensure that the notifications buffered in redis are correctly signed

WHY are we building?

When we rotate signing secrets we want to be able to verify that the redis buffers can still be read.

WHAT are we building?

A script to verify the notifications in the redis inboxes and inflights

VALUE created by our solution

Assurance that we are not going to lose notifications we've accepted

Acceptance Criteria

Given some context, when (X) action occurs, then (Y) outcome is achieved.

• [ ] Write a script to verify the notifications in our inboxes and inflights

QA Steps

• [ ] run the scripts against staging or production

[sastels]

https://github.com/cds-snc/notification-attic/pull/32

[jimleroyer]

I will test today.

[sastels]

I'll see if I can quickly change to grabbing a few at a time rather than the entire inbox at once.

[sastels]

changed to reading in chunks, and added a few flags, see https://github.com/cds-snc/notification-attic/pull/32

[jimleroyer]

I tested the script in staging and unfortunately, it resigned some keys but the wrongly signed notifications went through to the db save celery task where it's too late in the data pipeline to resign and get these processed correctly. The logic then produces errors on these badly signed notifications. This probably require a better testing plan maybe from my part; will sync with @sastels .

[jimleroyer]

Opened a new PR with updated scripts to use the inflight queues instead. Also added a preview mode. https://github.com/cds-snc/notification-attic/pull/34

[ben851]

@sastels to review