Closed zabeen closed 10 months ago
Extra dev notes from discussions with @disaev-nmdp : Q: should I get inserted/failed donors count from DonorImportHistory table? Or just simple message with WasSuccessful: false without any numbers?
Answer: I am not sure how useful the counts are for unsuccessful imports... but as we do import from file in batches, it's possible some updates did get processed before a problem occurs.
I think it would be useful to add a new prop, enum ImportFailureReason, with options of RequestDeadlettered and ErrorDuringImport (this can be a catch all for any of the exceptions that can be thrown during import). We could add a new prop to hold failure info, like we do for matching and search requests, in it we have the FailureReason enum and a string with the exception message.
@zabeen Testing status: Ok
Feature does not seem to be available yet in UAT or live. Could you please let me know when it is? I cannot test until then.
Feature does not seem to be available yet in UAT or live. Could you please let me know when it is? I cannot test until then.
Yes I'll let you know, hopefully soon! :)
@mmelchers The feature has been released to UAT-WMDA-ATLAS.
On completion of donor import, a message is currently sent to either
notifications
topic (on success) oralerts
topic (for failures) - the target of these messages are the support team.New topic
Need to add a new service bus topic,
donor-import-results
, that can notify the consumer service (i.e., search and match) that a donor import has completed, and its status.Message properties:
FileName
- name of the donor import fileWasSuccessful
- result of the uploadImportedDonorCount
- total number of donors from file that were successfully importedFailedDonorCount
- total number of donors that failed to be imported from the fileNew topic should be terraformed, with an
audit
subscription.Redirect dead-lettered requests
Further, add a new dead-letter queue listener function to forward dead-lettered import requests to the new results topic, with
"WasSuccessful": false
Stop success notifications
Once the new topic is available, success messages sent to the
notifications
topic will become redundant and should no longer be sent out. However, message toalerts
topic on failure should remain.Switching off success notifications should be safe and non-breaking. They are already switched off in AN live, as too many messages are published due to the frequency of updates.
In addition to the code that sends out the success notification, there is an app setting set by a terraform release var that toggles publishing of these notifications - the app setting and var should be removed.
Will raise a new ticket if further failure details are required in this message, for now, all failures are logged to AI, so a support agent can determine the cause of failure that way.