search

OpenFn / primero-progres

Interoperability solution for UNICEF <> UNHCR interagency referrals between Primero and Progres v4 systems.
https://openfn.github.io/primero-progres/
0 stars 2 forks source link

Data security feature: delete Messages from Inbox after processed #46

Closed aleksa-krolls closed 2 years ago

aleksa-krolls commented 2 years ago

Background, context, and business value

To satisfy data security & privacy requirements, UNICEF requests that we do not keep data sent to OpenFn stored on our servers. However, for this implementation, UNHCR's system Progres sends receipts via HTTP POST requests to the OpenFn inbox to trigger different job flows. Therefore, they'd like to request deletion of these receipts once triggered runs have been completed.

The specific request, in as few words as possible

Implement automation to:

  1. Delete messages from Inbox after all related runs* have been completed (no matter if these runs fail or succeed). This assumes we will rely on UNHCR to re-send payloads if we need to retry a transactions.
  2. Delete messages from Inbox where no runs are triggered. (If data is sent to OpenFn inbox, and no runs are triggered - i.e., the job is "off", then automatically delete the Message to ensure security.)

For any runs completed, these should not be deleted (even if the parent Message is) for 365 days and remain accessible via Activity History.

*Note that this project has multiple flow jobs triggered by messages received from Proges.

Resources

See drafted project docs: https://github.com/OpenFn/primero-progres OpenFn project*: https://www.openfn.org/projects/primero-progres-v4-prod/jobs

Testing Notes

They don't want to turn "on" this feature until the official go-live and plan to do some production testing before then (during which it will be helpful to keep things as is so we can inspect messages as needed for troubleshooting). Therefore, is there a way to toggle on/off this feature? Otherwise, how do you suggest we test/in what environment?

aleksa-krolls commented 2 years ago

@taylordowns2000 fyi - the client will be ready on Wednesday (Aug 24th) to implement this feature. Can you pls block time on your calendar accordingly to implement? Go-live will be the day after on the 25th.

cc @daissatou2

taylordowns2000 commented 2 years ago

@aleksa-krolls , this is in your court for testing!

aleksa-krolls commented 2 years ago

@taylordowns2000 tested and working! And documented in the Project Security Configuration Checklist. Thank you!