Implement Permanent Removal of Resources (Purging)

[alexanderkiel]

As Blaze implements versioning of resources, the delete interaction only marks resources as deleted. In some scenarios, for example for GDPR compliance, it might be necessary to really remove a resource from Blaze.

Other Work

• HAPI has the $expunge operation
• Firely has the $erase operation
• Google Cloud Healthcare API has $purge
• TXDB hat the evict operation
• Datomic has Excision
• EventStoreDB has hard-delete and Scavenging

FHIR Standard

• FHIR-16198

GDPR

• data has to be erased within 30 days

Plan

We like to implement:

Encryption of Paging Links

1995

Delete History

FHIR Spec #1382

DELETE [base]/[type]/[id]/_history - remove all versions of the resource except the current version (which if the resource has been deleted, will be an empty placeholder)

Delete History Version

FHIR Spec

DELETE [base]/[type]/[id]/_history/[vid] - remove the specified version of the resource. It is an error to remove the 'current' version. (Must first perform a regular delete, and can then delete the non-current version.)

Patient Purge

FHIR Spec #1298

POST /Patient/[id]/$purge - get rid of all current + historical data for a whole Patient compartment

• doing a cascading delete on resources referenced from elsewhere should fail
• It will be ok for $purge to take a rather long amount of time - even in the minutes
• deleting history is a new delete operation, called delete-history, that will prevent history output from that database value onwards. It's like a blocker on history output even if the resource would go live again. This also means that the newest delete-history entry is never garbage.

Implement Index Garbage Collection

1505

Implement Resource Store Garbage Collection

2171

Cut Off the Transaction Log

Implement Replication in Distributed Storage Mode without Transaction Log

[MM-Lehmann]

Must have (MVP): removal of a single resource by reference (id) Should have: cascading removal of all resources referencing a particular Patient(-id) Could have: cascading removal of all resource referencing any given resource ... by REST API

[MM-Lehmann]

Is there any other way to completely wipe the server via REST?

[alexanderkiel]

Is there any other way to completely wipe the server via REST?

No, you have to shutdown Blaze, delete the docker volume and restart it.

[JohannesOehm]

Are there any update on this? This prevents us currently from using Blaze in our DIC.

[alexanderkiel]

@JohannesOehm Would it be sufficient to be able to purge a single resource with all of it's history? That would be "Instance-Level Expunge" in HAPI. Would it be ok if metadata about the transactions that created/updated/deleted the resource will still exist but the resource contents are purged from disk?

[MM-Lehmann]

That would be sufficient for us

Am 19.10.2022 um 14:28, Alexander Kiel @.***> schrieb:

@JohannesOehm Would it be sufficient to be able to purge a single resource with all of it's history? That would be "Instance-Level Expunge" in HAPI. Would it be ok if metadata about the transactions that created/updated/deleted the resource will still exist but the resource contents are purged from disk?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

[JohannesOehm]

It would be better if the resources ID is also deleted, but we can replace the resource IDs, which currently hold the patients pseudonym with some random numbers, so it is also fine for us.