search

HumanCellAtlas / data-store

Design specs and prototypes for the HCA Data Storage System (DSS, "blue box")
https://dss.staging.data.humancellatlas.org/
Other
40 stars 6 forks source link

Create Service Level Agreement for redaction notifications #2312

Open kozbo opened 5 years ago

kozbo commented 5 years ago

Propose the following: (note all days are measured in business days)

  1. Day 0: data creator requests redaction
  2. By day 2: the data is hidden from access (tombstone marker placed) in the data store
  3. By day 4: data is at least hidden in all participating modules of the DCP
  4. By day 12: data is permanently removed from data store
  5. By day 14: data is permanently removed from all other modules (if not already)

@xbrianh @DailyDreaming Please review straw man ^^^

DailyDreaming commented 5 years ago

@kozbo I'd approve but feel I have no context for what a reasonable timeline is. I feel these things are best decided by the PMs or whomever gives legal advice to the organization.

justincc commented 5 years ago

Yeah, I think this needs to go to legal and compliance. Even removal can mean a lot of things - can it be left unlinked in a filesystem or does it need to be literally nuked? Does it need to disappear from all backups? This needs formal policy.

hannes-ucsc commented 5 years ago

Copying my comment from the Epic:

If the "data" (I assume that means DSS blobs, files and bundles) is immediately removed physically, Azul/DataBrowser cannot satisfy the SLA (assuming that the SLA requires or will require Azul/DataBrowser to also immediately remove any references to the data). If the data is first tombstoned and then physically removed, Azul would be able to cleanly remove the data from view within minutes. This is because Azul needs the metadata from a deleted bundle in order to reliably remove it from its index.