search

HumanCellAtlas / dcp

Data Coordination Platform manifest and integration tests.
3 stars 1 forks source link

Data Storage Service Data Integrity #74

Open rhiananthony opened 5 years ago

rhiananthony commented 5 years ago

We absolutely cannot lose data. What can we put in place to prevent and protect against this?

Matt Weiden could potentially champion this.

rhiananthony commented 5 years ago

Kylee - Mar 23, 2018 - 3:49PM @mattweiden1 That's great! Thank you. When you have the roadmap let me know and I can work to make sure all services get a ticket about it in their backlog.

Matt - Mar 23, 2018 - 11:59PM @kyleedegatano I've clarified this with Andrey. Since BB is the system of record, Data Integrity for DCP largely means covering this checklist for BB. We are working on an InfoSec roadmap for pre-release review, which will make suggestions general to all services (e.g. end to end checksumming).

Matt - Mar 22, 2018 - 11:51AM Flesh out data integrity checklist here: https://docs.google.com/document/d/1lhi0A7Bi6jTSAECrGuN4s4V3clkZXNtuNGsyTxvlzYM/edit

Matt - Mar 9, 2018 - 12:38PM @sampierson3 I think "Measures to prevent and protect" was left over from the ticket sketch. Deleting this now. I will "Double-Click" with Andrey when I return from vacation.

Sam - Mar9, 2018 - 11:55AM This is a good start. Some of these are actionable, however some are not, e.g. what does "measures to prevent and protect" mean? We need to double-click on all of these until we get to actionable items.

Matt - Mar 8, 2018 - 11:38AM @sampierson3 I've added the concrete actions he recommended earlier.

Sam - Mar 8, 2018 - 10:57AM Matt please can you work with Andrey to turn his thoughts into a set of actions, which we can then work with to see which things should be in this to-do list.

sampierson commented 5 years ago

I don't believe this is "In Progress". It hasn't been touched in 7 months. Ice-boxing.