Create data processing/deidentification manifest in advance of data challenge

[jshoughtaling]

The manifest should describe any/all modifications to the data during the ingestion and transformation processes. It would be intended to help analysts and challenge participants understand the extent to which the data has been modified prior to release.

Broadly, it could include the following:

Deidentification Descriptions

As a first approximation, we will apply a set of typical transformations used in other consortia that aggregate sensitive observational health data. These transformations include:

• Date shifting of all dates
  • Logic requires official definition and signoff
• Removal of source value fields and any other fields that can contain freetext information
• Removal of any sensitive concepts related to HIV
• Grouping of ages for those patients older than 90 (relative to date of delivery)
• Obfuscation of zip codes for regions with small populations
  • Note that this is not currently relevant because sites have not submitted LOCATION information

Privacy Checks

Following these processes, the privacy_scan_tool should be executed to confirm the absence of PHI... This tool does the following...

Others?

[heschmidt04]

Mentioning for discussion and context this CDS standard mentioned at April Bridge2AI event for the ideas behind the manifest.tsv

• Reference https://cds-specification.readthedocs.io/en/stable/specification/datatype-directory-structure/
• And this example from another team on how they laid out their common dataset directory structure using CDS using a JSON structure