search

CBIIT / cds-model

Model description files for the Cancer Data Service (draft)
4 stars 4 forks source link

CDS Model out of sync with Metadata Manifest Template #5

Closed bensonml closed 1 year ago

bensonml commented 1 year ago

Currently the CDS data model is out of sync with the Metadata Manifest template.

This (can) causes problems with data loading of cds metadata manifests into Bento CDS (via CBIIT/cds-etl, and CBIIT/icdc-dataloader)

  1. CDS Model has a branch add-program that was used for cds-etl by MLB to add a "Program" node (note that the metadata manifest does not include this node and should not explicitly have this node! -- perhaps the properties/fields could be 'captured' under Study?)
  2. CDS Model currently does not have all the same properties that exist as fields in the CDS Metadata Manifest Template v.1.3.1
  3. The CDS Metadata Manifest has been / is being updated to contain additional fields and have fields such as gender/sex as being optional
  4. Need to start creating releases (tags) to model the versioning in the CDS Metadata Manifest Template
  5. Consider updating/edit certain properties as being "nullable"
  6. (outside of the repo) inform the CDS TPM, team about need to keep CDS data model in sync with CDS metadata manifest template
  7. create tests to verify the metadata manifest template is in sync with model
  8. automate tests (github actions/jenkins, etc)
  9. improve documentation for CDS data model and metadata manifest for future maintainers
  10. sync the data dictionary value sets with property value sets
  11. may need to explore a way to have backwards compatibility of cds data model with the cds metadata manifest v1.2 format -- this may be out of scope of this issue/repo but clearly impacts how cds-etl transforms the data using the cds data model into the Neo4j loadable output .tsv files
CollectorsCorner commented 1 year ago

This has been resolved.