monarch-initiative / dipper

Data Ingestion Pipeline for Monarch
https://dipper.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
57 stars 26 forks source link

Streamline dipper and Unify OBAN association modeling #408

Open cmungall opened 7 years ago

cmungall commented 7 years ago

(this description may be iteratively refined)

Currently we follow the OBAN association model (essentially RDF reification) extending it with source-specific information. Cypher queries translate chains of associations into denormalized triples for loading into Solr. The overall process is not very transparent.

We have configuration and tests distributed across

Proposal

Preliminary definitions:

We will have a master location (in dipper repo?) with one YAML file per category pair. This YAML file will compile down to

TomConlin commented 7 years ago

I approve of this message