monarch-initiative / monarch-ingest

Data ingest application for Monarch Initiative knowledge graph using Koza
https://monarchinitiative.org
14 stars 1 forks source link

Get output from modular ingests before merge in Jenkins job #593

Open glass-ships opened 3 months ago

glass-ships commented 3 months ago

This PR adds a script to download the output of modular ingests from their releases (listed in a dictionary with the format "ingest-name": ["output_file_nodes", "output_file_edges"] as applies to your ingest)'

As well as a step in the ingest-all Jenkinsfile to run this script right before the merge step.

A test run of this updated Jenkins job can be found here (Note: this run failed because some modular ingests still have original_object as a column in their output, which causes cat-merge to fail)

@kevinschaper we should probably get @amc-corey-cox @madanucd and @AO33 access to Jenkins at some point.

Important Notes:

glass-ships commented 3 months ago

Possibly redundant, feel free to scrap